The most common matrices we meet in applications are , that is, they are square matrices which are equal to their transposes. In symbols, = .

johnnepaleseElectronics - Devices

Oct 10, 2013 (3 years and 10 months ago)

97 views

CHAPTER III
APPLICATIONS
1.Real Symmetric Matrices
The most common matrices we meet in applications are symmetric,that is,they
are square matrices which are equal to their transposes.In symbols,A
t
= A.
Examples.
·
1 2
2 2
¸
;
2
4
1 ¡1 0
¡1 0 2
0 2 3
3
5
are symmetric,but
2
4
1 2 2
0 1 3
0 0 4
3
5
;
2
4
0 1 ¡1
¡1 0 2
1 ¡2 0
3
5
are not.
Symmetric matrices are in many ways much simpler to deal with than general
matrices.
First,as we noted previously,it is not generally true that the roots of the char-
acteristic equation of a matrix are necessarily real numbers,even if the matrix has
only real entries.However,
if A is a symmetric matrix with real entries,then the roots of its charac-
teristic equation are all real.
Example 1.The characteristic equations of
·
0 1
1 0
¸
and
·
0 ¡1
1 0
¸
are
¸
2
¡1 = 0 and ¸
2
+1 = 0
respectively.Notice the dramatic e®ect of a simple change of sign.
The reason for the reality of the roots (for a real symmetric matrix) is a bit
subtle,and we will come back to it later sections.
The second important property of real symmetric matrices is that they are always
diagonalizable,that is,there is always a basis for R
n
consisting of eigenvectors for
the matrix.
111
112 III.APPLICATIONS
Example 2.We previously found a basis for R
2
consisting of eigenvectors for
the 2 £2 symmetric matrix
A =
·
2 1
1 2
¸
The eigenvalues are ¸
1
= 3;¸
2
= 1,and the basis of eigenvectors is
½
v
1
=
·
1
1
¸
;v
2
=
·
¡1
1
¸¾
:
If you look carefully,you will note that the vectors v
1
and v
2
not only form a basis,
but they are perpendicular to one another,i.e.,v
1
¢ v
2
= 1(¡1) +1(1) = 0.
The perpendicularity of the eigenvectors is no accident.It is always the case for
a symmetric matrix by the following reasoning.
First,recall that the dot product of two column vectors u and v in R
n
can be
written as a row by column product
u ¢ v = u
t
v = [ u
1
u
2
:::u
n
]
2
6
6
4
v
1
v
2
.
.
.
v
n
3
7
7
5
=
n
X
i=1
u
i
v
i
:
Suppose now that Au = ¸u and Av = ¹v,i.e.,u and v are eigenvectors for A with
corresponding eigenvalues ¸ and ¹.Assume ¸ 6
= ¹.Then
(1) u ¢ (Av) = u
t
(Av) = u
t
(¹v) = ¹(u
t
v) = ¹(u ¢ v):
On the other hand,
(2) (Au) ¢ v = (Au)
t
v = (¸u)
t
v = ¸(u
t
v) = ¸(u ¢ v):
However,since the matrix is symmetric,A
t
= A,and
(Au)
t
v = (u
t
A
t
)v = (u
t
A)v = u
t
(Av):
The ¯rst of these expressions is what was calculated in (2) and the last was calcu-
lated in (1),so the two are equal,i.e.,
¹(u ¢ v) = ¸(u ¢ v):
If u ¢ v 6
= 0,we can cancel the common factor to conclude that ¹ = ¸,which is
contrary to our assumption,so it must be true that u ¢ v = 0,i.e.,u?v.
We summarize this as follows.
Eigenvectors for a real symmetric matrix which belong to di®erent eigen-
values are necessarily perpendicular.
This fact has important consequences.Assume ¯rst that the eigenvalues of A
are distinct and that it is real and symmetric.Then not only is there a basis
consisting of eigenvectors,but the basis elements are also mutually perpendicular.
2.REAL SYMMETRIC MATRICES 113
This is reminiscent of the familiar situation in R
2
and R
3
,where coordinate axes
are almost always assumed to be mutually perpendicular.For arbitrary matrices,
we may have to face the prospect of using`skew'axes,but the above remark tells
us we can avoid this possibility in the symmetric case.
In two or three dimensions,we usually require our basis vectors to be unit vectors.
There is no problemwith that here.Namely,if u is not a unit vector,we can always
obtain a unit vector by dividing u by its length juj.Moreover,if u is an eigenvector
for A with eigenvalue ¸,then any nonzero multiple of u is also such an eigenvector,
in particular,the unit vector
1
juj
u is.
Example 2,revisited.The eigenvectors v
1
and v
2
both have length
p
2.So
we replace them by the corresponding unit vectors
1
p
2
v
1
=
"
1
p
2
1
p
2
#
1
p
2
v
2
=
"
¡
1
p
2
1
p
2
#
which also constitute a basis for R
2
.
There is some special terminology which is commonly used in linear algebra for
the familiar concepts discussed above.Two vectors are said to be orthogonal if
they are perpendicular.A unit vector is said to be normalized.The idea is that if
we started with a non-unit vector,we would produce an equivalent unit vector by
dividing it by its length.The latter process is called normalization.Finally,a basis
for R
n
consisting of mutually perpendicular unit vectors is called an orthonormal
basis.
Exercises for Section 1.
1.(a) Find a basis of eigenvectors for A =
·
¡3 4
4 3
¸
.
(b) Check that the basis vectors are orthogonal,and normalize them to yield an
orthonormal basis.
2.(a) Find a basis of eigenvectors for A =
·
¡3 2
8 3
¸
.
(b) Are the basis vectors orthogonal to one another?If not what might be the
problem?
3.(a) Find a basis of eigenvectors for A =
2
4
1 0 1
0 1 0
1 0 1
3
5
.
(b) Check that the basis vectors are orthogonal,and normalize them to yield an
orthonormal basis.
4.Let A =
2
4
1 4 3
4 1 0
3 0 1
3
5
.Find an orthonormal basis of eigenvectors.
5.Let A be a symmetric n£n matrix,and let P be any n£n matrix.Show that
P
t
AP is also symmetric.
114 III.APPLICATIONS
2.Repeated Eigenvalues,The Gram{Schmidt Process
We now consider the case in which one or more eigenvalues of a real symmetric
matrix A is a repeated root of the characteristic equation.It turns out that we can
still ¯nd an orthonormal basis of eigenvectors,but it is a bit more complicated.
Example 1.Consider
A =
2
4
¡1 1 1
1 ¡1 1
1 1 ¡1
3
5
:
The characteristic equation is
det
2
4
¡1 ¡¸ 1 1
1 ¡1 ¡¸ 1
1 1 ¡1 ¡¸
3
5
= ¡(1 +¸)((1 +¸)
2
¡1) ¡1(¡1 ¡¸ ¡1) +1(1 +1 +¸)
= ¡(1 +¸)(¸
2
+2¸) +2(¸ +2)
= ¡(¸
3
+3¸
2
¡4) = 0:
Using the method suggested in Chapter 2,we may ¯nd the roots of this equation by
trying the factors of the constant term.The roots are ¸ = 1,which has multiplicity
1,and ¸ = ¡2,which has multiplicity 2.
For ¸ = 1,we need to reduce
A¡I =
2
4
¡2 1 1
1 ¡2 1
1 1 ¡2
3
5
!
2
4
1 1 ¡2
0 ¡3 3
0 3 ¡3
3
5
!
2
4
1 0 ¡1
0 1 ¡1
0 0 0
3
5
:
The general solution is v
1
= v
3
;v
2
= v
3
with v
3
free.A basic eigenvector is
v
1
=
2
4
1
1
1
3
5
but we should normalize this by dividing it by jv
1
j =
p
3.This gives
u
1
=
1
p
3
2
4
1
1
1
3
5
:
For ¸ = ¡2,the situation is more complicated.Reduce
A+2I =
2
4
1 1 1
1 1 1
1 1 1
3
5
!
2
4
1 1 1
0 0 0
0 0 0
3
5
2.REPEATED EIGENVALUES,THE GRAM{{SCHMIDT PROCESS 115
which yields the general solution v
1
= ¡v
2
¡v
3
with v
2
;v
3
free.This gives basic
eigenvectors
v
2
=
2
4
¡1
1
0
3
5
;v
3
=
2
4
¡1
0
1
3
5
:
Note that,as the general theory predicts,v
1
is perpendicular to both v
2
and v
3
.
(The eigenvalues are di®erent).Unfortunately,v
2
and v
3
are not perpendicular to
each other.However,with a little e®ort,this is easy to remedy.All we have to do
is pick another basis for the subspace spanned by fv
2
;v
3
g.The eigenvectors with
eigenvalue ¡2 are exactly the non-zero vectors in this subspace,so any basis will
do as well.Hence,we arrange to pick a basis consisting of mutually perpendicular
vectors.
It is easy to construct the new basis.Indeed we need only replace one of the two
vectors.Keep v
2
,and let v
0
3
= v
3
¡cv
2
where c is chosen so that
v
2
¢ v
0
3
= v
2
¢ v
3
¡cv
2
¢ v
2
= 0;
i.e.,take c =
v
2
¢ v
3
v
2
¢ v
2
.(See the diagram to get some idea of the geometry behind
this calculation.)
v
3
2
3
v
v
We have
v
2
¢ v
3
v
2
¢ v
2
=
1
2
v
0
3
= v
3
¡
1
2
v
2
=
2
4
¡1
0
1
3
5
¡
1
2
2
4
¡1
1
0
3
5
=
2
4
¡
1
2
¡
1
2
1
3
5
:
We should also normalize this basis by choosing
u
2
=
1
jv
2
j
v
2
=
1
p
2
2
4
¡1
1
0
3
5
;u
3
=
1
jv
0
3
j
v
0
3
=
r
2
3
2
4
¡
1
2
¡
1
2
1
3
5
:
Putting this all together,we see that
u
1
=
1
p
3
2
4
1
1
1
3
5
;u
2
=
1
p
2
2
4
¡1
1
0
3
5
;u
3
=
r
2
3
2
4
¡
1
2
¡
1
2
1
3
5
form an orthonormal basis for R
3
consisting of eigenvectors for A.
116 III.APPLICATIONS
The Gram{Schmidt Process.In Example 1,we used a special case of a more
general algorithm in order to construct an orthonormal basis of eigenvectors.The
algorithm,called the Gram{Schmidt Process works as follows.Suppose
fv
1
;v
2
;:::;v
k
g
is a linearly independent set spanning a certain subspace W of R
n
.We construct
an orthonormal basis for W as follows.Let
v
0
1
= v
1
v
0
2
= v
2
¡
v
2
¢ v
0
1
v
0
1
¢ v
0
1
v
0
1
v
0
3
= v
3
¡
v
3
¢ v
0
1
v
0
1
¢ v
0
1
v
0
1
¡
v
3
¢ v
0
2
v
0
2
¢ v
0
2
v
0
2
.
.
.
v
0
k
= v
k
¡
k¡1
X
j=1
v
k
¢ v
0
j
v
0
j
¢ v
0
j
v
0
j
:
It is not hard to see that each new v
0
k
is perpendicular to those constructed
before it.For example,
v
0
1
¢ v
0
3
= v
0
1
¢ v
3
¡
v
3
¢ v
0
1
v
0
1
¢ v
0
1
v
0
1
¢ v
0
1
¡
v
3
¢ v
0
2
v
0
2
¢ v
0
2
v
0
1
¢ v
0
2
:
However,we may suppose that we already know that v
0
1
¢ v
0
2
= 0 (from the previous
stage of the construction),so the above becomes
v
0
1
¢ v
0
3
= v
0
1
¢ v
3
¡v
3
¢ v
0
1
= 0:
The same argument works at each stage.
It is also not hard to see that at each stage,replacing v
j
by v
0
j
in
fv
0
1
;v
0
2
;:::;v
0
j¡1
;v
j
g
does not change the subspace spanned by the set.Hence,for j = k,we conclude
that fv
0
1
;v
0
2
;:::;v
0
k
g is a basis for W consisting of mutually perpendicular vectors.
Finally,to complete the process simply divide each v
0
j
by its length
u
j
=
1
jv
0
j
j
v
0
j
:
Then fu
1
;:::;u
k
g is an orthonormal basis for W.
2.REPEATED EIGENVALUES,THE GRAM{{SCHMIDT PROCESS 117
Example 2.Consider the subspace of R
4
spanned by
v
1
=
2
6
4
¡1
1
0
1
3
7
5
;v
2
=
2
6
4
¡1
1
1
0
3
7
5
;v
3
=
2
6
4
1
0
0
1
3
7
5
:
Then
v
0
1
=
2
6
4
¡1
1
0
1
3
7
5
v
0
2
=
2
6
4
¡1
1
1
0
3
7
5
¡
2
3
2
6
4
¡1
1
0
1
3
7
5
=
2
6
6
4
¡
1
3
1
3
1
¡
2
3
3
7
7
5
v
0
3
=
2
6
4
1
0
0
1
3
7
5
¡
0
3
2
6
4
¡1
1
0
1
3
7
5
¡
¡1
15
9
2
6
6
4
¡
1
3
1
3
1
¡
2
3
3
7
7
5
=
2
6
6
4
4
5
1
5
3
5
3
5
3
7
7
5
:
Normalizing,we get
u
1
=
1
p
3
2
6
4
¡1
1
0
1
3
7
5
u
2
=
3
p
15
2
6
6
4
¡
1
3
1
3
1
¡
2
3
3
7
7
5
=
1
p
15
2
6
4
¡1
1
3
¡2
3
7
5
u
3
=
5
p
35
2
6
6
4
4
5
1
5
3
5
3
5
3
7
7
5
=
1
p
35
2
6
4
4
1
3
3
3
7
5
:
The Principal Axis Theorem.The Principal Axis Theorem asserts that
the process outlined above for ¯nding mutually perpendicular eigenvectors always
works.
If A is a real symmetric n£n matrix,there is always an orthonormal basis
for R
n
consisting of eigenvectors for A.
Here is a summary of the method.If the roots of the characteristic equation are
all di®erent,then all we need to do is ¯nd an eigenvector for each eigenvalue and if
necessary normalize it by dividing by its length.If there are repeated roots,then
it will usually be necessary to apply the Gram{Schmidt process to the set of basic
eigenvectors obtained for each repeated eigenvalue.
118 III.APPLICATIONS
Exercises for Section 2.
1.Apply the Gram{Schmidt Process to each of the following sets of vectors.
(a)
8
<
:
2
4
1
0
1
3
5
;
2
4
2
1
0
3
5
9
=
;
(b)
8
>
<
>
:
2
6
4
1
0
2
0
3
7
5
;
2
6
4
1
1
0
1
3
7
5
;
2
6
4
0
2
1
¡1
3
7
5
9
>
=
>
;
.
2.Find an orthonormal basis of eigenvectors for A =
2
4
1 1 1
1 1 1
1 1 1
3
5
.
3.Find an orthonormal basis of eigenvectors for
A =
2
4
¡1 2 2
2 ¡1 2
2 2 ¡1
3
5
:
Hint:3 is an eigenvalue.
4.Let fv
1
;v
2
;v
3
g be a linearly independent set.Suppose fv
0
1
;v
0
2
;v
0
3
g is the set
obtained (before normalizing) by the Gram-Schmidt Process.(a) Explain why v
0
2
is not zero.(b) Explain why v
0
3
is not zero.
The generalization of this to an arbitrary linearly independent set is one rea-
son the Gram-Schmidt Process works.The vectors produced by that process are
mutually perpendicular provided they are non-zero,and so they form a linearly
independent set.Since they are in the subspace W spanned by the original set of
vectors and there are just enough of them,they must form a basis a basis for W.
3.Change of Coordinates
As we have noted previously,it is probably a good idea to use a special basis like
an orthonormal basis of eigenvectors.Any problem associated with the matrix A
is likely to take a particularly simple form when expressed relative to such a basis.
To study this in greater detail,we need to talk a bit more about changes of
coordinates.Although the theory is quite general,we shall concentrate on some
simple examples.
In R
n
,the entries in a column vector x may be thought of as the coordinates
x
1
;x
2
;:::;x
n
of the vector with respect to the standard basis.To simplify the
algebra,let's concentrate on one speci¯c n,say n = 3.In that case,we may make
the usual identi¯cations e
1
= i;e
2
= j;e
3
= k for the elements of the standard
basis.Suppose fv
1
;v
2
;v
3
g is another basis.The coordinates of x with respect to
the new basis|call them x
0
1
;x
0
2
;x
0
3
|are de¯ned by the relation
(1) x = v
1
x
0
1
+v
2
x
0
2
+v
3
x
0
3
= [ v
1
v
2
v
3
]
2
4
x
0
1
x
0
2
x
0
3
3
5
= [ v
1
v
2
v
3
] x
0
:
3.CHANGE OF COORDINATES 119
One way to viewthis relation is as a systemof equations in which the old coordinates
x =
2
4
x
1
x
2
x
3
3
5
are given,and we want to solve for the new coordinates
x
0
=
2
4
x
0
1
x
0
2
x
0
3
3
5
:
The coe±cient matrix of this system
P = [ v
1
v
2
v
3
]
is called the change of basis matrix.It's columns are the old coordinates of the new
basis vectors.
The relation (1) may be rewritten
(2) x = Px
0
and it may also be interpreted as expressing the`old'coordinates of a vector in
terms of its`new'coordinates.This seems backwards,but it is easy to turn it
around.Since the columns of P are linearly independent,P is invertible and we
may write instead
(3) x
0
= P
¡1
x
where we express the`new'coordinates in terms of the`old'coordinates.
Example 1.Suppose in R
2
we pick a new set of coordinate axes by rotating
each of the old axes through angle µ in the counterclockwise direction.Call the
old coordinates (x
1
;x
2
) and the new coordinates (x
0
1
;x
0
2
).According to the above
discussion,the columns of the change of basis matrix P come from the old coordi-
nates of the new basis vectors,i.e.,of unit vectors along the new axes.From the
diagram,these are
·
cos µ
sinµ
¸ ·
¡sinµ
cos µ
¸
:
120 III.APPLICATIONS
x
1
x'
1
x
2
x'
2


Hence,
·
x
1
x
2
¸
=
·
cos µ ¡sinµ
sinµ cos µ
¸·
x
0
1
x
0
2
¸
:
The change of basis matrix is easy to invert in this case.(Use the special rule which
applies to 2 £2 matrices.)
·
cos µ ¡sinµ
sinµ cos µ
¸
¡1
=
1
cos
2
µ +sin
2
µ
·
cos µ sinµ
¡sinµ cos µ
¸
=
·
cos µ sinµ
¡sinµ cos µ
¸
(You could also have obtained this by using the matrix for rotation through angle
¡µ.) Hence,we may express the`new'coordinates in terms of the`old'coordinates
through the relation
·
x
0
1
x
0
2
¸
=
·
cos µ sinµ
¡sinµ cos µ
¸·
x
1
x
2
¸
:
For example,suppose µ = ¼=6.The new coordinates of the point with original
coordinates (2;6) are given by
·
x
0
1
x
0
2
¸
=
·
p
3=2 1=2
¡1=2
p
3=2
¸·
2
6
¸
=
·
p
3 +3
¡1 +3
p
3
¸
:
So with respect to the rotated axes,the coordinates are (3 +
p
3;3
p
3 ¡1).
Orthogonal Matrices.You may have noticed that the matrix P obtained in
Example 1 has the property P
¡1
= P
t
.This is no accident.It is a consequence of
the fact that its columns are mutually perpendicular unit vectors.Indeed,
The columns of an n £n matrix form an orthonormal basis for R
n
if and
only if its inverse is its transpose.
An n £n real matrix with this property is called orthogonal.
3.CHANGE OF COORDINATES 121
Example 2.Let
P =
·
3
5
¡
4
5
4
5
3
5
¸
:
The columns of P are
u
1
=
·
3
5
4
5
¸
;u
2
=
·
¡
4
5
3
5
¸
;
and it is easy to check that these are mutually perpendicular unit vectors in R
2
.
To see that P
¡1
= P
t
,it su±ces to show that
P
t
P =
·
3
5
4
5
¡
4
5
3
5
¸·
3
5
¡
4
5
4
5
3
5
¸
=
·
1 0
0 1
¸
:
Of course,it is easy to see that this true by direct calculation,but it may be more
informative to write it out as follows
P
t
P =
·
(u
1
)
t
(u
2
)
t
¸
[ u
1
u
2
] =
·
u
1
¢ u
1
u
1
¢ u
2
u
2
¢ u
1
u
2
¢ u
2
¸
where the entries in the product are exhibited as row by column dot products.The
o® diagonal entries are zero because the vectors are perpendicular,and the diagonal
entries are ones because the vectors are unit vectors.
The argument for n£n matrices is exactly the same except that there are more
entries.
Note.The terminology is very confusing.The de¯nition of an orthogonal matrix
requires that the columns be mutually perpendicular and also that they be unit
vectors.Unfortunately,the terminology reminds us of the former condition but not
of the latter condition.It would have been better if such matrices had been named
`orthonormal'matrices rather than`orthogonal'matrices,but that is not how it
happened,and we don't have the option of changing the terminology at this late
date.
The Principal Axis Theorem Again.As we have seen,given a real sym-
metric n £n matrix A,the Principal Axis Theorem assures us that we can always
¯nd an orthonormal basis fv
1
;v
2
;:::;v
n
g for R
n
consisting of eigenvectors for A.
Let
P = [ v
1
v
2
:::v
n
]
122 III.APPLICATIONS
be the corresponding change of basis matrix.As in Chapter II,Section 5,we have
Av
1
= v
1
¸
1
= [ v
1
v
2
:::v
n
]
2
6
6
4
¸
1
0
.
.
.
0
3
7
7
5
Av
2
= v
2
¸
2
= [ v
1
v
2
:::v
n
]
2
6
6
4
0
¸
2
.
.
.
0
3
7
7
5
.
.
.
Av
n
= v
n
¸
n
= [ v
1
v
2
:::v
n
]
2
6
6
4
0
0
.
.
.
¸
n
3
7
7
5
where some eigenvalues ¸
j
for di®erent eigenvectors might be repeated.These
equations can be written in a single matrix equation
A[ v
1
v
2
:::v
n
] = [ v
1
v
2
:::v
n
]
2
6
6
4
¸
1
0:::0
0 ¸
2
:::0
.
.
.
.
.
.:::
.
.
.
0 0:::¸
n
3
7
7
5
or
AP = PD
where D is a diagonal matrix with eigenvalues (possibly repeated) on the diagonal.
This may also be written
(4) P
¡1
AP = D:
Since we have insisted that the basic eigenvectors form an orthonormal basis,the
change of basis matrix P is orthogonal,and we have P
¡1
= P
t
.Hence,(4) can be
written in the alternate form
(5) P
t
AP = D with P orthogonal:
Example 3.Let A =
·
¡7 14
14 7
¸
.The characteristic equation of A turns out
to be ¸
2
¡ 625 = 0,so the eigenvalues are ¸ = §25.Calculation shows that an
orthonormal basis of eigenvectors is formed by
u
1
=
·
3
5
4
5
¸
for ¸ = 25 and u
2
=
·
¡
4
5
3
5
¸
for ¸ = ¡25:
Hence,we may take P to be the orthogonal matrix
·
3
5
¡
4
5
4
5
3
5
¸
:
The reader should check in this case that
P
t
AP =
·
3
5
4
5
¡
4
5
3
5
¸
:
·
¡7 14
14 7
¸·
3
5
¡
4
5
4
5
3
5
¸
=
·
25 0
0 ¡25
¸
:
3.CHANGE OF COORDINATES 123
Appendix.A Proof of the Principal Axis Theorem.
The following section outlines how the Principal Axis Theorem is proved for the
very few of you who may insist on seeing it.It is not necessary for what follows.
In view of the previous discussions,we can establish the Principal Axis Theorem
by showing that there is an orthogonal n £n matrix P such that
(6) AP = PD or equivalently P
t
AP = D
where D is a diagonal matrix with the eigenvalues of A (possibly repeated) on its
diagonal.
The method is to proceed by induction on n.
If n = 1 there really isn't anything to prove.(Take P = [ 1 ].) Suppose the
theorem has been proved for (n ¡1) £(n ¡1) matrices.Let u
1
be a unit eigen-
vector for A with eigenvalue ¸
1
.Consider the subspace W consisting of all vectors
perpendicular to u
1
.It is not hard to see that W is an n¡1 dimensional subspace.
Choose (by the Gram{Schmidt Process) an orthonormal basis fw
2
;w
2
:::;w
n
g for
W.Then fu
1
;w
2
;:::;w
n
g is an orthonormal basis for R
n
,and
Au
1
= u
1
¸
1
= [ u
1
w
2
:::w
n
]
|
{z
}
P
1
2
6
6
4
¸
1
0
.
.
.
0
3
7
7
5
:
This gives the ¯rst column of AP
1
,and we want to say something about its remain-
ing columns
Aw
2
;Aw
2
;:::;Aw
n
:
To this end,note that if w is any vector in W,then Aw is also a vector in W.For,
we have
u
1
¢ (Aw) = (u
1
)
t
Aw = (u
1
)
t
A
t
w = (Au
1
)
t
w = ¸
1
(u
1
)
t
w) = ¸
1
(u
1
¢ w) = 0;
which is to say,Aw is perpendicular to u
1
if w is perpendicular to u
1
.It follows
that each Aw
j
is a linear combination just of w
2
;w
3
;:::;w
n
,i.e.,
Aw
j
= [ u
1
w
2
:::w
n
]
2
6
6
4
0
¤
.
.
.
¤
3
7
7
5
where`¤'denotes some unspeci¯ed entry.Putting this all together,we see that
AP
1
= P
1
2
6
6
4
¸
1
0:::0
0
.
.
.A
0
0
3
7
7
5
|
{z
}
A
1
124 III.APPLICATIONS
where A
0
is an (n ¡1) £(n ¡1) matrix.P
1
is orthogonal (since its columns form
an orthonormal basis) so
P
1
t
AP
1
= A
1
;
and it is not hard to derive from this the fact that A
1
is symmetric.Because of
the structure of A
1
,this implies that A
0
is symmetric.Hence,by induction we may
assume there is an (n ¡1) £(n ¡1) orthogonal matrix P
0
such that A
0
P
0
= P
0
D
0
with D
0
diagonal.It follows that
A
1
2
6
6
4
1 0:::0
0
.
.
.P
0
0
3
7
7
5
|
{z
}
P
2
=
2
6
6
4
¸
1
0:::0
0
.
.
.A
0
0
3
7
7
5
2
6
6
4
1 0:::0
0
.
.
.P
0
0
3
7
7
5
=
2
6
6
4
¸
1
0:::0
0
.
.
.A
0
P
0
0
3
7
7
5
=
2
6
6
4
¸
1
0:::0
0
.
.
.P
0
D
0
0
3
7
7
5
=
2
6
6
4
1 0:::0
0
.
.
.P
0
0
3
7
7
5
|
{z
}
P
2
2
6
6
4
¸
1
0:::0
0
.
.
.D
0
0
3
7
7
5
|
{z
}
D
= P
2
D:
Note that P
2
is orthogonal and D is diagonal.Thus,
AP
1
P
2
|
{z
}
P
= P
1
A
1
P
2
= P
1
P
2
|
{z
}
P
D
or AP = PD:
However,a product of orthogonal matrices is orthogonal|see the Exercises|so P
is orthogonal as required.
This completes the proof.
There is one subtle point involved in the above proof.We have to know that
a real symmetric n £n matrix has at least one real eigenvalue.This follows from
the fact,alluded to earlier,that the roots of the characteristic equation for such a
matrix are necessarily real.Since the equation does have a root,that root is the
desired eigenvalue.
Exercises for Section 3.
1.Find the change of basis matrix for a rotation through (a) 30 degrees in the
counterclockwise direction and (b) 30 degrees in the clockwise direction
2.Let P(µ) be the matrix for rotation of axes through µ.Show that P(¡µ) =
P(µ)
t
= P(µ)
¡1
.
4.CLASSIFICATION OF CONICS AND QUADRICS 125
3.An inclined plane makes an angle of 30 degrees with the horizontal.Change
to a coordinate system with x
0
1
axis parallel to the inclined plane and x
0
2
axis
perpendicular to it.Use the change of variables formula derived in the section
to ¯nd the components of the gravitational acceleration vector ¡gj in the new
coordinate system.Compare this with what you would get by direct geometric
reasoning.
4.Let A =
·
1 2
2 1
¸
.Find a 2£2 orthogonal matrix P such that P
t
AP is diagonal.
What are the diagonal entries?
5.Let A =
2
4
1 4 3
4 1 0
3 0 1
3
5
.Find a 3 £ 3 orthogonal matrix P such that P
t
AP is
diagonal.What are the diagonal entries?
6.Show that the product of two orthogonal matrices is orthogonal.How about
the inverse of an orthogonal matrix?
7.The columns of an orthogonal matrix are mutually perpendicular unit vectors.
Is the same thing true of the rows?Explain.
4.Classi¯cation of Conics and Quadrics
The Principal Axis Theorem derives its name from its relation to classifying
conics,quadric surfaces,and their higher dimensional analogues.
The general quadratic equation
ax
2
+bxy +cy
2
+dx +ey = f
(with enough of the coe±cients non-zero) de¯nes a curve in the plane.Such a curve
is generally an ellipse,a hyperbola,a parabola,all of which are called conics,or
two lines,which is considered a degenerate case.(See the Appendix to this section
for a review.)
Examples.
x
2
+
y
2
4
= 1
x
2
¡y
2
= 1
x
2
¡2xy +2y
2
= 1
x
2
+2xy ¡y
2
+3x ¡5y = 10
If the linear terms are not present (d = e = 0 and f 6
= 0),we call the curve a
central conic.It turns out to be an ellipse or hyperbola (but its axes of symmetry
may not be the coordinate axes) or a pair of lines in the degenerate case.Parabolas
can't be obtained this way.
126 III.APPLICATIONS
In this section,we shall show how to use linear algebra to classify such central
conics and higher dimensional analogues such as quadric surfaces in R
3
.Once you
understand the central case,it is fairly easy to reduce the general case to that.
(You just use completion of squares to get rid of the linear terms in the same way
that you identify a circle with center not at the origin from its equation.)
In order to apply the linear algebra we have studied,we adopt a more systematic
notation,using subscripted variables x
1
;x
2
instead of x;y.
Consider the central conic de¯ned by the equation
f(x) = a
11
x
1
2
+2a
12
x
1
x
2
+a
22
x
2
2
= C
(The reason for the 2 will be clear shortly.)
It is more useful to express the function f as follows.
f(x) = (x
1
a
11
+x
2
a
21
)x
1
+(x
1
a
12
+x
2
a
22
)x
2
= x
1
(a
11
x
1
+a
12
x
2
) +x
2
(a
21
x
1
+a
22
x
2
);
where we have introduced a
21
= a
12
.The above expression may also be written in
matrix form
f(x) =
2
X
j;k=1
x
j
a
jk
x
k
= x
t
Ax
where A is the symmetric matrix of coe±cients.
Note what has happened to the coe±cients.The coe±cients of the squares
appear on the diagonal of A,while the coe±cient of the cross term2bx
1
x
2
is divided
into two equal parts.Half of it appears as b in the 1;2 position (corresponding to the
product x
1
x
2
) while the other half appears as b in the 2;1 position (corresponding
to the product x
2
x
1
which of course equals x
1
x
2
).So it is clear why the matrix is
symmetric.
This may be generalized to n > 2 in a rather obvious manner.Let A be a real
symmetric n £n matrix,and de¯ne
f(x) =
n
X
j;k=1
x
j
a
jk
x
k
= x
t
Ax:
For n = 3 this may be written explicitly
f(x) = (x
1
a
11
+x
2
a
21
+x
3
a
31
)x
1
+(x
1
a
12
+x
2
a
22
+x
3
a
32
)x
2
+(x
1
a
13
+x
2
a
23
+x
3
a
33
)x
3
= a
11
x
1
2
+a
22
x
2
2
+a
33
x
3
2
+2a
12
x
1
x
2
+2a
13
x
1
x
3
+2a
23
x
2
x
3
:
The rule for forming the matrix A from the equation for f(x) is the same as
in the 2 £ 2 case.The coe±cients of the squares are put on the diagonal.The
4.CLASSIFICATION OF CONICS AND QUADRICS 127
coe±cient of a cross term involving x
i
x
j
is split in half,with one half put in the
i;j position,and the other half is put in the j;i position.
The level set de¯ned by
f(x) = C
is called a central hyperquadric.It should be visualized as an n ¡ 1 dimensional
curved object in R
n
.For n = 3 it will be an ellipsoid or a hyperboloid (of one or
two sheets) or perhaps a degenerate`quadric'like a cone.(As in the case of conics,
we must also allow linear terms to encompass paraboloids.)
If the above descriptions are accurate,we should expect the locus of the equation
f(x) = C to have certain axes of symmetry which we shall call its principal axes.It
turns out that these axes are determined by an orthonormal basis of eigenvectors for
the coe±cient matrix A.To see this,suppose fu
1
;u
2
;:::;u
n
g is such a basis and
P = [ u
1
u
2
:::u
n
] is the corresponding orthogonal matrix.By the Principal
Axis Theorem,P
t
AP = D is diagonal with the eigenvalues,¸
1

2
;:::;¸
n
,of A
appearing on the diagonal.Make the change of coordinates x = Px
0
where x
represents the`old'coordinates and x
0
represents the`new'coordinates.Then
f(x) = x
t
Ax = (Px
0
)
t
A(Px
0
) = (x
0
)
t
P
t
APx
0
= (x
0
)
t
Dx
0
:
Since D is diagonal,the quadratic expression on the right has no cross terms,i.e.
(x
0
)
t
Dx
0
= [ x
0
1
x
0
2
¢ ¢ ¢ x
0
n
]
2
6
6
4
¸
1
0 ¢ ¢ ¢ 0
0 ¸
2
¢ ¢ ¢ 0
.
.
.
.
.
.¢ ¢ ¢
.
.
.
0 0 ¢ ¢ ¢ ¸
n
3
7
7
5
2
6
6
4
x
0
1
x
0
2
.
.
.
x
0
n
3
7
7
5
= ¸
1
(x
0
1
)
2

2
(x
0
2
)
2
+¢ ¢ ¢ +¸
n
(x
0
n
)
2
:
In the new coordinates,the equation takes the form
¸
1
(x
0
1
)
2

2
(x
0
2
)
2
+¢ ¢ ¢ +¸
n
(x
0
n
)
2
= C
and its graph is usually quite easy to describe.
Example 1.We shall investigate the conic f(x;y) = x
2
+4xy +y
2
= 1.First
rewrite the equation
[ x y ]
·
1 2
2 1
¸·
x
y
¸
= 1:
(Note how the 4 was split into two symmetrically placed 2s.) Next,¯nd the eigen-
values of the coe±cient matrix by solving
det
·
1 ¡¸ 2
2 1 ¡¸
¸
= (1 ¡¸)
2
¡4 = ¸
2
¡2¸ ¡3 = 0:
This equation is easy to factor,and the roots are ¸ = 3;¸ = ¡1.
For ¸ = 3,to ¯nd the eigenvectors,we need to solve
·
¡2 2
2 ¡2
¸·
v
1
v
2
¸
= 0:
128 III.APPLICATIONS
Reduction of the coe±cient matrix yields
·
¡2 2
2 ¡2
¸
!
·
1 ¡1
0 0
¸
with the general solution v
1
= v
2
,v
2
free.A basic normalized eigenvector is
u
1
=
1
p
2
·
1
1
¸
:
For ¸ = ¡1,a similar calculation (which you should make) yields the basic
normalized eigenvector
u
2
=
1
p
2
·
¡1
1
¸
:
(Note that u
1
?u
2
as expected.)
From this we can form the corresponding orthogonal matrix P and make the
change of coordinates
·
x
y
¸
= P
·
x
0
y
0
¸
;
and,according to the above analysis,the equation of the conic in the newcoordinate
system is
3(x
0
)
2
¡(y
0
)
2
= 1:
It is clear that this is a hyperbola with principal axes pointing along the new axes.
x
x
y
y
Example 2.Consider the quadric surface de¯ned by
x
1
2
+x
2
2
+x
3
2
¡2x
1
x
3
= 1:
We take
f(x) = x
1
2
+x
2
2
+x
3
2
¡2x
1
x
3
= [ x
1
x
2
x
3
]
2
4
1 0 ¡1
0 1 0
¡1 0 1
3
5
2
4
x
1
x
2
x
3
3
5
:
4.CLASSIFICATION OF CONICS AND QUADRICS 129
Note how the coe±cient in ¡2x
1
x
3
was split into two equal parts,a ¡1 in the
1;3-position and a ¡1 in the 3;1-position.The coe±cients of the other cross terms
were zero.As usual,the coe±cients of the squares were put on the diagonal.
The characteristic equation of the coe±cient matrix is
det
2
4
1 ¡¸ 0 ¡1
0 1 ¡¸ 0
¡1 0 1 ¡¸
3
5
= (1 ¡¸)
3
¡(1 ¡¸) = ¡(¸ ¡2)(¸ ¡1)¸ = 0
Thus,the eigenvalues are ¸ = 2;1;0.
For ¸ = 2,reduce
2
4
¡1 0 ¡1
0 ¡1 0
¡1 0 ¡1
3
5
!
2
4
1 0 1
0 1 0
0 0 0
3
5
to obtain v
1
= ¡v
3
;v
2
= 0 with v
3
free.Thus,
v
1
=
2
4
¡1
0
1
3
5
is a basic eigenvector for ¸ = 2,and
u
1
=
1
p
2
2
4
¡1
0
1
3
5
is a basic unit eigenvector.
Similarly,for ¸ = 1 reduce
2
4
0 0 ¡1
0 0 0
¡1 0 0
3
5
!
2
4
1 0 0
0 0 1
0 0 0
3
5
which yields v
1
= v
3
= 0 with v
2
free.Thus a basic unit eigenvector for ¸ = 1 is
u
2
=
2
4
0
1
0
3
5
:
Finally,for ¸ = 0,reduce
2
4
1 0 ¡1
0 1 0
¡1 0 1
3
5
!
2
4
1 0 ¡1
0 1 0
0 0 0
3
5
:
This yields v
1
= x
3
;v
2
= 0 with v
3
free.Thus,a basic unit eigenvector for ¸ = 0 is
u
3
=
1
p
2
2
4
1
0
1
3
5
:
130 III.APPLICATIONS
The corresponding orthogonal change of basis matrix is
P = [ u
1
u
2
u
3
] =
2
4
¡
1
p
2
0
1
p
2
0 1 0
1
p
2
0
1
p
2
3
5
:
Moreover,putting x = Px
0
,we can express the equation of the quadric surface in
the new coordinate system
(1) 2x
0
1
2
+1x
0
2
2
+0x
0
3
2
= 2x
0
1
2
+x
0
2
2
= 1:
Thus it is easy to see what this quadric surface is:an elliptical cylinder perpendic-
ular to the x
0
1
;x
0
2
plane.(This is one of the degenerate cases.) The three`principal
axes'in this case are the two axes of the ellipse in the x
0
1
;x
0
2
plane and the x
0
3
axis,
which is the central axis of the cylinder.
1
2
3
new axes. The new axes are
labelled.
Tilted cylinder relative to the
Representing the graph in the new coordinates makes it easy to understand its
geometry.Suppose,for example,that we want to ¯nd the points on the graph
which are closest to the origin.These are the points at which the x
0
1
-axis intersects
the surface.These are the points with new coordinates x
0
1
= §
1
p
2
;x
0
2
= x
0
3
= 0.If
you want the coordinates of these points in the original coordinate system,use the
change of coordinates formula
x = Px
0
:
Thus,the old coordinates of the minimum point with new coordinates (1=
p
2;0;0)
are given by
2
4
¡
1
p
2
0
1
p
2
0 1 0
1
p
2
0
1
p
2
3
5
2
4
1
p
2
0
0
3
5
=
2
4
¡
1
2
0
1
2
3
5
:
4.CLASSIFICATION OF CONICS AND QUADRICS 131
Appendix.A review of conics and quadrics.
You are probably familiar with certain graphs when they arise in standard con-
¯gurations.
In two dimensions,the central conics have equations of the form
§
x
2
a
2
§
y
2
b
2
= 1:
If both signs are +,the conic is an ellipse.If one sign is + and one is ¡,then the
conic is a hyperbola.The + goes with the axis which crosses the hyperbola.Some
examples are sketched below.
Ellipse Hyperbola
Two ¡ signs result in an empty graph,i.e.,there are no points satisfying the
equation.
Parabolas arise from equations of the form y = px
2
with p 6
= 0.
Parabolas
For x = py
2
,the parabola opens along the positive or negative y-axis.
There are also some degenerate cases.For example,
x
2
a
2
¡
y
2
b
2
= 0 de¯nes two lines
which intersect at the origin.
In three dimensions,the central quadrics have equations of the form
§
x
2
a
2
§
y
2
b
2
§
z
2
c
2
= 1:
132 III.APPLICATIONS
If all three signs are +,the quadric is an ellipsoid.If two of the three signs are +
and one is ¡,the quadric is a hyperboloid of one sheet.If one of the two signs is
+ and the other two are ¡,the quadric is a hyperboloid of two sheets.Notice that
the number of sheets is the same as the number of ¡ signs.It is not hard to ¯gure
out how the quadric is oriented,depending on how the signs are arranged.The
`axis'of a hyperboloid is labeled by the variable whose sign in the equation is in
the minority,i.e.,the ¡ sign in the one sheet case and the + sign in the two sheet
case.
Hyperboloid of one sheet
Ellipsoid
Hyperboloid of two sheets
If all three signs are ¡,we get an empty graph.
Paraboloids arise from equations of the form
z = §
x
2
a
2
§
y
2
b
2
;
or similar equations with x;y;z rearranged.If both signs are + or both are ¡,then
the quadric is an elliptic paraboloid or`bowl'.The bowl opens up along the axis
of the variable appearing on the left of the equation if the signs are + and it opens
along the negative axis of that variable if the signs are ¡.If one sign is + and the
other is ¡,the surface is a hyperbolic paraboloid or`saddle'.Equations of the form
z = cxy;c 6
= 0 also describe saddles.
Elliptic paraboloid
Hyperbolic paraboloid
There are many degenerate cases.One example would be
x
2
a
2
+
y
2
b
2
¡
z
2
c
2
= 0:
5.CLASSIFICATION OF CONICS AND QUADRICS 133
Its graph is a double cone with elliptical cross sections.Another would be
§
x
2
a
2
§
y
2
b
2
= 1
with at least one + sign.Its graph is a`cylinder'perpendicular to the x;y-plane.
The cross sections are ellipses or hyperbolas,depending on the combination of signs.
Cone
Cylinder
Exercises for Section 4.
1.Find the principal axes and classify the central conic x
2
+xy +y
2
= 1.
2.Identify the conic de¯ned by x
2
+4xy +y
2
= 4.Find its principal axes,and
¯nd the points closest and furthest (if any) from the origin.
3.Identify the conic de¯ned by 2x
2
+72xy +23y
2
= 50.Find its principal axes,
and ¯nd the points closest and furthest (if any) from the origin.
4.Find the principal axes and classify the central quadric de¯ned by
x
2
¡y
2
+z
2
¡4xy ¡4yz = 1:
5.(Optional) Classify the surface de¯ned by
x
2
+2y
2
+z
2
+2xy +2yz ¡z = 0:
Hint:This is not a central quadric.To classify it,¯rst apply the methods of the
section to the quadratic expression x
2
+2y
2
+z
2
+2xy+2yz to ¯nd a new coordinate
system in which this expression has the form ¸
1
x
02

2
y
02

3
z
02
.Use the change
of coordinates formula to express z in terms of x
0
;y
0
,and z
0
and then complete
squares to eliminate all linear terms.At this point,it should be clear what the
surface is.
134 III.APPLICATIONS
5.Conics and the Method of Lagrange Multipliers
There is another approach to ¯nding the principal axes of a conic,quadric,or
hyperquadric.Consider for an example an ellipse in R
2
centered at the origin.One
of the principal axes intersects the conic in the two points at greatest distance from
the origin,and the other intersects it in the two points at least distance from the
origin.Similarly,two of the three principal axes of a central ellipsoid in R
3
may be
obtained in this way.Thus,if we didn't know about eigenvalues and eigenvectors,
we might try to ¯nd the principal axes by maximizing (or minimizing) the function
giving the distance to the origin subject to the quadratic equation de¯ning the conic
or quadric.In other words,we need to minimize a function given a constraint among
the variables.Such problems are solved by the method of Lagrange multipliers,
which you learned in your multidimensional calculus course.
Here is a review of the method.Suppose we want to maximize (minimize) the
real valued function f(x) = f(x
1
;x
2
;:::;x
n
) subject to the constraint g(x) =
g(x
1
;x
2
;:::;x
1
) = c.For n = 2,this has a simple geometric interpretation.The
locus of the equation g(x
1
;x
2
) = c is a level curve of the function g,and we want
to maximize (minimize) the function f on that curve.Similarly,for n = 3,the level
set g(x
1
;x
2
:x
3
) = c is a surface in R
3
,and we want to maximize (minimize) f on
that surface.
g( ) = c
x
g( ) = c
x
n = 2. Level curve in the plane.
n = 3. Level surface in space.
Examples.Maximize f(x;y) = x
2
+y
2
on the ellipse g(x;y) = x
2
+4y
2
= 3.
(This is easy if you draw the picture.)
Minimize f(x;y;z) = 2x
2
+ 3xy + y
2
+ xz ¡ 4z
2
on the sphere g(x;y;z) =
x
2
+y
2
+z
2
= 1.
Minimize f(x;y;z;t) = x
2
+ y
2
+ z
2
¡ t
2
on the`hypersphere'g(x;y;z;t) =
x
2
+y
2
+z
2
+t
2
= 1.
We shall concentrate on the case of n = 3 variables,but the reasoning for any
n is similar.We want to maximize (or minimize) f(x) on a level surface g(x) = c
in R
3
,where as usual we abbreviate x = (x
1
;x
2
;x
3
).At any point x on the level
surface at which such an extreme value is obtained,we must have
(1) rf(x) = ¸rg(x)
for some scalar ¸.
5.CONICS AND THE METHOD OF LAGRANGE MULTIPLIERS 135
is parallel to g
maximum point
f
g
other point
f
(1) is a necessary condition which must hold at the relevant points.(It doesn't by
itself guarantee that there is a maximum or a minimum at the point.There could
be no extreme value at all at the point.) In deriving this condition,we assume
implicitly that the level surface is smooth and has a well de¯ned normal vector
rg 6
= 0,and that the function f is also smooth.If these conditions are violated at
some point,that point could also be a candidate for a maximum or minimum.
Taking components,we obtain 3 scalar equations for the 4 variables x
1
;x
2
;x
3
;¸.
We would not expect,even in the best of circumstances to get a unique solution
from this,but the de¯ning equation for the level surface
g(x) = c
provides a 4th equation.We still won't generally get a unique solution,but we
will usually get at most a ¯nite number of possible solutions.Each of these can
be examined further to see if f attains a maximum (or minimum) at that point in
the level set.Notice that the variable ¸ plays an auxiliary role since we really only
want the coordinates of the point x.(In some applications,¸ has some signi¯cance
beyond that.) ¸ is called a Lagrange multiplier.
The method of Lagrange multipliers often leads to a set of equations which is
di±cult to solve.However,in the case of quadratic functions f,there is a typical
pattern which emerges.
Example 1.Suppose we want to minimize the function f(x;y) = x
2
+4xy +y
2
on the circle x
2
+y
2
= 1.For this problem n = 2,and the level set is a curve.Take
g(x;y) = x
2
+y
2
.Then rf = h2x +4y;4x +2yi,rg = h2x;2yi,and rf = ¸rg
yields the equations
2x +4y = ¸(2x)
4x +2y = ¸(2y)
to which we add
x
2
+y
2
= 1:
136 III.APPLICATIONS
After canceling a common factor of 2,the ¯rst two equations may be written in
matrix form
·
1 2
2 1
¸·
x
y
¸
= ¸
·
x
y
¸
which says that
·
x
y
¸
is an eigenvector for the eigenvalue ¸,and the equation x
2
+ y
2
= 1 says it is a
unit eigenvector.You should know how to solve such problems,and we leave it to
you to make the required calculations.(See also Example 1 in the previous section
where we made these calculations in another context.) The eigenvalues are ¸ = 3
and ¸ = ¡1.For ¸ = 3,a basic unit eigenvector is
u
1
=
1
p
2
·
1
1
¸
;
and every other eigenvector is of the form cu
1
.The latter will be a unit vector
if and only jcj = 1,i.e.,c = §1.We conclude that ¸ = 3 yields two solutions of
the Lagrange multiplier problem:(1=
p
2;1=
p
2) and (¡1=
p
2;¡1=
p
2).At each of
these points f(x;y) = x
2
+4xy +y
2
= 3.
For ¸ = ¡1,we obtain the basic unit eigenvector
u
2
=
1
p
2
·
¡1
1
¸
;
and a similar analysis (which you should do) yields the two points:(1=
p
2;¡1=
p
2)
and (¡1=
p
2;1=
p
2).At each of these points f(x;y) = x
2
+4xy +y
2
= ¡1.
Max
Max
Min
Min
Hence,the function attains its maximum value at the ¯rst two points and its
minimum value at the second two.
5.CONICS AND THE METHOD OF LAGRANGE MULTIPLIERS 137
Example 2.Suppose we want to minimize the function g(x;y) = x
2
+y
2
(which
is the square of the distance to the origin) on the conic f(x;y) = x
2
+4xy +y
2
= 1.
Note that this is basically the same as the previous example except that the roles
of the two functions are reversed.The Lagrange multiplier condition rg = ¸rf is
the same as the condition rf = (1=¸)rg provided ¸ 6
= 0.(¸ 6
= 0 in this case since
otherwise rg = 0,which yields x = y = 0.However,(0;0) is not a point on the
conic.) We just solved that problem and found eigenvalues 1=¸ = 3 or 1=¸ = ¡1.
In this case,we don't need unit eigenvectors,so to avoid square roots we choose
basic eigenvectors
v
1
=
·
1
1
¸
and
·
¡1
1
¸
corresponding respectively to ¸ = 3 and ¸ = ¡1.The endpoint of v
1
does not lie
on the conic,but any other eigenvector for ¸ = 3 is of the form cv
1
,so all we need
to do is adjust c so that the point satis¯es the equation f(x;y) = x
2
+4xy+y
2
= 1.
Substituting (x;y) = (c;c) yields 6c
2
= 1 or c = §1=
p
6.Thus,we obtain the
two points (1=
p
6;1=
p
6) and (¡1=
p
6;¡1=
p
6).For ¸ = ¡1,substituting (x;y) =
(¡c;c) in the equation yields ¡2c
2
= 1 which has no solutions.
Thus,the only candidates for a minimum (or maximum) are the ¯rst pair of
points:(1=
p
6;1=
p
6) and (¡1=
p
6;¡1=
p
6).A simple calculation shows these are
both 1=
p
3 units from the origin,but without further analysis,we can't tell if this
is the maximum,the minimum,or neither.However,it is not hard to classify this
conic|see the previous section|and discover that it is a hyperbola.Hence,the
two points are minimum points.
The Rayleigh-Ritz Method.Example 1 above is typical of a certain class
of Lagrange multiplier problems.Let A be a real symmetric n £ n matrix,and
consider the problem of maximizing (minimizing) the quadratic function f(x) =
x
t
Ax subject to the constraint g(x) = jxj
2
= 1.This is called the Rayleigh{Ritz
problem.For n = 2 or n = 3,the level set jxj
2
= 1 is a circle or sphere,and for
n > 3,it is called a hypersphere.
Alternately,we could reverse the roles of the functions f and g,i.e.,we could
try to maximize (minimize) the square of the distance to the origin g(x) = jxj
2
on the level set f(x) = 1.Because the Lagrange multiplier condition in either
case asserts that the two gradients rf and rg are parallel,these two problems are
very closely related.The latter problem|¯nding the points on a conic,quadric,
or hyperquadric furthest from (closest to) the origin|is easier to visualize,but
the former problem|maximizing or minimizing the quadratic function f on the
hypersphere jxj = 1 |is easier to compute with.
Let's go about applying the Lagrange Multiplier method to the Rayleigh{Ritz
problem.The components of rg are easy:
@g
@x
i
= 2x
i
;i = 1;2;:::n:
The calculation of rf is harder.First write
f(x) =
n
X
j=1
x
j
(
n
X
k=1
a
jk
x
k
)
138 III.APPLICATIONS
and then carefully apply the product rule together with a
jk
= a
kj
.The result is
@f
@x
i
= 2
n
X
j=1
a
ij
x
j
i = 1;2;:::;n:
(Work this out explicitly in the cases n = 2 and n = 3 if you don't believe it.)
Thus,the Lagrange multiplier condition rf = ¸rg yields the equations
2
n
X
j=1
a
ij
x
j
= ¸(2x
i
) i = 1;2;:::;n
which may be rewritten in matrix form (after canceling the 2's)
(3) Ax = ¸x:
To this we must add the equation of the level set
g(x) = jxj
2
= 1:
Thus,any potential solution x is a unit eigenvector for the matrix Awith eigenvalue
¸.Note also that for such a unit eigenvector,we have
f(x) = x
t
Ax = x
t
(¸x) = ¸x
t
x = ¸jxj
2
= ¸:
Thus the eigenvalue is the extreme value of the quadratic function at the point on
the (hyper)sphere given by the unit eigenvector.
The upshot of this discussion is that for a real symmetric matrix A,the Rayleigh{
Ritz problem is equivalent to the problem of ¯nding an orthonormal basis of eigen-
vectors for A.
The Rayleigh{Ritz method may be used to show that the characteristic equa-
tion of a real symmetric matrix only has real eigenvalues.This was an issue left
unresolved in our earlier discussions.Here is an outline of the argument.The
hypersphere g(x) = jxj
2
= 1 is a closed bounded set in R
n
for any n.It follows
from a basic theorem in analysis that any continuous function,in particular the
quadratic function f(x),must attain both maximum and minimum values on the
hypersphere.Hence,the Lagrange multiplier problem always has solutions,which
by the above algebra amounts to the assertion that the real symmetric matrix A
must have at least one eigenvalue.This suggests a general procedure for showing
that all the eigenvalues are real.First ¯nd the largest eigenvalue by maximizing
the quadratic function f(x) on the set jxj
2
= 1.Let x = u
1
be the corresponding
eigenvector.Change coordinates by choosing an orthonormal basis starting with
u
1
.Then the additional basis elements will span the subspace perpendicular to u
1
and we may obtain a lower dimensional quadratic function by restricting f to that
subspace.We can now repeat the process to ¯nd the next smaller real eigenvalue.
Continuing in this way,we will obtain an orthonormal basis of eigenvectors for A
and each of the corresponding eigenvalues will be real.
6.NORMAL MODES 139
Exercises for Section 5.
1.Find the maximumand minimumvalues of the function f(x;y) = x
2
+y
2
given
the constraint x
2
+xy +y
2
= 1.
2.Find the maximumand/or minimumvalue of f(x;y;z) = x
2
¡y
2
+z
2
¡4xy¡4yz
subject to x
2
+y
2
+z
2
= 1.
3.(Optional) The derivation of the Lagrange multiplier condition rf = ¸rg
assumes that the rg 6
= 0,so there is a well de¯ned tangent`plane'at the potential
maximum or minimum point.However,a maximum or minimum could occur at a
point where rg = 0,so all such points should also be checked.(Similarly,either
f or g might fail to be smooth at a maximum or minimum point.) With these
remarks in mind,¯nd where f(x;y;z) = x
2
+ y
2
+ z
2
attains its minimum value
subject to the constraint g(x;y;z) = x
2
+y
2
¡z
2
= 0.
4.Consider as in Example 2 the problem of maximizing f(x;y) = x
2
+4xy +y
2
given the constraint x
2
+y
2
= 1.This is equivalent to maximizing F(x;y) = xy on
the circle x
2
+y
2
= 1.(Why?) Draw a diagramshowing the circle and selected level
curves F(x;y) = c of the function F.Can you see why F(x;y) attains its maximum
at (1=
p
2;1=
p
2) and (¡1=
p
2;¡1=
p
2) without using any calculus?Hint:consider
how the level curves of F intersect the circle and decide from that where F is
increasing,and where it is decreasing on the circle.
6.Normal Modes
Eigenvalues and eigenvectors are an essential tool in solving systems of linear
di®erential equations.We leave an extended treatment of this subject for a course
in di®erential equations,but it is instructive to consider an interesting class of vi-
bration problems that have many important scienti¯c and engineering applications.
We start with some elementary physics you may have encountered in a physics
class.Imagine an experiment in which a small car is placed on a track and connected
to a wall though a sti® spring.With the spring in its rest position,the car will
just sit there forever,but if the car is pulled away from the wall a small distance
and then released,it will oscillate back and forth about its rest position.If we
assume the track is so well greased that we can ignore friction,this oscillation will
in principle continue forever.
x
k
m
We want to describe this situation symbolically.Let x denote the displacement
of the car from equilibrium,and suppose the car has mass m.Hooke's Law tells
140 III.APPLICATIONS
us that there is a restoring force of the form F = ¡kx where k is a constant called
the spring constant.Newton's second law relating force and acceleration tells us
(1) m
d
2
x
dt
2
= ¡kx:
This is also commonly written
d
2
x
dt
2
+
k
m
x = 0.You may have learned how to solve
this di®erential equation in a previous course,but in this particular case,it is not
really necessary.From the physical characteristics of the solution,we can pretty
much guess what it should look like.
(2) x = Acos(!t)
where A is the amplitude of the oscillation and!is determined by the frequency
or rapidity of the oscillation.It is usually called the angular frequency and it is
related to the actual frequency f by the equation
!= 2¼f:
A is determined by the size of the initial displacement.It gives the maximum
displacement attained as the car oscillates.!however is determined by the spring
constant.To see how,just substitute (2) in (1).We get
m(¡!
2
Acos(!t)) = ¡kAcos(!t)
which after canceling common factors yields
m!
2
= k
or!=
r
k
m
:
The above discussion is a bit simpli¯ed.We could not only have initially dis-
placed the car from rest,but we could also have given it an initial shove or velocity.
In that case,the maximal displacement would be shifted in time.The way to
describe this symbolically is
x = Acos(!t +±)
where ± is called the phase shift.This complication does not change the basic
character of the problem since it is usually the fundamental vibration of the system
that we are interested in,and that turns out to be the same if we include a possible
phase shift.
We now want to generalize this to more than one mass connected by several
springs.This may seem a bit bizarre,but it is just a model for situations com-
monly met in scienti¯c applications.For example,in chemistry,one often needs
to determine the basic vibrations of a complex molecule.The molecule consists of
atoms`connected'by interatomic forces.As a ¯rst approximation,we may treat
the atoms as point masses and the forces between them as linear restoring forces
from equilibrium positions.Thus the mass-spring model may tell us something
useful about real problems.
6.NORMAL MODES 141
Example 1.Consider the the con¯guration of masses and springs indicated
below,where m is the common mass of the two particles and k is the common
spring constant of the three springs.
x
1
x
2
k
m
k
m k
Look at the ¯rst mass.When it is displaced a distance x
1
to the right from
equilibrium,it will be acted upon by two forces.Extension of the spring on the
left will pull it back with force ¡kx
1
.At the same time,the spring in the middle
will push or pull it depending on whether it is compressed or stretched.If x
2
is
the displacement of the second mass from equilibrium,the change in length of the
second spring will be x
1
¡x
2
,so the force on the ¯rst mass will be ¡k(x
1
¡x
2
).
This yields a total force of
¡kx
1
¡k(x
1
¡x
2
) = ¡2kx
1
+kx
2
:
A similar analysis works for the second mass.Thus,we obtain the system of
di®erential equations
m
d
2
x
1
dt
2
= ¡2kx
1
+kx
2
m
d
2
x
2
dt
2
= kx
1
¡2kx
2
:
The system may also be rewritten in matrix form
(3) m
d
2
x
dt
2
=
·
¡2k k
k ¡2k
¸
x where x =
·
x
1
x
2
¸
:
Note that the matrix on the right is a symmetric matrix.This is always the case
in such problems.It is an indirect consequence of Newton's third law which asserts
that the forces exerted by two masses on each other must be equal and opposite.
To solve this,we look for solutions of the form
x
1
= v
1
cos(!t)
x
2
= v
2
cos(!t)(4)
In such a solution,the two particles oscillate with the same frequency but with
possibly di®erent amplitudes v
1
and v
2
.Such a solution is called a normal mode.
General motions of the system can be quite a bit more complicated.First of all,
we have to worry about possible phase shifts.More important,we also have to
142 III.APPLICATIONS
allow for linear combinations of the normal modes in which there is a mixture
of di®erent frequencies.In this way the situation is similar to that of a musical
instrument which may produce a complex sound which can be analyzed in terms
of basic frequencies or harmonics.We leave such complications for another course.
Here we content ourselves at doing the ¯rst step,which is to ¯nd the fundamental
oscillations or normal modes.
(4) may be rewritten in matrix form
(5) x = vcos(!t)
where!and v 6
= 0 are to be determined.Then
d
2
x
dt
2
= ¡!
2
vcos(!t)
Hence,putting (5) in (3) yields
m(¡!
2
vcos(!t)) =
·
¡2k k
k ¡2k
¸
vcos(!t):
Now factor out the common scalar factor cos(!t) to obtain
¡!
2
mv =
·
¡2k k
k ¡2k
¸
v:
Note that the`amplitude'v is a vector in this case,so we cannot cancel it as we
did in the case of a single particle.The above equation may now be rewritten
·
¡2 1
1 ¡2
¸
v = ¡!
2
m
k
v:
This is a tri°e messy,but if we put abbreviate ¸ = ¡!
2
m
k
for the scalar on the
right,we can write it
·
¡2 1
1 ¡2
¸
v = ¸v:
This equation should look familiar.It says that v is an eigenvector for the
matrix on the left,and that ¸ = ¡!
2
m
k
is the corresponding eigenvalue.However,
we know how to solve such problems.First we ¯nd the eigenvalues by solving
the characteristic equation.For each eigenvalue,we can ¯nd the corresponding
frequency!from!=
r
¸m
k
.Next,for each eigenvalue,we can determine basic
eigenvectors as before.
In this example,the characteristic equation is
det
·
¡2 ¡¸ 1
1 ¡2 ¡¸
¸
= (¡2 ¡¸)
2
¡1
= ¸
2
+4¸ +4 ¡1
= ¸
2
+4¸ +3
= (¸ +1)(¸ +3) = 0:
6.NORMAL MODES 143
Hence,the roots are ¸ = ¡1 (!=
p
k=m) and ¸ = ¡3 (!=
p
3k=m).
For ¸ = ¡1 (!=
p
k=m),¯nding the eigenvectors results in reducing the matrix
·
¡2 +1 1
1 ¡2 +1
¸
=
·
¡1 1
1 ¡1
¸
!
·
1 ¡1
0 0
¸
:
Hence,the solution is v
1
= v
2
with v
2
free.A basic solution vector for the subspace
of solutions is
v
1
=
·
1
1
¸
:
The corresponding normal mode has the form
x =
·
1
1
¸
cos(
p
k=mt):
Note that x
1
(t) = x
2
(t) for all t,so the two particles move together in tandem with
the same angular frequency
p
k=m.This behavior of the particles is a consequence
of the fact that the components of the basic vector v
1
are equal.
Similarly,for ¸ = ¡3 (!=
p
3k=m),we have
·
¡2 +3 1
1 ¡2 +3
¸
=
·
1 1
1 1
¸
!
·
1 1
0 0
¸
:
The solution is v
1
= ¡v
2
with v
2
free,and a basic solution vector for the system is
v
2
=
·
¡1
1
¸
:
The corresponding normal mode is is
x =
·
¡1
1
¸
cos(
p
3k=mt):
Note that x
1
(t) = ¡x
2
(t) for all t,so the two masses move opposite to one another
with the same amplitude and angular frequency
r
3k
m
.
Note that in the above example,we could have determined the two vectors v
1
and v
2
by inspection.As noted,the ¯rst corresponds to motion in which the
particles move in tandem and the spring between them experiences no net change
in length.The second corresponds to motion in which the particles move back
and forth equal amounts in opposite directions but with the same frequency.This
would have simpli¯ed the problem quite a lot.For,if you know an eigenvector of
a matrix,it is fairly simple to ¯nd the corresponding eigenvalue,and hence the
angular frequency.In fact,it is often true that careful consideration of the physical
arrangement of the particles,with particular attention to any symmetries that may
be present,may suggest possible normal modes with little or no calculation.
144 III.APPLICATIONS
Relation to the Principal Axis Theorem.As noted above normal mode
problems typically result in systems of the form
(7)
d
2
x
dt
2
= Ax
where A is a real symmetric matrix.(In the case that all the particles have the
same mass,A =
1
m
K,where K is a symmetric matrix of`spring constants'.If the
masses are di®erent,the situation is a bit more complicated,but the problem may
still be restated in the above form.)
If P is a matrix with columns the elements of a basis of eigenvectors for A,then
we saw earlier that
AP = PD
where D is a diagonal matrix with the eigenvalues on the diagonal.Assume we
make the change of coordinates
x = Px
0
:
Then
d
2
Px
0
dt
2
= APx
0
P
d
2
x
0
dt
2
= APx
0
d
2
x
0
dt
2
= P
¡1
APx
0
= Dx
0
:
However,since Dis diagonal,this last equation may be written as n scalar equations
d
2
x
0
j
dt
2
= ¸
j
x
0
j
j = 1;2;:::;n:
In the original coordinates,the motions of the particles are`coupled'since the
motion of each particle may a®ect the motion of the other particles.In the new
coordinate system,these motions are`decoupled'.The new coordinates are called
normal coordinates.Each x
0
j
may be thought of as the displacement of one of n
¯ctitious particles,each of which oscillates independently of the others in one of n
mutually perpendicular directions.The physical signi¯cance in terms of the original
particles of each normal coordinate is a but murky,but they presumably represent
underlying structure of some importance.
Example 1,revisited.
d
2
x
dt
2
=
k
m
·
¡2 1
1 ¡2
¸
x:
A basis of eigenvectors for the coe±cient matrix is as before
½
v
1
=
·
1
1
¸
;v
2
=
·
¡1
1
¸¾
:
6.NORMAL MODES 145
If we divide the vectors by their lengths,we obtain the orthonormal basis
½
1
p
2
·
1
1
¸
;
1
p
2
·
¡1
1
¸¾
:
This in turn leads to the change of coordinates matrix
P =
"
1
p
2
¡
1
p
2
1
p
2
1
p
2
#
1
x'
2
x'
x
1
x
2

/
4

/
4
If you look carefully,you will see this represents a rotation of the original x
1
;x
2
-
axes through an angle ¼=4.However,this has nothing to do with the original ge-
ometry of the problem.x
1
and x
2
stand for displacements of two di®erent particles
along the same one dimensional axis.The x
1
;x
2
plane is a ¯ctitious con¯guration
space in which a single point represents a pair of particles.It is not absolutely
clear what a rotation of axes means for this plane,but the new normal coordinates
x
0
1
;x
0
2
obtained thereby give us a formalism in which the normal modes appear as
decoupled oscillations.
Exercises for Section 6.
1.Determine the normal modes,including frequencies and relative motions for
the system
m
d
2
x
1
dt
2
= k(x
2
¡x
1
) = ¡kx
1
+kx
2
m
d
2
x
2
dt
2
= k(x
1
¡x
2
) +k(x
3
¡x
2
) = kx
1
¡2kx
2
+kx
3
m
d
2
x
3
dt
2
= k(x
2
¡x
3
) = kx
2
¡kx
3
146 III.APPLICATIONS
x
2
x
1
k
k m
x
3
m
m
Note that since the masses are not ¯xed to any wall,one possibility is that they
will together move freely at constant velocity without oscillating.This is re°ected in
the linear algebra by one zero eigenvalue which does not actually give an oscillatory
solution.Ignore that eigenvalue and the corresponding eigenvector.
2.Determine the normal modes,including frequencies and relative motions for
the system
m
d
2
x
1
dt
2
= ¡kx
1
+k(x
2
¡x
1
) = ¡2kx
1
+kx
2
m
d
2
x
2
dt
2
= k(x
1
¡x
2
) +k(x
3
¡x
2
) = kx
1
¡2kx
2
+kx
3
m
d
2
x
3
dt
2
= k(x
2
¡x
3
) ¡kx
3
= kx
2
¡2kx
3
x
2
x
1
k
k
k m
x
3
m
m
k
3.Suppose a normal mode problem involving two particles has one normal mode
in which the displacements satisfy x
1
= 2x
2
for all time.What relation do the
displacements have for the other normal mode?
4.A system of two particles is similar to the example in the text except that one
end is free.It is described by the system
d
2
x
dt
2
=
k
m
·
¡5 2
2 ¡2
¸
x where x =
·
x
1
x
2
¸
:
Find the normal modes.
5.A system of two particles is as in the example in the text except that one end
is free.It is described by the system
d
2
x
dt
2
=
k
m
·
¡4 2
2 ¡2
¸
x where x =
·
x
1
x
2
¸
:
Find the normal modes.
7.REVIEW 147
6.Acertain molecule has three normal modes.One is degenerate and corresponds
to the eigenvalue ¸ = 0.The eigenvector for this degenerate mode is
2
4
1
1
1
3
5
.The
relative motions for another normal mode satisfy x
1
= x
3
;x
2
= ¡2x
3
.What
relations do the relative motions for the third normal mode satisfy?
7.Review
Exercises for Section 7.
1.The Gram-Schmidt process fails when applied to the set of vectors
8
>
<
>
:
2
6
4
1
2
1
3
3
7
5
;
2
6
4
2
3
1
5
3
7
5
;
2
6
4
3
5
2
8
3
7
5
9
>
=
>
;
in R
4
.Explain why.
2.Let A =
2
4
0 1 1
1 0 1
1 1 0
3
5
.
(a) Find the eigenvalues and eigenvectors of A.
(b) Find an orthonormal basis for R
3
consisting of eigenvectors for A.
(c) Find an orthogonal matrix P such that P
t
AP is diagonal.What is P
t
AP?
3.What is wrong with the following statement?If the columns of an n£n matrix
P are mutually perpendicular,then P is orthogonal.
4.Consider the matrix A =
·
2 ¡2
¡2 5
¸
which has eigenvalues ¸ = 6;1.
(a) Find the eigenvectors of A.
(b) Consider the conic section 2x
2
¡4xy +5y
2
= 24:Find an orthogonal matrix
P such that the coordinate change
·
x
y
¸
= P
·
u
v
¸
transforms the equation of the
conic into the form®u
2
+¯v
2
= 24 (that is,into an equation with zero cross term).
(c) Sketch the conic section of part (b).Include in the same sketch the xy axes
and the uv axes.
5.Use the methods introduced in this course to sketch the graph of the equation
2x
2
+y
2
+z
2
+4yz = 6:
6.A system of two particles with displacements x
1
and x
2
satis¯es the system of
di®erential equations
m
d
2
x
1
dt
2
= ¡3kx
1
+2kx
2
m
d
2
x
2
dt
2
= 2kx
1
¡3kx
2
148 III.APPLICATIONS
Find the normal modes.Include the`angular frequencies'!and the initial dis-
placements (u
1
;u
2
) for each normal mode.
7.Determine whether or not each of the following matrices may be diagonalized.
In each case,explain your answer.Using general principles may help you avoid
di±cult computations.
(a) A =
2
4
1 0 0
1 1 0
1 1 1
3
5
.
(b) B =
2
4
3 ¡1 1
0 2 0
1 ¡1 3
3
5
.Note:The characteristic polynomial of B is ¡(¸ ¡
2)
2
(¸ ¡4).
(c) C =
2
6
6
6
4
1 2 1 1 1
2 0 1 0 1
1 1 0 4 0
1 0 4 0 5
1 1 0 5 0
3
7
7
7
5
.