Symmetric matrices
Properties of real symmetric matrices
I
Recall that a matrix A 2 R
nn
is symmetric if A
T
= A.
I
For real symmetric matrices we have the following two
crucial properties:
I
All eigenvalues of a real symmetric matrix are real.
I
Eigenvectors corresponding to distinct eigenvalues are
orthogonal.
I
To show these two properties,we need to consider
complex matrices of type A 2 C
nn
,where C is the set of
complex numbers z = x +iy where x and y are the real
and imaginary part of z and i =
p
1.
I
C
n
is the set of ncolumn vectors with components in C
and similarly C
nn
is the set of n n matrices with complex
numbers as its entries.
I
We write the complex conjugate of z as z
= x iy.For
u 2 C
n
and A 2 C
nn
,we denote by u
2 C
n
and
A
2 C
nn
,their complex conjugates,obtained by taking
the complex conjugate of each of their components.
Properties of real symmetric matrices
I
We write the complex conjugate of z as z
= x iy.
I
For u 2 C
n
,we denote by u
2 C
n
its complex conjugate,
obtained by taking the complex conjugate of each of its
components,i.e.,(u
)
i
= (u
i
)
.
I
Similarly,for A 2 C
nn
,we denote by A
2 C
nn
,the
complex conjugate of A,obtained by taking the complex
conjugate of each of its entries,i.e.,(A
)
ij
= (A
ij
)
.
I
Note that for complex numbers we have z
1
= z
2
iff z
1
= z
2
.
I
This property clearly extends to complex vectors and
matrices:
I
For u;v 2 C
n
we have u = v iff u
= v
and for
A;B 2 C
nn
,we have A = B iff A
= B
.
I
Furthermore,(Au)
= A
u
and (A
)
T
= (A
T
)
.
Eigenvalues of a symmetric real matrix are real
I
Let 2 C be an eigenvalue of a symmetric A 2 R
nn
and
let u 2 C
n
be a corresponding eigenvector:
Au = u:(1)
I
Taking complex conjugates of both sides of (1),we obtain:
A
u
=
u
;i.e.,Au
=
u
:(2)
I
Now,we premultiply (1) with (u
)
T
to obtain:
(u
)
T
u = (u
)
T
(Au) = ((u
)
T
A)u
= (A
T
u
)
T
u since (Bv)
T
= v
T
B
T
= (Au
)
T
u since A
T
= A
= (
u
)
T
u =
(u
)
T
u:using ( 2)
I
Thus,(
)(u
)
T
u = 0.
I
But u,being an eigenvector is nonzero and
(u
)
T
u =
P
n
i =1
u
i
u
i
> 0 since at least one of the
components of u is nonzero and for any complex number
z = a +ib we have z
z = a
2
+b
2
0.
I
Hence =
,i.e., and hence u are both real.
I
Eigenvectors of distinct eigenvalues of a symmetric
real matrix are orthogonal
I
Let A be a real symmetric matrix.
I
Let Au
1
=
1
u
1
and Au
2
=
2
u
2
with u
1
and u
2
nonzero
vectors in R
n
and
1
;
2
2 R.
I
Premultiplying both sides of the ﬁrst equation above with
u
T
2
,we get:
u
T
2
u
1
= u
T
2
(Au
1
) = (u
T
2
A)u
1
= (A
T
u
2
)
T
u
1
= (Au
2
)
T
u
1
=
2
u
T
2
u
1
:
I
Thus,(
1
2
)u
T
2
u
1
= 0.
I
Therefore,
1
6=
2
implies:u
T
2
u
1
= 0 as required.
I
If an eigenvalue has multiplicity m say then we can
always ﬁnd a set of m orthonormal eigenvectors for .
I
We conclude that by normalizing the eigenvectors of A,we
get an orthonormal set of vectors u
1
;u
2
;:::;u
n
.
Properties of positive deﬁnite symmetric matrices
I
Suppose A 2 R
n
is a symmetric positive deﬁnite matrix,
i.e.,A = A
T
and
8x 2 R
n
n f0g:x
T
Ax > 0:(3)
I
Then we can easily show the following properties of A.
I
All diagonal elements are positive:In (3),put x with x
j
= 1
for j = i and x
j
= 0 for j 6= i,to get A
ii
> 0.
I
The largest element in magnitude in the entire matrix
occurs in the diagonal:Fix i 6= j between 1 and n.In (3),
put x with x
k
= 1 for k = i,x
k
= 1 for k = j and x
k
= 0 for
j 6= k 6= i,to get jA
ij
j < max(A
ii
;A
jj
).
I
All leading principle minors (i.e.,the 1 1,2 2,3 3,
:::;mm matrices in the upper left corner) are positive
deﬁnite:In (3),put x with x
k
= 0 for k > m to prove that the
top left mm matrix is positive deﬁnite.
Spectral decomposition
I
We have seen in the previous pages and in lecture notes
that if A 2 R
nn
is a symmetric matrix then it has an
orthonormal set of eigenvectors u
1
;u
2
;:::;u
n
corresponding to (not necessarily distinct) eigenvalues
1
;
2
;:::;
n
,then we have:
I
The spectral decomposition:Q
T
AQ = where
I
Q = [u
1
;u
2
;:::;u
n
] is an orthogonal matrix with Q
1
= Q
T
and = diag(
1
;
2
;:::;
n
) is diagonal.
I
Assume A 2 R
n
reprsents the linear map f:R
n
!R
n
in
the standard basis of R
n
.
I
Then,the matrix S:= Q
1
2 R
nn
is the matrix for the
change of basis into one in which f is reprsented by
B:= .More generally,B = SAS
1
:
old coordinates
S
A
//
old coordinates
S
new coordinates
B
//
new coordinates
Singular value decomposition (SVD) I
I
Let A 2 R
mn
be an arbitrary matrix.
I
Then A
T
A 2 R
nn
and AA
T
2 R
mm
are symmetric
matrices.
I
They are also positive semideﬁnite since for example
x
T
A
T
Ax = (Ax)
T
(Ax) = (kAxk
2
)
2
0.
I
We will show that A = USV
T
,called the SVD of A,where
V 2 R
nn
and U 2 R
mm
are orthogonal matrices whereas
the matrix S = U
T
AV 2 R
mn
is diagonal with
S = diag(
1
;
2
;
3
;:::;
p
) where p = min(m;n) and the
nonnegative numbers
1
2
3
:::
p
0 are the
singular values of A.
I
If r is the rank of A then A has exactly r positive singular
values
1
2
3
:::
r
> 0 with
r +1
=
r +2
=:::=
p
= 0.
Singular value decomposition II
I
Note that if the SVD for A as above exists then,since
U
T
U = I
m
,we have A
T
A = VS
T
U
T
USV
T
= VS
T
SV
T
,
where S
T
S = diag(
2
1
;
2
2
;
2
3
;:::;
2
p
) 2 R
nn
is a diagonal
matrix,thus giving the spectral decomposition of the
positive semideﬁnite matrix A
T
A.
I
This gives us a method to ﬁnd the SVD of A.
I
Obtain the eigenvalues
2
1
2
2
2
3
:::
2
p
0 and
the corresponding eigenvectors v
1
; ;v
p
of A
T
A.If p < n,
the other eigenvalues of A are zero with corresponding
eigenvectors v
p+1
; ;v
n
which make the orthogonal
matrix V = [v
1
; ;v
n
].
I
From the SVD we have AV = US,thus when
i
> 0,i.e.,
for 1 i r,we get
1
i
Av
i
= u
i
.
I
Extend the set u
1
; ;u
r
to an orthonormal basis
u
1
; ;u
r
; u
m
of R
m
which gives the orthogonal matrix
U = [u
1
; ;u
m
].
Singular value decomposition III
I
Note the following useful facts.
I
For 1 i r,the vector u
i
is an eigenvector of AA
T
with
eigenvalue
2
i
.Check!
I
AA
T
is similar to SS
T
(with identical eigenvalues) and A
T
A
is similar to S
T
S (with identical eigenvalues).
I
The diagonal elements of the diagonal matrices
S
T
S 2 R
nn
and SS
T
2 R
mm
are
2
1
;
2
2
;
2
3
;:::;
2
p
followed by n p zeros and mp zeros respectively.
I
The singular values of A are the positive square roots of
the eigenvalues of AA
T
or A
T
A.
Enter the password to open this PDF file:
File name:

File size:

Title:

Author:

Subject:

Keywords:

Creation Date:

Modification Date:

Creator:

PDF Producer:

PDF Version:

Page Count:

Preparing document for printing…
0%
Comments 0
Log in to post a comment