Symmetric matrices

johnnepaleseΗλεκτρονική - Συσκευές

10 Οκτ 2013 (πριν από 3 χρόνια και 8 μήνες)

88 εμφανίσεις

Symmetric matrices
Properties of real symmetric matrices
I
Recall that a matrix A 2 R
nn
is symmetric if A
T
= A.
I
For real symmetric matrices we have the following two
crucial properties:
I
All eigenvalues of a real symmetric matrix are real.
I
Eigenvectors corresponding to distinct eigenvalues are
orthogonal.
I
To show these two properties,we need to consider
complex matrices of type A 2 C
nn
,where C is the set of
complex numbers z = x +iy where x and y are the real
and imaginary part of z and i =
p
1.
I
C
n
is the set of n-column vectors with components in C
and similarly C
nn
is the set of n n matrices with complex
numbers as its entries.
I
We write the complex conjugate of z as z

= x iy.For
u 2 C
n
and A 2 C
nn
,we denote by u

2 C
n
and
A

2 C
nn
,their complex conjugates,obtained by taking
the complex conjugate of each of their components.
Properties of real symmetric matrices
I
We write the complex conjugate of z as z

= x iy.
I
For u 2 C
n
,we denote by u

2 C
n
its complex conjugate,
obtained by taking the complex conjugate of each of its
components,i.e.,(u

)
i
= (u
i
)

.
I
Similarly,for A 2 C
nn
,we denote by A

2 C
nn
,the
complex conjugate of A,obtained by taking the complex
conjugate of each of its entries,i.e.,(A

)
ij
= (A
ij
)

.
I
Note that for complex numbers we have z
1
= z
2
iff z

1
= z

2
.
I
This property clearly extends to complex vectors and
matrices:
I
For u;v 2 C
n
we have u = v iff u

= v

and for
A;B 2 C
nn
,we have A = B iff A

= B

.
I
Furthermore,(Au)

= A

u

and (A

)
T
= (A
T
)

.
Eigenvalues of a symmetric real matrix are real
I
Let  2 C be an eigenvalue of a symmetric A 2 R
nn
and
let u 2 C
n
be a corresponding eigenvector:
Au = u:(1)
I
Taking complex conjugates of both sides of (1),we obtain:
A

u

= 

u

;i.e.,Au

= 

u

:(2)
I
Now,we pre-multiply (1) with (u

)
T
to obtain:
(u

)
T
u = (u

)
T
(Au) = ((u

)
T
A)u
= (A
T
u

)
T
u since (Bv)
T
= v
T
B
T
= (Au

)
T
u since A
T
= A
= (

u

)
T
u = 

(u

)
T
u:using ( 2)
I
Thus,( 

)(u

)
T
u = 0.
I
But u,being an eigenvector is non-zero and
(u

)
T
u =
P
n
i =1
u

i
u
i
> 0 since at least one of the
components of u is non-zero and for any complex number
z = a +ib we have z

z = a
2
+b
2
 0.
I
Hence  = 

,i.e., and hence u are both real.
I
Eigenvectors of distinct eigenvalues of a symmetric
real matrix are orthogonal
I
Let A be a real symmetric matrix.
I
Let Au
1
= 
1
u
1
and Au
2
= 
2
u
2
with u
1
and u
2
non-zero
vectors in R
n
and 
1
;
2
2 R.
I
Pre-multiplying both sides of the first equation above with
u
T
2
,we get:
u
T
2
u
1
= u
T
2
(Au
1
) = (u
T
2
A)u
1
= (A
T
u
2
)
T
u
1
= (Au
2
)
T
u
1
= 
2
u
T
2
u
1
:
I
Thus,(
1

2
)u
T
2
u
1
= 0.
I
Therefore,
1
6= 
2
implies:u
T
2
u
1
= 0 as required.
I
If an eigenvalue  has multiplicity m say then we can
always find a set of m orthonormal eigenvectors for .
I
We conclude that by normalizing the eigenvectors of A,we
get an orthonormal set of vectors u
1
;u
2
;:::;u
n
.
Properties of positive definite symmetric matrices
I
Suppose A 2 R
n
is a symmetric positive definite matrix,
i.e.,A = A
T
and
8x 2 R
n
n f0g:x
T
Ax > 0:(3)
I
Then we can easily show the following properties of A.
I
All diagonal elements are positive:In (3),put x with x
j
= 1
for j = i and x
j
= 0 for j 6= i,to get A
ii
> 0.
I
The largest element in magnitude in the entire matrix
occurs in the diagonal:Fix i 6= j between 1 and n.In (3),
put x with x
k
= 1 for k = i,x
k
= 1 for k = j and x
k
= 0 for
j 6= k 6= i,to get jA
ij
j < max(A
ii
;A
jj
).
I
All leading principle minors (i.e.,the 1 1,2 2,3 3,
:::;mm matrices in the upper left corner) are positive
definite:In (3),put x with x
k
= 0 for k > m to prove that the
top left mm matrix is positive definite.
Spectral decomposition
I
We have seen in the previous pages and in lecture notes
that if A 2 R
nn
is a symmetric matrix then it has an
orthonormal set of eigenvectors u
1
;u
2
;:::;u
n
corresponding to (not necessarily distinct) eigenvalues

1
;
2
;:::;
n
,then we have:
I
The spectral decomposition:Q
T
AQ =  where
I
Q = [u
1
;u
2
;:::;u
n
] is an orthogonal matrix with Q
1
= Q
T
and  = diag(
1
;
2
;:::;
n
) is diagonal.
I
Assume A 2 R
n
reprsents the linear map f:R
n
!R
n
in
the standard basis of R
n
.
I
Then,the matrix S:= Q
1
2 R
nn
is the matrix for the
change of basis into one in which f is reprsented by
B:= .More generally,B = SAS
1
:
old coordinates
S

A
//
old coordinates
S

new coordinates
B
//
new coordinates
Singular value decomposition (SVD) I
I
Let A 2 R
mn
be an arbitrary matrix.
I
Then A
T
A 2 R
nn
and AA
T
2 R
mm
are symmetric
matrices.
I
They are also positive semi-definite since for example
x
T
A
T
Ax = (Ax)
T
(Ax) = (kAxk
2
)
2
 0.
I
We will show that A = USV
T
,called the SVD of A,where
V 2 R
nn
and U 2 R
mm
are orthogonal matrices whereas
the matrix S = U
T
AV 2 R
mn
is diagonal with
S = diag(
1
;
2
;
3
;:::;
p
) where p = min(m;n) and the
non-negative numbers 
1
 
2
 
3
::: 
p
 0 are the
singular values of A.
I
If r is the rank of A then A has exactly r positive singular
values 
1
 
2
 
3
::: 
r
> 0 with

r +1
= 
r +2
=:::= 
p
= 0.
Singular value decomposition II
I
Note that if the SVD for A as above exists then,since
U
T
U = I
m
,we have A
T
A = VS
T
U
T
USV
T
= VS
T
SV
T
,
where S
T
S = diag(
2
1
;
2
2
;
2
3
;:::;
2
p
) 2 R
nn
is a diagonal
matrix,thus giving the spectral decomposition of the
positive semi-definite matrix A
T
A.
I
This gives us a method to find the SVD of A.
I
Obtain the eigenvalues 
2
1
 
2
2
 
2
3
::: 
2
p
 0 and
the corresponding eigenvectors v
1
;  ;v
p
of A
T
A.If p < n,
the other eigenvalues of A are zero with corresponding
eigenvectors v
p+1
;  ;v
n
which make the orthogonal
matrix V = [v
1
;  ;v
n
].
I
From the SVD we have AV = US,thus when 
i
> 0,i.e.,
for 1  i  r,we get
1

i
Av
i
= u
i
.
I
Extend the set u
1
;  ;u
r
to an orthonormal basis
u
1
;  ;u
r
;   u
m
of R
m
which gives the orthogonal matrix
U = [u
1
;  ;u
m
].
Singular value decomposition III
I
Note the following useful facts.
I
For 1  i  r,the vector u
i
is an eigenvector of AA
T
with
eigenvalue 
2
i
.Check!
I
AA
T
is similar to SS
T
(with identical eigenvalues) and A
T
A
is similar to S
T
S (with identical eigenvalues).
I
The diagonal elements of the diagonal matrices
S
T
S 2 R
nn
and SS
T
2 R
mm
are 
2
1
;
2
2
;
2
3
;:::;
2
p
followed by n p zeros and mp zeros respectively.
I
The singular values of A are the positive square roots of
the eigenvalues of AA
T
or A
T
A.