Symmetric matrices

Properties of real symmetric matrices

I

Recall that a matrix A 2 R

nn

is symmetric if A

T

= A.

I

For real symmetric matrices we have the following two

crucial properties:

I

All eigenvalues of a real symmetric matrix are real.

I

Eigenvectors corresponding to distinct eigenvalues are

orthogonal.

I

To show these two properties,we need to consider

complex matrices of type A 2 C

nn

,where C is the set of

complex numbers z = x +iy where x and y are the real

and imaginary part of z and i =

p

1.

I

C

n

is the set of n-column vectors with components in C

and similarly C

nn

is the set of n n matrices with complex

numbers as its entries.

I

We write the complex conjugate of z as z

= x iy.For

u 2 C

n

and A 2 C

nn

,we denote by u

2 C

n

and

A

2 C

nn

,their complex conjugates,obtained by taking

the complex conjugate of each of their components.

Properties of real symmetric matrices

I

We write the complex conjugate of z as z

= x iy.

I

For u 2 C

n

,we denote by u

2 C

n

its complex conjugate,

obtained by taking the complex conjugate of each of its

components,i.e.,(u

)

i

= (u

i

)

.

I

Similarly,for A 2 C

nn

,we denote by A

2 C

nn

,the

complex conjugate of A,obtained by taking the complex

conjugate of each of its entries,i.e.,(A

)

ij

= (A

ij

)

.

I

Note that for complex numbers we have z

1

= z

2

iff z

1

= z

2

.

I

This property clearly extends to complex vectors and

matrices:

I

For u;v 2 C

n

we have u = v iff u

= v

and for

A;B 2 C

nn

,we have A = B iff A

= B

.

I

Furthermore,(Au)

= A

u

and (A

)

T

= (A

T

)

.

Eigenvalues of a symmetric real matrix are real

I

Let 2 C be an eigenvalue of a symmetric A 2 R

nn

and

let u 2 C

n

be a corresponding eigenvector:

Au = u:(1)

I

Taking complex conjugates of both sides of (1),we obtain:

A

u

=

u

;i.e.,Au

=

u

:(2)

I

Now,we pre-multiply (1) with (u

)

T

to obtain:

(u

)

T

u = (u

)

T

(Au) = ((u

)

T

A)u

= (A

T

u

)

T

u since (Bv)

T

= v

T

B

T

= (Au

)

T

u since A

T

= A

= (

u

)

T

u =

(u

)

T

u:using ( 2)

I

Thus,(

)(u

)

T

u = 0.

I

But u,being an eigenvector is non-zero and

(u

)

T

u =

P

n

i =1

u

i

u

i

> 0 since at least one of the

components of u is non-zero and for any complex number

z = a +ib we have z

z = a

2

+b

2

0.

I

Hence =

,i.e., and hence u are both real.

I

Eigenvectors of distinct eigenvalues of a symmetric

real matrix are orthogonal

I

Let A be a real symmetric matrix.

I

Let Au

1

=

1

u

1

and Au

2

=

2

u

2

with u

1

and u

2

non-zero

vectors in R

n

and

1

;

2

2 R.

I

Pre-multiplying both sides of the ﬁrst equation above with

u

T

2

,we get:

u

T

2

u

1

= u

T

2

(Au

1

) = (u

T

2

A)u

1

= (A

T

u

2

)

T

u

1

= (Au

2

)

T

u

1

=

2

u

T

2

u

1

:

I

Thus,(

1

2

)u

T

2

u

1

= 0.

I

Therefore,

1

6=

2

implies:u

T

2

u

1

= 0 as required.

I

If an eigenvalue has multiplicity m say then we can

always ﬁnd a set of m orthonormal eigenvectors for .

I

We conclude that by normalizing the eigenvectors of A,we

get an orthonormal set of vectors u

1

;u

2

;:::;u

n

.

Properties of positive deﬁnite symmetric matrices

I

Suppose A 2 R

n

is a symmetric positive deﬁnite matrix,

i.e.,A = A

T

and

8x 2 R

n

n f0g:x

T

Ax > 0:(3)

I

Then we can easily show the following properties of A.

I

All diagonal elements are positive:In (3),put x with x

j

= 1

for j = i and x

j

= 0 for j 6= i,to get A

ii

> 0.

I

The largest element in magnitude in the entire matrix

occurs in the diagonal:Fix i 6= j between 1 and n.In (3),

put x with x

k

= 1 for k = i,x

k

= 1 for k = j and x

k

= 0 for

j 6= k 6= i,to get jA

ij

j < max(A

ii

;A

jj

).

I

All leading principle minors (i.e.,the 1 1,2 2,3 3,

:::;mm matrices in the upper left corner) are positive

deﬁnite:In (3),put x with x

k

= 0 for k > m to prove that the

top left mm matrix is positive deﬁnite.

Spectral decomposition

I

We have seen in the previous pages and in lecture notes

that if A 2 R

nn

is a symmetric matrix then it has an

orthonormal set of eigenvectors u

1

;u

2

;:::;u

n

corresponding to (not necessarily distinct) eigenvalues

1

;

2

;:::;

n

,then we have:

I

The spectral decomposition:Q

T

AQ = where

I

Q = [u

1

;u

2

;:::;u

n

] is an orthogonal matrix with Q

1

= Q

T

and = diag(

1

;

2

;:::;

n

) is diagonal.

I

Assume A 2 R

n

reprsents the linear map f:R

n

!R

n

in

the standard basis of R

n

.

I

Then,the matrix S:= Q

1

2 R

nn

is the matrix for the

change of basis into one in which f is reprsented by

B:= .More generally,B = SAS

1

:

old coordinates

S

A

//

old coordinates

S

new coordinates

B

//

new coordinates

Singular value decomposition (SVD) I

I

Let A 2 R

mn

be an arbitrary matrix.

I

Then A

T

A 2 R

nn

and AA

T

2 R

mm

are symmetric

matrices.

I

They are also positive semi-deﬁnite since for example

x

T

A

T

Ax = (Ax)

T

(Ax) = (kAxk

2

)

2

0.

I

We will show that A = USV

T

,called the SVD of A,where

V 2 R

nn

and U 2 R

mm

are orthogonal matrices whereas

the matrix S = U

T

AV 2 R

mn

is diagonal with

S = diag(

1

;

2

;

3

;:::;

p

) where p = min(m;n) and the

non-negative numbers

1

2

3

:::

p

0 are the

singular values of A.

I

If r is the rank of A then A has exactly r positive singular

values

1

2

3

:::

r

> 0 with

r +1

=

r +2

=:::=

p

= 0.

Singular value decomposition II

I

Note that if the SVD for A as above exists then,since

U

T

U = I

m

,we have A

T

A = VS

T

U

T

USV

T

= VS

T

SV

T

,

where S

T

S = diag(

2

1

;

2

2

;

2

3

;:::;

2

p

) 2 R

nn

is a diagonal

matrix,thus giving the spectral decomposition of the

positive semi-deﬁnite matrix A

T

A.

I

This gives us a method to ﬁnd the SVD of A.

I

Obtain the eigenvalues

2

1

2

2

2

3

:::

2

p

0 and

the corresponding eigenvectors v

1

; ;v

p

of A

T

A.If p < n,

the other eigenvalues of A are zero with corresponding

eigenvectors v

p+1

; ;v

n

which make the orthogonal

matrix V = [v

1

; ;v

n

].

I

From the SVD we have AV = US,thus when

i

> 0,i.e.,

for 1 i r,we get

1

i

Av

i

= u

i

.

I

Extend the set u

1

; ;u

r

to an orthonormal basis

u

1

; ;u

r

; u

m

of R

m

which gives the orthogonal matrix

U = [u

1

; ;u

m

].

Singular value decomposition III

I

Note the following useful facts.

I

For 1 i r,the vector u

i

is an eigenvector of AA

T

with

eigenvalue

2

i

.Check!

I

AA

T

is similar to SS

T

(with identical eigenvalues) and A

T

A

is similar to S

T

S (with identical eigenvalues).

I

The diagonal elements of the diagonal matrices

S

T

S 2 R

nn

and SS

T

2 R

mm

are

2

1

;

2

2

;

2

3

;:::;

2

p

followed by n p zeros and mp zeros respectively.

I

The singular values of A are the positive square roots of

the eigenvalues of AA

T

or A

T

A.

## Comments 0

Log in to post a comment