## Description

1. See period 11.ipynb.

2. Let a 4-by-2 matrix X have SVD X = USV T where U =

1

2

1 1

1 −1

1 −1

1 1

, S =

”

1 0

0 γ

#

, and V = √

1

2

”

1 1

1 −1

#

.

a) Express the solution to the least-squares problem arg minw ||Xw −y||2

2

as a function of U, S,V , and y.

b) Let y =

1

0

0

1

. Find the weights w that minimize ||Xw − y||2

2

as a function of

γ. Calculate ||Xw − y||2

2

and ||w||2

2

as a function of γ for this value of w. What

happens to ||w||2

2

as γ → 0?

c) Now consider a “low-rank” inverse. Instead of writing

(XTX)

−1XT =

X

p

i=1

1

σi

viu

T

i

where p is the number of columns of X (assumed less than the number of rows),

we approximate

(XTX)

−1XT ≈

Xr

i=1

1

σi

viu

T

i

In this approximation we only invert the largest r singular values, and ignore all

of them smaller than σr. If r = 1, use the low-rank inverse to find w, ||y−Xw||2

2

,

and ||w||2

2 when y =

1

0

0

1

as in part b). Compare ||y − Xw||2

2

, and ||w||2

2

to

the results for part b).

1 of 1