## Description

1. Let X =

1 1

1 0

0 1

.

a) Use the Gram-Schmidt orthogonalization procedure and hand calculation to find

an orthonormal basis for the space spanned by the columns of X. What geometric

object is described by the span of these bases?

b) Now interchange the columns of X, that is, define X˜ =

1 1

0 1

1 0

i. Do the columns of X span the same space as the columns of X˜ ?

ii. Use the Gram-Schmidt orthogonalization procedure to find an orthonormal

basis for the space spanned by the columns of X˜ . How does the geometric

object described by the span of this set of orthonormal bases compare to the

one in Part a?

iii. Are the bases vectors you found for X and X˜ the same? Does the space

spanned by the columns of a matrix depend on the order of the columns?

2. Let X =

1 1

1 0

0 1

as in the previous problem.

a) Place the orthonormal bases you found as columns of a matrix U.

b) Find UT U.

c) Since U contains a basis for space spanned by the columns of X you decide to

write each column of X as a linear combination of the columns of U: X =

U

a1 a2

. What is the dimension of a1? Briefly describe the meaning of a1

and a2.

d) Let A =

a1 a2

so that X = UA. Multiply both sides of this equation by

UT and solve for A.

1 of 2

3. Let the columns of an n-by-p (n > p) matrix X be linearly independent and U be an

orthonormal basis for the p-dimensional space spanned by the columns of X.

a) It can be shown that X = UT where T is a p-by-p invertible matrix. Briefly

explain why T should be invertible without resorting to a mathematical proof.

That is, explain why this result is intuitively reasonable.

b) Use the result in the previous item to show that the projection onto the space

spanned by X is identical to that onto the space spanned by U. That is, show

Px = X(XTX)

−1XT = PU = U(UT U)

−1UT

. Hint: Recall that (AB)

−1 =

B−1A−1

.

c) Express PU without a matrix inverse.

4. Consider the matrix and vector

X =

1 1

1 0

0 1

and b =

1

2

1

.

Note that X is defined identically in the preceding problems.

a) Make a sketch of the orthonormal bases U and the columns of X in three dimensions.

b) Use U and the result of the previous problem to compute the LS estimate ˆb =

X(XTX)

−1XT b.

5. Let z =

”

1

1

#

and define Q = zzT

.

a) Sketch the surface y = x

TQx where x =

”

x1

x2

#

. If you find 3-D sketching too

difficult, you may draw a contour map with labeled contours.

b) Let w =

”

1

−1

#

. Sketch the subspace spanned by z and the subspace spanned

by w on your sketch of the surface y = x

TQx.

c) Does the problem minx x

TQx have a unique solution?

d) Is Q 0? Is Q 0?

2 of 2