## Description

1. Many problems with matrices are easier to work out when the matrices are partitioned nicely, often

into rows or columns. Let A be an n × m matrix and B an m × p matrix. Denote the i-th row of A

by a

i and the j-th column of B by b

j

. Show the following

(a) AB =

Ab1

|Ab2

| . . . |Abp

(b) AB =

a

1B

a

2B

.

.

.

a

nB

(c) [AB]ij = a

i

b

j

Notation: For a matrix M, where the entry of the i-th row and j-th column is mij , we use the

notation

M

ij = mij .

2. Let A be an n × n (i.e., square) real matrix, and denote the entry of the i-th row and j-th column by

aij . The trace of a matrix is the sum of its diagonal entries, and thus

trace of A = tr(A) = Xn

i=1

aii.

Compute the trace of the following matrices:

(a) A =

1 2 3

4 5 6

7 8 9

1

(b) Let x =

x1

x2

.

.

.

xn

be an vector in R

n. Compute trace

xx>

.

(c) Suppose that K is an n×m real matrix and Q is a square n×n matrix. Let k

i be the i-th column

of K. Compute

tr(K>QK)

where K> denotes the transpose of K. Your answer should be in terms of k

i and Q.

3. Recall that a real matrix M is symmetric if it is equal to its transpose: M> = M. Hence, a symmetric

matrix is square.

(a) By hand, compute the eigenvalues and eigenvectors of M =

2 1

1 3

.

(b) Let v

1 and v

2 be the e-vectors computed above and compute (v

1

)

>v

2

.

(c) Show that M = A>A is symmetric for any real n × m matrix A. [Yes, this is trivial, but useful

to know].

(d) Use the rand command in MATLAB to form 10 different real n × m matrices A, for n ≥ m of

your choice, as long as some of the m are greater than or equal to 3. For each matrix, do the

following

i. Form the m × m matrix M = A>A.

ii. Use the eig command in MATLAB to compute e-values and e-vectors of M.

iii. Denoting the e-vectors by {v

1

, · · · , vm}, form the “inner product” (v

i

)

>v

j

for a few of the

e-vectors, with i 6= j, and see what you get. Note that (v

i

)

>v

j

should be a real number.

iv. Sum up the e-values and compare to the trace of the matrix.

v. Multiply the e-values and compare to the determinant of the matrix (use det command in

MATLAB to compute the determinant),

vi. Summarize your observations on the e-values and the e-vectors in a few sentences. Please do

not report your matrices and calculations. There is no need to turn in your MATLAB code.

4. Let X be a Gaussian random variable with mean µ and standard deviation σ > 0; hence X has density

fX(x) = 1

σ

√

2π

e

−

(x−µ)

2

2σ2

If you prefer to call it a normal distribution, that is fine too!

(a) On the same graph, plot the density for µ = 0 and σ = 1 as well as for µ = 0 and σ = 3.

Obviously, you cannot represent −∞ < x < ∞, so just choose a reasonable subset.

(b) For µ = 2 and σ = 5.0, determine

i. P{X ≥ 4}

ii. P{−2 ≤ X ≤ 4}

iii. P(X ∈ A), where A = [−2, 4] ∪ [8, 100]

(c) What is the density of the random variable Y = 2X + 4? Use the X defined in (b).

5. Let X and Y be jointly distributed random variables with joint density

fXY (x, y) = K(x + y)

2

, 0 ≤ x ≤ 1, 0 ≤ y ≤ 2.

(a) Determine the constant K so that fXY (x, y) is a density. Fix K at this value for the rest of the

problem.

(b) Determine the marginal distributions of X and Y ; in particular, give their densities.

(c) Determine the conditional distribution of X given Y = y; in particular, give the conditional

density.

6. Use the method of Lagrange multipliers to solve the following minimization problem, for x1 ∈ R and

x2 ∈ R:

min (x1)

2 + (x2)

2

subject to: x1 + 3×2 = 4.

No cheating: If you solve for x1 in terms of x2 and substitute into the minimization problem, you get

zero points! The purpose is really to review something you learned in Calculus on a simple problem.

7. Challenge Problem: Let X and Y be jointly Gaussian random variables with means µX = 1 and

µY = 2, and covariance matrix Σ =

3

√

5

√

5 2

. (If you prefer to call these bivariate normal random

variables, that is fine too.)

(a) Determine the marginal distributions of X and Y ; in particular, give their densities.

(b) Determine the conditional distribution of X given Y = y; in particular, give the conditional

density.

(c) Is the variance of X given Y = y greater or less than the variance of X without knowing anything

about the value of Y ?

(d) Plot the conditional density of X given Y = 10.

Remark: If you cannot work Prob. 7, do not worry too much about it. However, do take it as

motivation to start reading on your own about jointly Gaussian random variables. We will need this

material when we cover the Kalman filter.

3

Hints

Hints: Prob. 1 While these are technically proofs, they are direct proofs, and the approach for each of the

three is the same: simply show that the ij entry of the left hand side is equal to the ij entry of the right

hand side. Note that by definition of matrix multiplication,

[AB]ij =

Xm

k=1

aikbkj

Notation that is useful for (a): For a column vector, we let subscript i denote its i-th component. Hence,

[b

j

]i = bij .

Notation that is useful for (b): For a row vector, we let subscript j denote its j-th component. Hence,

[a

i

]j = aij .

Hints: Prob. 2 For facts about the trace operator, see

http://en.wikipedia.org/wiki/Trace_(linear_algebra).

Among the many facts on the above web page, it is worth knowing that if A is an n × m matrix and B is an

m × n matrix, then

trace(AB) = trace(BA).

This can be used to quickly simplify part (b), for example. For part (c), use your results on partitioned

matrices from Prob. 1.

Hints: Prob. 3 In MATLAB, try these commands:

» help rand

» A=rand(3,4)

The command defaults to the uniform distribution. In (d)-(iii), you can form ALL of the inner products

very quickly as follows:

• Define a matrix V with the columns being the eigenvectors; in fact, this is what the eig command

gives in MATLAB.

• Evaluate V

>V , and note from Prob. 1(c), that [V

>V ]ij = (v

i

)

>v

j

.

Hints: Prob. 4

• » help plot

• » help hold

• Part (b) requires integration of the density. Once you set up the integrals, you do NOT have to do

them analytically. You can do them numerically in MATLAB. Try » help quad.

• Part (c) We will review this later in the term. See Example 4 here:

https://www.statlect.com/probability-distributions/normal-distribution-linear-combinations

4

Hints: Prob. 5

• It must integrate to 1. You may find it easier to expand (x + y)

2 before integrating.

• Recall fX(x) = R

2

0

fXY (x, y)dy.

• There is a ratio of two densities involved!

Hints: Prob. 6 Review your Lagrange multipliers online, for example

• https://www.youtube.com/watch?v=ry9cgNx1QV8

• Google “lagrange multipliers” if you need more help.

Hints: Prob. 7

• No further hints given in office hours. This problem will not count towards assignment points.

• X and Y are Gaussian random variables. You can read their means and variances from the provided

data without doing any calculations.

• X given Y is also a Gaussian random variable. You can determine its mean and variance from the

given data with very simple calculations. If you try to derive it from

fX|Y (x|y) = fX,Y (x, y)

fY (y)

you are in for a lot of algebra.

I realize that you may not have covered this material in your undergraduate probability course, and we will review it later in the term. In the meantime, search on the

web to figure out the answer.

• If your search fails, look at the following link.

https://www.probabilitycourse.com/chapter5/5_3_2_bivariate_normal_dist.php

5