Sale!

# DDA2020 Assignment I solution

Original price was: \$35.00.Current price is: \$29.00.

Category:

5/5 - (1 vote)

## 1 Written Problems (6 points)

1. MLE minimizes KL divergence to the empirical distribution (Exercise
2.15 of Murphy’s book) (1 point)
2. Centering and ridge regression (Exercise 7.3 of Murphy’s book) (1
point)

3. Symmetric version of `2 regularized multinomial logistic regression (Exercise 8.5 of Murphy’s book) (1 point)
4. Elementary properties of `2 regularized logistic regression (Exercise 8.6
of Murphy’s book) (1 point)
5. Given the following denominator layout derivatives, (2 points)
• Differentiation of a scalar function w.r.t. a vector: If f(w) is a
scalar function of d variables, w is a d×1 vector, then differentiation
of f(w) w.r.t. w results in a d × 1 vector

df(w)
dw
=

∂f
∂w1
.
.
.

∂f
∂wd

• Differentiation of a vector function w.r.t. a vector: If f(w) is a
vector function of size h×1 and w is a d×1 vector, then differentiation
of f(w) w.r.t. w results in a d × h vector
df(w)
dw
=

∂f1
∂x1
. . .
∂fh
∂w1
.

.
.
.
.
.
.
.
.
∂f1
∂wd
. . .

∂fh
∂wd

1

Please prove the following derivatives, and X and y are not functions of
w:
d(X>w)
dw
= X,
d(y
>Xw)
dw
= X>y
d(w>Xw)
dw

= (X + X>)w

## 2 Programming (5 points)

Given Xw = y where X =

1 2
4 3
5 6
3 8
9 10

and y =

−1
0
1
0
0

which constitute an exemplary problem. Write a Python routine to find the least squares solution
for w given arbitrary X ∈ R5×2 and y ∈ R5×1

as a function routine (“def A1 MatricNumber(X,y)”) that takes in X and y as
inputs and generate (X>X)
−1 and w as outputs in a single file with filename

“A1 StudentMatriculationNumber.py”. Your Python routine should return the
least squares solution vector w (as numpy array) and two matrices X> and
(X>X)
−1

. Hint: you will need “import numpy as np” and its matrix manipulation functions.
Please use the python template provided to you. Remember to rename both
“A1 StudentMatriculationNumber.py” and “A1 MatricNumber” using your student matriculation number. For example, if your matriculation ID is 123456789,
then you should submit “A1 123456789.py” that contains the function “A1 123456789”.

The way we would run your code might be something like this: