Description
IIT-1 Theory (15 marks)
1. (2 marks) Briefly give the math for feedforward and backprop steps through a single residual block
in ResNets (no need for the full network; you can make necessary assumptions accordingly).
2. (2 marks) A convolutional neural network has 4 consecutive 3 × 3 convolutional layers with stride 1
and no pooling. How large is the support of (the set of image pixels which activate) a neuron in the
4th non-image layer of this network?
3. (2 marks) Consider a fully-connected deep network with one hidden layer. If the number of hidden
units is increased, what effect would this have on bias and variance of the learned model? Explain
briefly.
4. (3 marks) Consider a two-layer neural network as given below:
yk(x, w) = σ
X
M
j=1
w
(2)
kj h
X
D
i=1
w
(1)
ji xi + w
(1)
j0
+ w
(2)
k0
1
where h is a layer with a linear activation function, and σ(a) = {1 + exp (−a)}
−1
is the sigmoid
activation function. Your classmate tells that she can derive an equivalent network which computes
the same function, but with the final layer activation functions replaced with tanh(a)(= e
a−e−a
e
a+e−a ).
Is she correct? Can you prove and support her argument? (Hint: Think about relating σ(a) and
tanh(a), and then showing that the two networks’ parameters differ only by linear transformations.)
5. (3 marks) Consider a quadratic error defined by E(w) ≈ E(w∗
) + 1
2
(w − w∗
)
TH(w − w∗
), in which
the Hessian matrix H, evaluated at w∗ has an eigenvalue equation given by Hui = λiui
. Show
that the contours of constant error are ellipses whose axes are aligned with the eigenvectors ui
, with
lengths that are inversely proportional to the square root of the corresponding eigenvalues λi
. (Hint:
Note that it is possible to write w − w∗ as P
i αiui
for some αis (you can assume this, if you like!).
How does this help you?)
6. (3 marks) As part of improving customer service, Kaziranga National Park is designing a mobile
application where a visitor can take a photo of an animal, and the application identifies the same.
They have collected 20 images of all 200 species of animals in the park. They have access to a deep
learning model deployed at Olympic National Park, Washington trained on 1 million images from
1000 classes. Can you help the digitization team at Kaziranga National Park to develop their deep
learning model? What exact steps would you suggest?
2 Programming (35 marks)
• The programming questions are shared in “Assignment 3.zip”. Please follow the instructions in the
notebook. Turn-in the notebook via Google Classroom once you finish your work.
• Marks breakdown is as follows:
– Question 1: 17 marks (15 marks for code completion and 2 marks for reporting train & test
loss, final accuracies and the plots as requested.)
– Question 2: 4 marks
– Question 3: 6 marks
– Question 4: 8 marks
2
