Description
Q1 (6%). x = [5, −3, −1, 2]T
is a 4-dimensional vector. Calculate the following values:
1. the squared ℓ2-norm of x:
∥x∥
2
2 =
2. the ℓ1-norm of x:
∥x∥1 =
3. the inner product of x and a, where a = [4, −2, 6, −1]T
:
a
T x =
Q2 (4%). The matrix A ∈ R
2×3 and vector b ∈ R
3 are defined in the following:
A =
6 1 −2
−5 7 9
and b =
−4
5
2
.
Caculate the following values:
1. the matrix-vector product:
Ab =
2. the matrix-matrix product:
AAT =
1
Q3 (6%). Let x = [x1, x2, x3] and y =
x
2
1
2 + loge x2 −
x1
x3
. Calculate the following:
d y
d x
at x = [9, 1,
1
2
]:
Hint: the answer should be a three dimensional vector.
Q4 (4%). X is an n × d matrix, y is an n × 1 vector, and w is an d × 1 vector. Let f(w) =
∥Xw − y∥
2
2 + λ∥w∥
2
2
. Calculate the following:
∂ f(w)
∂ w =
Q5 (80%). In this problem, you will need to write python codes and build logistic regression classifiers on breast cancer dataset to predict whether the cancer is malignant or benign
on the patients. The dataset contains 569 samples and 30 features. You may refer to https:
//archive.ics.uci.edu/ml/datasets/breast+cancer+wisconsin+(diagnostic) for more information about the dataset.
Your implementations should include:
• Load data, clean data and partition them into training and testing data.
• Build logistic regression and L2-regularized logistic regression models.
• Implement three gradient descent algorithms for each model: Batch Gradient Descent (GD),
Mini-Batch Gradient Descent (MB-SGD) and Stochastic Gradient Descent (SGD).
• Compare the loss curve of three gradient descent algorithms (GD/MB-SGD/SGD).
• Compare logistic regression and regularized version in terms of training and testing error.
• Try to tune different parameters (regularization parameter, learning rate, etc.) to see their
effects.
You may find more detailed procedure in the IPython notebook file. You could use sklearn or any
other packages to load and process the data, but you can not directly use the package to train the
model.
2 CS 583 Assignment 1
