Description
Problem 1 (10 points) Different class conditional probabilities. Consider a classification
problem with features in R
d and labels in {−1, +1}. Consider the class of linear classifiers of
the form ( ¯w, 0), namely all the classifiers (hyper planes) that pass through the origin (or t = 0).
Instead of logistic regression, suppose the class probabilities are given by the following function,
where X¯ ∈ R
d are the features:
P
y = +1|X, ¯ w¯
=
1
2
1 +
w¯ · X¯
p
1 + ( ¯w · X¯)
2
!
, (1)
where ¯w · X¯ is the dot product between ¯w and X¯.
Suppose we obtain n examples (X¯
i
, yi) for i = 1, . . . , n.
1. Show that the log-likelihood function is
J( ¯w) = −n log 2 +Xn
i=1
log
1 +
yi( ¯w · X¯
i)
p
1 + ( ¯w · X¯
i)
2
!
. (2)
2. Compute the gradient of J( ¯w) and write one step of gradient ascent. Namely fill in the blank:
w¯j+1 = ¯wj + η ·
hint: use the chain rule and ∇w¯w¯ · X¯ = X¯.
In Problem 2, and Problem 3, we will study linear regression. We will assume in both the
problems that w
0 = 0. (This can be done by translating the features and labels to have mean zero,
