Description
1. Suppose X ∈ R
p and Y ∈ {−1, 1}. For any real-valued function f on R
p
, let
L(Y, f(X)) denote the loss function for measuring errors between Y and f(X). Let
f
∗ = arg minf EL(Y, f(X)), where the expectation is taken over the joint distribution of X and Y . Show that
(a) (Logistic Regression) If L(y, f(x)) = log[1 + exp(−yf(x))], then f
∗
(x) =
log P r(Y =1|X=x)
P r(Y =−1|X=x)
.
(b) (SVM) If L(y, f(x)) = [1 − yf(x)]+, then f
∗
(x) = sign[P r(Y = 1|X = x) −
1
2
].
(c) (Regression) If L(y, f(x)) = [y − f(x)]2
, then f
∗
(x) = 2P r(Y = 1|X = x) − 1.
(d) (AdaBoost) If L(y, f(x)) = exp[−yf(x)], then f
∗
(x) = 1
2
log P r(Y =1|X=x)
P r(Y =−1|X=x)
.
2. Get the “Ripley dataset” (synth.tr) from the website http://www.stats.ox.ac.uk/pub/PRNN/.
The dataset contains two predictors and a binary outcome.
(a) Construct a linear support vector classifier.
(b) Construct a support vector classifier with Radial kernel.
(c) Construct a classifier using AdaBoost algorithm (with 50 boosting iterations)
with decision stumps as weak learners.
Select the tuning parameter involved in SVM models appropriately. For each
method, compute the test error and its standard error on the test set (synth.te).
Provide a simple graphical visualization of the produced classification models (i.e.
something similar to Figure 2.2 in the textbook [ESL]) and discuss your results.
1 P9120
