Description
1. Suppose you have the following data
Observation X1 X2 Class
1 3 1 +1
2 3 -1 +1
3 6 1 +1
4 6 -1 +1
5 1 0 -1
6 0 1 -1
7 0 -1 -1
8 -1 0 -1
(a) [6 points] Find the equation of the hyperplane (in terms of w) WITHOUT solving a quadratic
programming (QP) problem. Make a sketch of the problem (i.e., plot the data, unique hyperplane and corresponding dashed lines).
(b) [1 point] Calculate the margin.
(c) [2 points] Find the α’s of the SVM for classification (again WITHOUT solving a QP problem).
2. In this problem (based on Problem 7 on p. 371), you will use support vector classification in order
to predict whether a given car gets high or low gas mileage based on the Auto data set. You can
download the data set at http://www-bcf.usc.edu/~gareth/ISL/data.html. It can be helpful
to do the lab starting on p. 359 of the textbook first.
(a) [1 point] Create a class variable that takes on a “1” for cars with gas mileage above the
median, and a “-1” for cars with gas mileage below the median.
(b) [5 points] Fit a support vector classifier with linear kernel to the data with various values of
cost (i.e., the parameter C from class), in order to predict whether a car gets high or low gas
mileage. Report the cross-validation errors associated with different values of this parameter.
Comment on your results.
1
(c) [5 points] Now repeat (b), this time using SVMs with radial basis and polynomial kernels,
with different values of gamma, degree and cost. Comment on your results.
In class, I said the gaussian kernel had the following form: K(Xi
, Xj ) = exp(−kXi−Xjk
2
2
/h2
).
However, in this software package e1071, the radial basis function has the form K(Xi
, Xj ) =
exp(−γkXi − Xjk
2
2
). Consequently, γ = 1/h2
.
(d) [1 point] Which kernel function do you prefer and why?
3. (a) [6 points] Make an R program that solves the LS-SVM for nonlinear regression given the
parameters γ and bandwidth h for the following gaussian kernel (do not implemenent the
cross-validation procedure)
K(Xi
, Xj ) = 1
√
2π
exp
−
kXi − Xjk
2
2
2h
2
.
(b) [3 points] Try your program on the following function (i.e., experiment with some values of γ
and h)
X <- seq(0, 1, length.out = 200)
y <- (sin(2*pi*(x-0.5)))^2 + rnorm(200, 0, 0.2)
Describe the effect when the two tuning parameters γ and h change. Finally, try the values
γ = 9.4365 and h = 0.16006. Does this seems a good fit to you?
2
