Sale!

Homework 2 AMATH 563 solved

Original price was: $30.00.Current price is: $25.00.

Category:

Description

5/5 - (1 vote)

Problems
Theory

1. Suppsoe ฮ“ : ๐’ณ ร—๐’ณ โ†’ R is a PDS kernel. Prove that โˆ€๐‘ฅ, ๐‘ฅโ€ฒ โˆˆ ๐’ณ it holds that |ฮ“(๐‘ฅ, ๐‘ฅโ€ฒ
)|
2 โ‰ค ฮ“(๐‘ฅ, ๐‘ฅ)ฮ“(๐‘ฅ
โ€ฒ
, ๐‘ฅโ€ฒ
).
2. Given a kernel ๐พ on ๐’ณ define its normalized version as
๐พยฏ (๐‘ฅ, ๐‘ฅโ€ฒ
) =
โŽง
โŽชโŽจ
โŽชโŽฉ
0 if ๐พ(๐‘ฅ, ๐‘ฅ) = 0 or ๐พ(๐‘ฅ
โ€ฒ
, ๐‘ฅโ€ฒ
) = 0
๐พ(๐‘ฅ, ๐‘ฅโ€ฒ
)
โˆš๏ธ€
๐พ(๐‘ฅ, ๐‘ฅ)
โˆš๏ธ€
๐พ(๐‘ฅ
โ€ฒ
, ๐‘ฅโ€ฒ)

Otherwise.
Show that if ๐พ is PDS then so is ๐พยฏ .
3. Show that the following kernels on R
๐‘‘ are PDS:
โ€ข Polynomial kernel: ๐พ(๐‘ฅ, ๐‘ฅโ€ฒ
) = (๏ธ
๐‘ฅ
๐‘‡ ๐‘ฅ
โ€ฒ + ๐‘
)๏ธ๐›ผ
for ๐‘ > 0 and ๐›ผ โˆˆ N.
โ€ข Exponential kernel: ๐พ(๐‘ฅ, ๐‘ฅโ€ฒ
) = exp(๐‘ฅ
๐‘‡ ๐‘ฅ
โ€ฒ
).

โ€ข RBF kernel: ๐พ(๐‘ฅ, ๐‘ฅโ€ฒ
) = exp(โˆ’๐›พ
2โ€–๐‘ฅ โˆ’ ๐‘ฅ
โ€ฒโ€–
2
2
).
4. Let ฮฉ โŠ† R
๐‘‘ and let {๐œ“๐‘—}
๐‘›
๐‘—=1 be a sequence of continuous functions on ฮฉ and {๐œ†๐‘—}
๐‘›

๐‘—=1 a sequence of
non-negative numbers. Show that ๐พ(๐‘ฅ, ๐‘ฅโ€ฒ
) = โˆ‘๏ธ€๐‘›
๐‘—=1 ๐œ†๐‘—๐œ“๐‘— (๐‘ฅ)๐œ“๐‘— (๐‘ฅ
โ€ฒ
) is a PDS kernel on ฮฉ.

5. Show that: (i) if ๐พ and ๐พโ€ฒ are two reproducing kernels for an RKHS โ„‹, then they have to be the
same. (ii) the RKHS of a PDS kernel ๐พ is unique.

Computation

Download the MNIST training and test .csv files from Canvas and load them on your computer. I suggest
you use Python or MATLAB for this excercise.
โ€ข Use Principle Component Analysis (PCA) on the training set to reduce the dimension of your input.
How many modes do you need to preserve 95% of the variance in the training set?

โ€ข Extract the digits 1 and 9 from the training set. Use kernel regression to design and train a classifier
to distinguish these digits using three different kernels of your choosing (I suggest RBF, Polynomila,
and linear). It is a good idea to use PCA to reduce your input dimensions here.

Also, you may use
cross validation to tune your kernel/regularization/nugget parameters if you need them. Present the
training and test error of your method.
โ€ข Repeat the above experiment for the digits (3, 8), (1, 7), and (5, 2).

โ€ข Write a report of a maximum of four pages, outlining your results and findings.