Sale!

# Homework 2 AMATH 563 solved

Original price was: \$30.00.Current price is: \$25.00.

Category:

5/5 - (1 vote)

## Problems Theory

1. Suppsoe ฮ : ๐ณ ร๐ณ โ R is a PDS kernel. Prove that โ๐ฅ, ๐ฅโฒ โ ๐ณ it holds that |ฮ(๐ฅ, ๐ฅโฒ
)|
2 โค ฮ(๐ฅ, ๐ฅ)ฮ(๐ฅ
โฒ
, ๐ฅโฒ
).
2. Given a kernel ๐พ on ๐ณ define its normalized version as
๐พยฏ (๐ฅ, ๐ฅโฒ
) =
โง
โชโจ
โชโฉ
0 if ๐พ(๐ฅ, ๐ฅ) = 0 or ๐พ(๐ฅ
โฒ
, ๐ฅโฒ
) = 0
๐พ(๐ฅ, ๐ฅโฒ
)
โ๏ธ
๐พ(๐ฅ, ๐ฅ)
โ๏ธ
๐พ(๐ฅ
โฒ
, ๐ฅโฒ)

Otherwise.
Show that if ๐พ is PDS then so is ๐พยฏ .
3. Show that the following kernels on R
๐ are PDS:
โข Polynomial kernel: ๐พ(๐ฅ, ๐ฅโฒ
) = (๏ธ
๐ฅ
๐ ๐ฅ
โฒ + ๐
)๏ธ๐ผ
for ๐ > 0 and ๐ผ โ N.
โข Exponential kernel: ๐พ(๐ฅ, ๐ฅโฒ
) = exp(๐ฅ
๐ ๐ฅ
โฒ
).

โข RBF kernel: ๐พ(๐ฅ, ๐ฅโฒ
) = exp(โ๐พ
2โ๐ฅ โ ๐ฅ
โฒโ
2
2
).
4. Let ฮฉ โ R
๐ and let {๐๐}
๐
๐=1 be a sequence of continuous functions on ฮฉ and {๐๐}
๐

๐=1 a sequence of
non-negative numbers. Show that ๐พ(๐ฅ, ๐ฅโฒ
) = โ๏ธ๐
๐=1 ๐๐๐๐ (๐ฅ)๐๐ (๐ฅ
โฒ
) is a PDS kernel on ฮฉ.

5. Show that: (i) if ๐พ and ๐พโฒ are two reproducing kernels for an RKHS โ, then they have to be the
same. (ii) the RKHS of a PDS kernel ๐พ is unique.

## Computation

you use Python or MATLAB for this excercise.
โข Use Principle Component Analysis (PCA) on the training set to reduce the dimension of your input.
How many modes do you need to preserve 95% of the variance in the training set?

โข Extract the digits 1 and 9 from the training set. Use kernel regression to design and train a classifier
to distinguish these digits using three different kernels of your choosing (I suggest RBF, Polynomila,
and linear). It is a good idea to use PCA to reduce your input dimensions here.

Also, you may use
cross validation to tune your kernel/regularization/nugget parameters if you need them. Present the
training and test error of your method.
โข Repeat the above experiment for the digits (3, 8), (1, 7), and (5, 2).

โข Write a report of a maximum of four pages, outlining your results and findings.