L5_KRR

Question 1

Kernel-Methoden können implizit beliebig komplexe versteckte Ebenen modellieren

Answer 1

Kernel methods can implicitly model arbitrarily complex hidden layers

Question 2

Multilayer Perceptrons (MLP) can

Answer 2

learn linearly non separable problems

Question 3

Universal Approximation Theorem

Answer 3

With enough intermediate variables, a network with a single hidden layer can approximate any reasonable function of the input.

Question 4

Kernelizing linear methods

Answer 4

1. Map the data into a (high dimensional) feature space

2. Look for linear relations in the feature space

Question 5

Kernel Trick

Answer 5

Any algorithm for vectorial data that can be expressed only in terms of scalar products between vectors can be performed implicitly in the feature space associated with any kernel, by replacing each scalar product by a kernel evaluation.

Question 6

A big problem with high dimensional features spaces

Answer 6

When the dimensionality increases, the volume of the space increases so fast that the available data becomes sparse. The amount of data needed for a reliable result often grows exponentially with the Dimensionality

Question 7

What about the curse of dimensionality?

Representer Theorem

Answer 7

In a regularized learning problem, the optimal weights w in feature space are a linear combination of the training examples in feature space φ(xi ): sum αi φ(xi )

Question 8

Kernel methods are memory-based methods: 3

Kernels as Similarity Measures

Answer 8

• store the entire training set
• define similarity of data points by kernel function
• new predictions require comparision with previously learned examples

Question 9

When do humans percieve stimuli as similar?

Answer 9

Perceptual Similarity of new data x decays exponentially with distance from prototype μ

Question 10

Kernel Methods - Pros

Answer 10

+ Powerful Modeling Tool
(non-linear problems become linear in kernel space)
+ Omnipurpose Kernels
(Gaussian works well in many cases)
+ Kernel methods can handle symbolic objects
+ When you have less data points than your data has dimensions kernel methods can offer a dramatic speedup

Question 11

Kernel Methods - Cons

Answer 11

– Difficult to understand what’s happening in kernel space
– Model complexity increases with number of data points
→ If you have too much data, kernel methods can be slow