Non-linear transforms

Question 1

Why are non-linear transforms useful

Answer 1

Many models, like linear classification, linear regression, logistic regression, are based on a score variable s that is a linear function of the weights ( s = w’ x ).
This assumption is not enough when, for example, data are not linearly separable in a classification problem.
In these cases, a transforms of the feature space can be used.

Question 2

Steps of a Learning procedure with a non-linear transform

Answer 2

z = Φ(x) is the tranform

1) zn = Φ(xn) for n=1,…,N
Data are transformed in a different space, where linear models can be applied

2) selection of the hypothesis h(z) using algorithms for linear models
for example h(z) = sign( w’ z ) in classification

3) computation of the hypothesis wrt the original data
- > h(x) = h( w’ Φ(x) )

Question 3

Generalization issues

Answer 3

• If Φ is selected without looking at the data, generalization properties don’t change
• But, if Φ is selected looking at the data, a data snooping is said to happen, so the generalization properties are worst