Unit 4: Support Vector Machines and Flexible Discriminants

Question 1

Explain the role of kernel functions in SVM.

Answer 1

Kernel Functions: Transform data into higher-dimensional space to make it easier to classify.
Common Kernels:

Linear Kernel: Suitable for linearly separable data.
    Equation: K(x,y)=xTyK(x,y)=xTy
Polynomial Kernel: Suitable for non-linear data.
    Equation: K(x,y)=(xTy+c)dK(x,y)=(xTy+c)d (where cc is a constant and dd is the degree).
Radial Basis Function (RBF) Kernel: Effective for non-linear classification.
    Equation: K(x,y)=e−γ∣∣x−y∣∣2K(x,y)=e−γ∣∣x−y∣∣2 (where γγ is a parameter).

Question 2

What are Support Vector Machines, and how do they work?

Answer 2

Support Vector Machines (SVM): A supervised learning algorithm used for classification and regression tasks.
Working Principle:

Hyperplane: SVM finds the optimal hyperplane that maximizes the margin between classes.
Support Vectors: Data points closest to the hyperplane that influence its position.

Question 3

What are flexible discriminants, and how do they differ from traditional methods?

Answer 3

Flexible Discriminants: Models that adapt to the structure of the data without strict assumptions (e.g., normality).
Differences from Traditional Methods:

Assumption-Free: Do not rely on assumptions about the distribution of data.
Non-Parametric: Can model complex relationships.
Applications: Used in scenarios where traditional models (like linear regression) may fail.