Chapter 4

Question 1

What is parameter learning or parametric modeling ?

Answer 1

It is the tuning of the parameters so that the model fits the data as well as possible.

Question 2

What dose logistic regression do ?

Answer 2

Logistic regression applies linear models to class probability estimation.

Question 3

What is the difference between Linear regression, Support vector machine, and Logistic regression ?

Answer 3

The three of them are similar instances of the basic fundamental technique: fitting a linear model to data.
But, they use different objective function.

Question 4

What is the difference between Support vector machine, and Logistic regression ?

Answer 4

logistic regression separates the 2 classes completely.
Where support vector machine is almost midway between classes, it miss classifies some points.

The methods produce different boundaries because they are optimizing different functions.

Question 5

How does the linear discriminant function help in ranking predicted probability ?

Answer 5

given the equation f(x) > 0 then belong to class +
f(x) <= 0 then belong to class -
fx will be zero when siting on the decision boundary.
fx will be small when x is near boundary.
fx will be large when x is far from boundary

Question 6

What is the difference between Logistic regression and Tree induction ?

Answer 6

In classification trees the decision boundary is perpendicular to the instance space axes, meaning that the tree select a single attribute at a time.

In linear regression the decision boundary can be in any direction, meaning that linear classifiers use a weighted combination of all attributes.

A classification tree can segments the instance space recursively, it can cut up the instance space arbitrarily.

A linear classifier is limited to a single division into 2 segments.

Question 7

What is kernel function in SVM ?

Answer 7

Kernel function maps the original features to some other feature space.

Question 8

What is fitting a model to data ?

Answer 8

It is a measure of how well a model generalizes to other data different then the ones that it was trained on.
A model that is well-fitted produces more accurate outcomes. A model that is over-fitted matches the data too closely.

Question 9

How can you apply logistic regression for the case that your explanatory (or input) data has categorical variables ?

Answer 9

Logistic regression models for binary target variables allow us to estimate the probability of the outcome.
So, it is a notion of likelihood rather than class membership.
Using dummy variables (1 , 0)

Question 10

What is the importance of kernel trick in SVM ?

Answer 10

If we find a way to map the data from 2-dimensional space to 3-dimensional space, we will be able to find a decision surface that clearly divides between different classes.

Question 11

What is the difference between linear and non-linear kernel in SVM ?

Answer 11

Linear Kernel is used when the data is Linearly separable. It is mostly used when there are a Large number of Features in a particular Data Set.

When we cannot separate data linearly we use Non – Linear kernel. It transforms data into another dimension so that the data can be classified.