Classification

Question 1

How is LDA similar to Logistic regression?

Answer 1

Both LDA and Logistic regression produce linear decision boundary.

Question 2

True Positive

Answer 2

Actual = 1 | Prediction = 1

Question 3

False Negative

Answer 3

Actual = 1 | Prediction = 0

Question 4

When can accuracy be misleading?

Answer 4

When we have imbalanced data set, and prediction of minority class is critical.

Question 5

What is Macro Precision (or Recall)

Answer 5

Average of Precision (or Recall) of all classes.

Question 6

What is Confusion Matrix?

Answer 6

Confusion matrix is a matrix which shows the relation between the actual value and the predicted value.

Question 7

Why do we only calculate Precision and Recall for positive class?

Answer 7

This is because our area of concern is positive class.

Spam Detection, Spam - Positive Anomaly Detection, Anomaly - Positive Disease Detection, Diseases - Positive

Question 8

What is Weighted Precision (or Recall)?

Answer 8

Weighted Average of Precision (or Recall) of all classes.

Question 9

How much accuracy is good?

Answer 9

It depends on the problem.
- In the case of medical data, even the accuracy of 99% is bad.

Question 10

What is accuracy?

Answer 10

Correct Prediction/Total Prediction

Question 11

Problem with accuracy

Answer 11

Accuracy does not tell about the nature of incorrect prediction.

Actual = 0 | Prediction = 1
Actual = 1 | Prediction = 0

The above both cases are different, however in terms of accuracy they are same.

Question 12

Type I error

Answer 12

False Positive

Question 13

When to use Logistic regression or LDA?

Answer 13

LDA assumes that the observations are drawn from a Gaussian distribution.

• When the data follows Gaussian distribution then LDA performs better than Logistic regression.
• If Gaussian assumptions are not met, Logistic regression can out perform LDA.

Question 14

Type II error

Answer 14

False Negative

Question 15

Precision

Answer 15

How accurate you are at predicting positives i.e., of all the classes that we predicted positive, how many actually are!

Precision = TP/(TP+FP)

Question 16

How is LDA different from Logistic Regression?

Answer 16

• In case of LDA co and c1 are computed using estimated mean and variance from a normal distribution.
• In the case of Logistic regression, B0 and B1 are estimated using maximum likelihood.

Question 17

Why do we consider harmonic mean not Arithmetic mean in F1 Score.

Answer 17

This is because harmonic mean penalizes the smaller value, be it Precision or Recall.

Harmonic Mean is more tilted towards smaller value.

Question 18

True Negative

Answer 18

Actual = 0 | Prediction = 0

Question 19

False Positive

Answer 19

Actual = 0 | Prediction = 1

Question 20

F1 - Score

Answer 20

Maintains the balance between Precision and recall, when both Precision and recall are necessary.

Harmonic mean of Precision and Recall.

F1 = 2.Precision.Recall/(Precision+Recall)

Question 21

When to use QDA for classification?

Answer 21

• QDA assumes a quadratic decision boundary.
• QDA serves as a compromise between KNN and linear LDA and Logistic regression.
• It performs better in the presence of a limited number of training observations because it does make some assumptions about the form of decision boundary.

Question 22

Recall

Answer 22

Of all the classes that are positive, how many we predicted correctly.

Recall = TP/(TP+FN)

Question 23

Why can’t we use linear regression for classification?

Answer 23

This is because, there is no natural way to convert a qualitative response variable with more than two levels into a quantitative response that is ready for Linear regression.

However, for a classification problem of two variables, we could fit a linear regression after 0/1 coding of variable.