ML-02 - Logistic regression & classification

Question 1

ML-02 - Logistic regression & classification

Describe what a classification problem is.

Answer 1

A classification problem is when the output (𝑦) is discrete or categorical, e.g., email classified as spam, not spam.

Question 2

ML-02 - Logistic regression & classification

What are the two types of classification problems?

Answer 2

• Binary, 2 classes (Dog or not dog; cat or dog)
• Multiclass (Dog, cat, giraffe or zebra)

Question 3

ML-02 - Logistic regression & classification

Why do we use logistic regression over linear regression for classification problems?

Answer 3

Logistic regression handles discrete output for classification, unlike linear regression’s continuous values.

Question 4

ML-02 - Logistic regression & classification

What’s the most common logistic function?

Answer 4

The sigmoid function (See image).

Question 5

ML-02 - Logistic regression & classification

What’s the name of the pictures function? (See image)

Answer 5

The sigmoid function.

Question 6

ML-02 - Logistic regression & classification

How do we apply logistic regression?

Answer 6

(See image)

Question 7

ML-02 - Logistic regression & classification

How do we interpret the outputs of a logistic regression model?

Answer 7

Estimated probability of the positive case being true.

E.g. P(y = 1 | x ; w)

Question 8

ML-02 - Logistic regression & classification

What is another name for the outputs of a logistic function?

Answer 8

Logits.

Question 9

ML-02 - Logistic regression & classification

What is a logit?

Answer 9

The output values of a logistic function are often called logits.

Question 10

ML-02 - Logistic regression & classification

What is a linear decision boundary?

Answer 10

A line that separates one class of data from another class.

(See image)

Question 11

ML-02 - Logistic regression & classification

What is a non-linear decision boundary?

Answer 11

An equation that separates different classes of data.

(See image)

Question 12

ML-02 - Logistic regression & classification

What is an advantage to using non-linear decision boundaries?

Answer 12

They can represent more complex decision boundaries.

Question 13

ML-02 - Logistic regression & classification

How do you write a decision boundary in matrix form?

Answer 13

(See image)

Question 14

ML-02 - Logistic regression & classification

What is the loss function used for logistic regression?

Answer 14

Cross-entropy loss

Question 15

ML-02 - Logistic regression & classification

What’s another name for cross-entropy loss?

Answer 15

Log loss

Question 16

ML-02 - Logistic regression & classification

What’s another name for log loss?

Answer 16

Cross-entropy loss

Question 17

ML-02 - Logistic regression & classification

What’s the formula for cross-entropy loss?

Answer 17

(See image)

Question 18

ML-02 - Logistic regression & classification

What loss fomula is depicted? (See image)

Answer 18

Cross-entropy loss

Question 19

ML-02 - Logistic regression & classification

What loss function would you use for a binary class problem, when using logistic regression?

Answer 19

Binary cross-entropy loss

Question 20

ML-02 - Logistic regression & classification

What is the formula for binary cross-entropy loss?

Answer 20

(See image)

Question 21

ML-02 - Logistic regression & classification

What loss function is this? (See image)

Answer 21

Binary cross-entropy loss

Question 22

ML-02 - Logistic regression & classification

What’s the full formula for binary cross-entropy loss?

Answer 22

(See image)

Question 23

ML-02 - Logistic regression & classification

What are 3 more advanced optimization algorithms mentioned in this chapter?

Answer 23

• CG
• BFGS
• L-BGFS

Question 24

ML-02 - Logistic regression & classification

What’s the optimization algorithm CG short for?

Answer 24

Conjugate gradient

Question 25

ML-02 - Logistic regression & classification

What’s the optimization algorithm BFGS short for?

Answer 25

Broyden, Fletcher, Goldfarb, and Shannon

Question 26

ML-02 - Logistic regression & classification

What’s the optimization algorithm CG short for?

Answer 26

Conjugate gradient

Question 27

ML-02 - Logistic regression & classification

What are the advantages of CG, BFGS and L-BFGS?

Answer 27

• no need to manually pick learning rate (𝛼).
• often faster (converging) than gradient descent.

Question 28

ML-02 - Logistic regression & classification

What are the disadvantages of CG, BFGS and L-BFGS?

Answer 28

They’re more complex than gradient descent.

Question 29

ML-02 - Logistic regression & classification

How does algorithm performance scale with data size?

Answer 29

Algorithms performance similar when dataset is large.
It’s not who has the best algorithm that wins, it’s who has the most/best data.

Question 30

ML-02 - Logistic regression & classification

How do you deal with computational cost as dataset size grows? (2)

Answer 30

• Use variants of gradient descent.
• Map/reduce + parallelism

Question 31

ML-02 - Logistic regression & classification

How do you parallel process in machine learning?

Answer 31

Split training dataset into different pieces and compute them in parallel, and then combine

(See image)

Question 32

ML-02 - Logistic regression & classification

What are the names of the two big GPU platforms/programming models?

Answer 32

• CUDA
• OpenCL

Question 33

ML-02 - Logistic regression & classification

Who’s behind CUDA?

Answer 33

Nvidia

Question 34

ML-02 - Logistic regression & classification

Who’s behind OpenCL?

Answer 34

Apple and Khronos Group

Question 35

ML-02 - Logistic regression & classification

What’s Nvidia’s GPU computing platform/programming model called?

Answer 35

CUDA

Question 36

ML-02 - Logistic regression & classification

What’s Apple + Khronos Group’s GPU computing platform/programming model called?

Answer 36

OpenCL

Question 37

ML-02 - Logistic regression & classification

How do you handle multiclass problems?

Answer 37

Use “1 vs. rest”?

Question 38

ML-02 - Logistic regression & classification

What is “1 vs. rest”?

Answer 38

• Treat multiclass classification as multiple binary class problems.
• Train 1 classifier for each problem.

(See image)

Question 39

ML-02 - Logistic regression & classification

What do you call the depicted type of classifier?

(See image)

Answer 39

1 vs. rest (for multiclass classification)