Classification 1

Question 1

Perceptron

Answer 1

Error-driven learning approach similar to logistic regression
Decision rule: wTx > 0
Gradient update (if incorrect):
- true pos: w <- w+x
- true neg: w <- w-x
- each training instance per update

Question 2

Loss

Answer 2

Classification can be viewed as minimizing loss.

Question 3

Online Optimization

Answer 3

Gradient over randomly sampled example

Question 4

Batch Optimization

Answer 4

Gradient over entire dataset. Inefficient over large training sets

Question 5

Mini-batch Optimization

Answer 5

Each time, calculating gradients from small subsets of examples. Most commonly used now

Question 6

Regularization

Answer 6

Keeping weights small to prevent overfitting. Ex: applying L2-norm penalty to all weights

Question 7

Evaluation

Answer 7

Calculates precision, recall, and accuracy using number of true/false positives and true/false negatives

Question 8

Precision Equation

Answer 8

precision = tp / (tp+fp)

Question 9

Recall Equation

Answer 9

recall = tp / (tp+fn)

Question 10

Accuracy Equation

Answer 10

accuracy = precision + recall = (tp+tn) / (tp+fp+tn+fn)

Question 11

Generative Models

Answer 11

• probabilistic models of P(x,y)
• compute P(y|x) by PREDICTING argmax y P(y|x) to classify

Question 12

Discriminative Models

Answer 12

• model P(y|x) directly by COMPUTING argmax y P(y|x)

Question 13

Naive Bayes

Answer 13

• generative model
• computes P(y|x) by PREDICTING argmax y P(y|x) to classify