General

Question 1

Stochastic Gradient Descent

Answer 1

Gradient descent algorithm that increments the parameters using only single observations at a time.

More efficient than batch gradient descent, especially with large datasets.

Question 2

Batch Gradient Descent

Answer 2

Gradient descent algorithm in which it is required to scan through the entire training set before taking a single step.

Question 3

Localized Linear Regression

Answer 3

A variant of traditional linear regression that uses only local data points around Xi to predict Yi

Question 4

Type I Error (False Positive)

Answer 4

Incorrectly rejecting the null hypothesis in favor of the alternative hypothesis when the null is true.

Same as alpha, set at the beginning of the experiment.

Question 5

Type II Error (False Negative)

Answer 5

Failing to reject the null hypothesis when it is false

Also known as beta. Note that power is (1 - beta)

Question 6

A\B Testing

Answer 6

Processing of testing two groups against a given desired measured to determine if there is a statistical difference

General Strategy:

1. Identify comparative statistic
2. Determine sample size
3. Analyze results

Question 7

SVMs : General description

Answer 7

Simple: Machine learning model that uses a hyperplane to differentiate and classify different groups of data

Detailed: SVM identified an appropriate hyperplane by attempting to maximize the margins between points between the closest points of each class to boundary.

If data that cannot be separated linearly, use transformations to map data into high dimensions.

Question 8

SVMs: Soft Margin Classification

Answer 8

A mechanism that serves to reduce the overfitting of maximum margin classification by penalizing misclassifications

Question 9

Bias / Variance Tradeoff

Answer 9

A tradeoff in machine learning models where you have the choice of reducing bias (how well a model fits a specific set of data) vs. reducing variance (how much performance of a mode varies across many datasets).

Question 10

Precision

Answer 10

TP / (TP + FP)

Measures the accuracy of positive predictions (but not necessarily identifying all of them).

Question 11

Recall

Answer 11

TP / (TP + FN)

Measures completeness of positive predictions

Question 12

F1 Score

Answer 12

2 / ((1/Precision) + 1/Recall))

Harmonic mean of precision and recall, ranging between 0 and 100%

Question 13

ROC Curve

Answer 13

Plots true positive rate (recall) against false positive rate. A good ROC curve goes toward the top left of the chart.

X = FPR
Y = TPR

Question 14

Specificity

Answer 14

TNR

Ratio of negative instances that are correctly classified as negative

Question 15

Lasso Regression

Answer 15

Check this

Question 16

Elastic Net

Answer 16

Check this

Question 17

Early Stopping

Answer 17

A way of regularizing a model by stopping training once validation error reaches a minimum

Question 18

Soft max Regression

Answer 18

Also known as multinomial logistic regression.

Classification with multiple classes. For each instance x, assigns a score s(x) for each class k, then estimates probability by applying a soft max function.

Soft max function is as followed:

Exp(s(x)) / summation(exp(s(x)))

Question 19

Cross-entropy

Answer 19

Loss function used to measure difference between predicted and true probability distributions. Penalizes low probability on true labels significantly.

-1/m ∑ ∑y log (p(k))

Essentially the mean of the -log(estimated probabilities).