Terms

Question 1

Supervised Learning

Answer 1

Algorithms are trained using well labeled training data

Question 2

Methods of solving linear regression

Answer 2

Singular Value Decomposition and QR Decomposition

Question 3

Difference between stochastic and gradient descent

Answer 3

For gradient descent, use all samples in training set to calculate loss.

In stochastic, use only one or a subset of training sample to calculate loss.

Question 4

Mean Square Error (MSE) for evaluating regression models

Answer 4

Measures how close a regression line is to a set of data points

Question 5

Root Mean Squared Error for evaluating regression models

Answer 5

Shows how far predictions fall from measured true values

Question 6

Bias

Answer 6

Error introduced by approximating the true underlying function, which can be quite complex, by a simpler model

Question 7

Low Bias

Answer 7

Fewer assumptions are taken to build the target function. So the model will closely match the training dataset

Question 8

High Bias

Answer 8

More assumptions are taken to build the target function. Model will not match the training dataset closely. So underfitting occurs

Question 9

Ways to reduce high bias

Answer 9

• Use a more complex model: model too simple
• Increase the number of features: make more complex
• Reduce regularization of the model: since regularization decreases/prevents overfitting which is not what you want here since high bias causes underfitting.
• Increase the size of the training data: provides the model with more examples to learn from the dataset

Question 10

Mean Absolute Error (MAE) for evaluating regression models

Answer 10

Measures of the average size of the mistakes in a collection of predictions w/out taking their direction into account

Question 11

Coefficient of determination (R^2)

Answer 11

Measures how well a statistical model predicts an outcome (from 0 to 1)

Question 12

Variance

Answer 12

It tells us how much a random variable is different from its expected value as you move from one training set to another. Shows how the performance of a model changes when trained on different subsets of training data

Question 13

What is overfitting?

Answer 13

Increased model complexity and so low bias and high variance.

So model does well on training set but can’t generalize to test set

Question 14

Underfitting

Answer 14

Simpler model. So high bias and low variance.

Question 15

Role of training set

Answer 15

Used to fit the model: train the model with data

Question 16

Role of validation set

Answer 16

Provide unbiased evaluation of a model while fine tuning hyperparameters.

Improves generalization of the model.

Question 17

Role of test set

Answer 17

Data model has never seen before.
Allows for an unbiased evaluation of the model.

Question 18

Cross validation

Answer 18

Separate your total training set into subsets: training and validation set. Evaluate and choose hyperparameters.

Do this iteratively, select different training and validation sets to reduce bias that would occur by selecting only one validation set

Question 19

K-fold cross validation and how big should k be

Answer 19

Cross validation method but dataset is divided into k parts. Each part has one validation set and k-1 training sets.

4 - small datasets
5- large ones

Question 20

When do we use logistic regression

Answer 20

Binary Classification like a churn model

Still a linear model because our come depends on the sun of the inputs and parameters not the product or quotient

Question 21

What is a sigmoid function

Answer 21

Activation function that limits output to between 0 and 1

Question 22

List evaluation metrics for classification methods

Answer 22

Accuracy, Precision, Recall, F1 score, Logistic/Cross Entropy Loss