Supervised Learning

Question 1

Model Evaluation Metrics - Which beta to use(high or low) - Fraud Detection

Answer 1

need a high recall, so need a high beta.

Question 2

Model Evaluation Metrics - R2

Answer 2

comparing model MSE from basic model MSE. The idea is that the model MSE should be lower than basic model MSE. If so, the ratio is small and 1 - ratio is close to 1.

Question 3

Ensemble Methods - Introducing Randomness to high variance models before ensembling. Two common options

Answer 3

Bootstrap the data - that is, sampling the data with a replacement and fitting your algorithm to the sampled data.

Subset the features - in each split of a decision tree an ensemble of only a subset of the total possible features are used.

Question 4

Decision Tree - Entropy - How to turn a bunch of products into sums? To make the probability calculate more manageable.

Answer 4

Take the log of each item and sum everything together

Question 5

Support Vector Machines - Which equation will help us split the data?

Answer 5

The function that splits the data will be x2 + y2 = 10 (10 being in the middle of 2 and 18)

Question 6

Decision Trees - Entropy - Confirm how to calculate probabilities of recreating ball sequence

Answer 6

Since you grab the ball, and put it back each time, these are independent events and probabilities are multiplied by each other. *blue on first row should be zero

Question 7

PyTorch - Batch Size?

Download and load the training data
trainset = datasets.MNIST(‘MNIST_data/’, download=True, train=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=64, shuffle=True)

Answer 7

Each run through the network will use 64 images, then matrix where first column is a flattened vector version of one image

Question 8

Ensemble Methods - Adaboost hyperparameters

Answer 8

base_estimator: The model utilized for the weak learners (Warning: Don’t forget to import the model that you decide to use for the weak learner).

n_estimators: The maximum number of weak learners used.

Question 9

Bayes Theorem - Naive Bayes Fip Step. Use example below

Answer 9

Flip the event and conditional.

P(A | B) becomes P(B|A) * P(A). Think in terms of a diagram.

Question 10

What is feature scaling, two common scalings?

Answer 10

transforming your data into a common range of values. There are two common scalings:

Standardizing

Normalizing

Allows faster converenge, training less sensitive to the scale of features

Question 11

Support Vector Machines - Describe C Parameter

Answer 11

• C hyper-parameter determines how flexible we are willing to be with the points that fall on the wrong side of our dividing boundary
• Constant that attaches itself to classification error
• Large C = forcing your boundary to have fewer errors than when it is a small value. If too large, may not get converence with small error allotment
• Small C = Focus on large margin

Question 12

Ensemble Methods - Explain Bias

Answer 12

When a model has high bias, this means that means it doesn’t do a good job of bending to the data. An example of an algorithm that usually has high bias is linear regression. Even with completely different datasets, we end up with the same line fit to the data. When models have high bias, this is bad.

Question 13

Linear Regression - Describe the Square Trick

Answer 13

Its the absolute trick and some. Multiply the distance of the point from the line against the scaled slope and y-intercept. More smart as it gives the line a smarter distance to change to get closer to the point.

Question 14

Answer 14

make a prediction, calculate the error, update weights and bias with gradient of error(scaled by learning rate)

Question 15

Linear Regression - Describe the absolute trick

Answer 15

Adding values to the slope and y intercept to make the line come closer to points. The value added to the slope should be the horizontal distance(p) and the value added to the y-intercept is arbitrary, but typically use 1. Then, must down scale these added values by a learning rate so the line doesn’t overshoot the point.

Question 16

Support Vector Machines - Describe Margin Error

Answer 16

norm of the vector W squared. AKA, square all coefficients and sum. You want a small error as this indicates a larger margin.

Question 17

Support Vector Machines - Describe the Kernel Trick

Answer 17

Transforming data from lower dimensions to higher dimensions in order to split with higher dimensional hyperplane. Then, project back to lower dimensional world with polynomial of certain degree.

Question 18

Describe Lasso Regularization

Answer 18

Allows for feature selection

Formula squishes certain coefficients to zero, while non zero coefficients inidcate relevancy

use an alpha(lambda) multiplied by sum of the absolute value of each coefficient. Adds this to error

Question 19

Ensemble Method - Boosting Simple Example

Answer 19

Instead of just taking most common answer, use answers from friends who are well versed in each question. Use answer from philospher friend for philosohy question, use answer from sports friend for sports question etc

Question 20

Support Vector Machines - Define gamma in terms of sigma

Answer 20

If gamma is large, the sigma is small(curve is narrow). Vice Versa

Question 21

Ensemble Methods - Basic idea of Random Forest

Answer 21

Take Subset of data and build decision tree of these columns. Repeat process with other random subset, then use most popular prediction as the prediction

Question 22

Model Evaluation Metrics - What is recall

Answer 22

Question 23

Ensemble Methods - High level what are they and name two popular options

Answer 23

Take a bunch of models and join together to get a better model

Bagging(Bootstrap aggregating) and Boosting

Question 24

Bayes Theorem - Naive Bayes - Normalize Step

Answer 24

Take conditional probabilities for all events(Spam and Ham), then normalize the values. (each probabilty over the sum of possible probabilities)

Question 25

Preprocessing - What to do in this scenario

Answer 25

Subtle, but you only need two columns since there are only 3 possibilities. If you include all three, you are duplicating data and certain models may have trouble

Question 26

Decision Trees - Describe Entropy

Answer 26

How much freedom do you have to move around

Question 27

PyTorch - What does the transform function do? transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)), ])

Answer 27

first transform images to tensors, then convert pixel values from 0-1 range to a -1 to 1 range You are subtracting the mean(0.5) from each color channel(3), then dividing (0.5) from each color channel. Ensures variance is zero centered which makes learning easier

Question 28

Ensemble Methods - Bagging Simple example

Answer 28

Have all our friends take a true/false test and for each question use the most common answer

Question 29

Describe Gradient Descent

Answer 29

Take the derivative of an error function and move in the negative direction. The negative direction gives us the fast way to decrease the error function

Question 30

Support Vector Machines - Describe Margin and Classification Error

Answer 30

When linearly separating data, Margins maximize the distance from the linear boundary to the closest points (called the support vectors). Incorrectly classifed points within the Margin is the Margin Error Any errors outside the margin are considered classification errors.

Question 31

Training and Tuning - K Fold Cross Validation Shuffle

Answer 31

Instead of equal splits, randomly creating different training and testing buckets K times

Question 32

Model Evaluation Metrics - Quiz

Answer 32

Question 33

Model Evaluation Metrics - ROC- What does score rerpresent

Answer 33

Area under ROC.

Question 34

Support Vector Machines - Where would SVM make cuts?

Answer 34

two lines to maximize margin

Question 35

Model Evaluation Metrics - F1 score

Answer 35

The harmonic mean of recall and precision Will always be lower than arithmetic mean, so which ever score is lower, it will be closer to that, and thus raise a "red flag"

Question 36

Support Vector Machines - Describe Classification Error

Answer 36

* Split data with line that represents Mx + b = 0, * Add margin lines = Mx + b = -1, Mx + b = 1 * From Margin lines, create lines going up and down. * Find incorrectly classified points inside and out margins, * Associate a value based on point location, add all together

Question 37

Ensemble Methods - Downside of Random Forests

Answer 37

They are random, there are better ways to choose which data to subset

Question 38

Linear Regression - What variables can you change to move a line

Answer 38

Slope and Y intercept

Question 39

Model Evaluation Metrics - Create an Accuracy Function

Answer 39

Question 40

Support Vector Machines -What is it and name three popular versions?

Answer 40

popular algorithm used for classification problems 1. Maximum Margin Classifier 2. Classification with Inseparable Classes 3. Kernel Methods

Question 41

Model Evaluation Metrics - When Accuracy is not good

Answer 41

If your accuracy is high, but your not detecting errors. Can occur when data is skewed with high number of positive versus low number of errors

Question 42

Decision Trees - Entropy - How to calculate probability of independent events if there are 5,000. Whats the downside?

Answer 42

Multiply each event, computationally expensive, small changes in one value can lead to large changes in outcome. We want something more manageable

Question 43

Bayes Theorem - Discuss Naive Bayes

Answer 43

Involves multiple events and assumes independence For P(A & B), we assume events are independent. If they were depependent, they couldn't occur together. Think P(being HOT & Cold). This can't happen, however, Naive says they can. Just multiplying all events together, multiplying by the "given" and normalizing ratio.

Question 44

Support Vector Machines - Gamma Parameter

Answer 44

Small = wide RBF, may underfit, may generalize better Large = narrow RBF, may overfit. Similiar to Large C in classificaiton where it attempts to classify every point correctly

Question 45

two categories of supervised learning

Answer 45

Question 46

Training & Tuning - Describe Model Complexity Graph

Answer 46

One one end, your model underfits the data(high bias and doesn't do well on either training or validation) On other end, your model overfits(high variance, too complex, fits training data well but doesn't generalize well In middle, your model does pretty good on training and validation. Look for models where validation error is increasing but training error is reducing.

Question 47

Ensemble Methods - What are common default weak learners

Answer 47

Decision Trees

Question 48

What is normalizing?

Answer 48

data are scaled between 0 and 1 Value - min/ max-min

Question 49

Decision Tree - Entropy - Simplified Entropy Equation

Answer 49

probabilty \* log of probability sum across and take negative value.

Question 50

Ensemble Methods - Bagging Describe in more detailed

Answer 50

Take random cuts of data(weake learners), then superimpose over each other and vote(if two or more are red then red, two or more are blue, then blue) Model will cut data according to votes

Question 51

Visualize Mean Absolute Error

Answer 51

Question 52

Model Evaluation Metrics - Purpose behind an ROC curve

Answer 52

provide a score that shows how well we split the data. 1 for perfect, .5 for random and above .5 for anything else

Question 53

Ensemble Methods - Adaboost weighting

Answer 53

* weight all data points at 1 * minimize sum of weights of incorrectly classified points * After first cut, calculate weight by taking natural log of correct/incorrect * multiply incorrect weights by the weight * Repeat

Question 54

What type of gradient descent is used most often

Answer 54

Mini - Batching - Split data into mini batches of equal size, update weights based on each mini batch Calculating error for ALL points(either by batch or one by one(stochastic) is slow

Question 55

Training and Tuning - Learning Curves

Answer 55

Method of detecting if model is overfitting or underfitting. As more data points are used, training error increases and CV error descreases. Look at convergence point to determine if model is overfitting or underfitting

Question 56

Bayes Theorem - High Level Description

Answer 56

Involves a Prior and Posterior Probability. Use new information to update prior, this becomes the posterior.

Question 57

Decision Trees - Entropy described by probability

Answer 57

How much freedom do you have to move around or rearrange the balls

Question 58

Ensemble Methods - Explain Variance

Answer 58

When a model has high variance, this means that it changes drastically to meet the needs of every point in our dataset. Linear models like the one above is low variance, but high bias. A decision tree, as a high variance algorithm, will attempt to split every point into it's own branch if possible. This is a trait of high variance, low bias algorithms - they are extremely flexible to fit exactly whatever data they see.

Question 59

Ensemble Methods - Adaboost high level

Answer 59

1. Split data to minimize errors 2. punish misclassified points and use a weak learner to focus on these points. Fit this line 3. Repeat step 2 for first weak learner 4. Combine and fit

Question 60

Ensemble Methods - Combining Weights

Answer 60

Superimpose all weak learner models For each weak learner model, input the positive and negative weight value accordingly. Where sum for each region is positive, then classify positive, where negative, classify negative

Question 61

Support Vector Machines - Where would SVM split and how?

Answer 61

Use kernel trick to move from 1-D line to 2-D(Plane) where points are placed on a parabola instead of a line. Find line that cuts the parabola cleanly, equate the line to the parabola and solve. These is where SVM makes cuts.

Question 62

Support Vector Machines - RBF(Radial Basis Function)

Answer 62

* using functions to build mountains over each point * record values in a vector of all mountains over each point. * plug them into higher dimensional space * find equation of hyperplane that splits data * Take constants of the equation of the plane * plug points at these constants and find line that splits dat (Where hyperplane intersects mountains)

Question 63

Model Evaluation Metrics = What is precision of this model

Answer 63

Question 64

What is standardizing

Answer 64

Taking each value of your column, subtracting the mean of the column, and then dividing by the standard deviation of the column. interpreted as the number of standard deviations the original value was from the mean.

Question 65

Training and Tuning - K-Fold Cross Validation

Answer 65

Split data into Training and Testing K Times. Each pass the bucket of training and testing is different. Take average result for all runs

Question 66

Support Vector Machines - Describe the degree of the polynomial kernel. Describe a degree 3 polynomial kernel

Answer 66

A hyperparameter we use during training to find best possible model

Question 67

Support Vector Machines - What is a kernel? Describe different kernels

Answer 67

Set of functions that will come to help us out. Linear Kernal - can only use x and y to create a line which separates data Polynomial Kernel - Add, xy x2 and y2. Can create many more functions to separate data RBF Kernel - Build mountains over each point

Question 68

PyTorch - Describe the dimension in the F.softmax function ``` def forward(self, x): ''' Forward pass through the network, returns the output logits ''' ``` x = self.fc1(x) x = F.relu(x) x = self.fc2(x) x = F.relu(x) x = self.fc3(x) x = F.softmax(x, dim=1)

Answer 68

asks which dimension of the tensor dim 0 - the batch size dim 1 - vector of images. This is the dim we want

Question 69

Decision Trees - Hyperparmaters - Describe Maximum Depth

Answer 69

largest length between the root to a leaf. A tree of maximum length kk can have at most 2k2k leaves.

Question 70

Model Evaluation Metrics - Discuss Boundaries of Beta and impact on precision and recall

Answer 70

Question 71

Log Transformation - Provide example

Answer 71

Question 72

Model Evaluation Metrics - How is ROC calaculated

Answer 72

Calculate True Positive and False Positive Rates for all splits of data. Then plot

Question 73

Answer 73

must drop the point values into the equation to determine q prime

Question 74

Support Vector Machines - What does Sigma relate to in a normal distribution

Answer 74

The width of the mountain/curve

Question 75

Model Evaluation Metrics - F-beta score

Answer 75

Used when you want your model to care more about either precision or recall. Its a weight added to the F1 score to swing the value either way

Question 76

3 main categories of machine learning

Answer 76

Question 77

Decision Trees - Entropy - Once you have the sum of the negative logs, what is the next step

Answer 77

Take the average

Question 78

Support Vector Machines - Margin Error - Describe W Vector

Answer 78

* Used in distance calculation between two lines * Random vector that runs from orgin and intersects second line * Based on intersection points(p,q) and the equation of the line the vector intersects (Wx = 1), the 1/|W| square represents the distance from Wx = 0 to Wx = 1. Multiply by 2 since lines are equidistant and 2/|W| squared represents the Margin Error.

Question 79

Model Evaluation Metrics - Type 1 and 2 errors

Answer 79

Question 80

Decision Trees - Entropy - Why take the negative log of each probability event

Answer 80

Since probabilities are less than 1, the log will be negative. Thus, to turn the values to positive, we take the negative log

Question 81

Decision Trees - Hyperparameters - Describe minimum number of samples per leaf

Answer 81

Question 82

Decision Trees - Hyperparameters - Maximum Features and Minimum Number of samples per split

Answer 82

min num on split - gotta have at least x amount before you can split Maximum Features -

Question 83

Two specific cases to use feature scaling

Answer 83

* When your algorithm uses a distance based metric to predict. * If you don't, then predictions will be misleading * When you incorporate regularization. * if you don't, then you unfairly punsih features with smaller or larger ranges

Question 84

Ensemble Methods - Weak versus Strong Learners

Answer 84

Weak learners = our friends who take test Strong Learner = Genius who combins all answers

Question 85

Two common error functions in linear regression

Answer 85

Mean Absolute Error - Make all errors positive so the negatives don't cancel each other out. Mean Squared Error - Take all errors and square them to make them non-negative. This gives you the area of a sqaure around each point. Sum and Average than multiply by 1/2 to facilitate taking derivative

Question 86

Support Vector Machines - Describe the x2 + y2 = 10 in single and multiple dimensions

Answer 86

Question 87

Model Evaluation Metrics - Precision

Answer 87

Accuracy of Diagnosed Positive Group

Question 88

Describe Supervised Learning

Answer 88

observing and associating patterns of labeled data, take this training and assingn labels to new and unlabeled data.

Question 89

Model Evaluation Metrics - Recall

Answer 89

Accuracy of Positive Group

Question 90

Decision Trees - Entropy describe by knowledge

Answer 90

Less entropy = less room to move around = more knowledge you have

Question 91

Model Evaluation Metrics - Describe a confusion matrix

Answer 91

Rows = Positive versus negative Columns = Guessed Positive, Guessed Negative

Question 92

Decision Trees - Information Gain - How to calculate?

Answer 92

* Change in Entropy between part node and children node * Parent Entropy is always 1 * Take weighted average of the children

Question 93

Bayes Theorem - Known versus Inferred?

Answer 93

Known You know a P(A) and you know P(R | A) Inferred Once we know the event R has occurred, we infer P(A | R) Find conditional probability of event and divide into possible events that have occurred.

Question 94

Answer 94

Since the point is below the line, the intercept decreases; since the point has a negative x-value, the slope increases. If point was above line, then you would add the alpha and p\*alpha

Question 95

Decision Trees - Hyperparameters - Impact on overfitting/underfitting for small/large samples per leaf and small large depth

Answer 95

Large depth very often causes overfitting, since a tree that is too deep, can memorize the data. Small depth can result in a very simple model, which may cause underfitting. Small minimum samples per leaf may result in leaves with very few samples, which results in the model memorizing the data, or in other words, overfitting. Large minimum samples may result in the tree not having enough flexibility to get built, and may result in underfitting.

Question 96

Explain Batch vs Stochastic Gradient Decent

Answer 96

Batch - Calculate error for all points, then update weights Stochastic - calculate error for one point, then update weights

Question 97

Bayes Theorem - Naive Bayes - Be Naive Step

Answer 97

Split into a product of simple factors. Then, multiply by Prob of Event Do this for all possible events (Spam & Ham)

Question 98

Vizualize Mean Squared Error

Answer 98

Question 99

Decision Trees - Entropy - Formula - Describe the formal notation

Answer 99

1. find prob of each event 2. take negative log 3. multiple by occurences of event 4. Take average 5. Repeat for each probability 6. Sum

Question 100

Negative Indexing - What is the difference between the following: X = data[: , :-1] y = data[: , -1]

Answer 100

X will grab all rows and all columns except the last Y will grab all rows and just the last column

Question 101

Log-Transformation of Skewed Data - Why do it?

Answer 101

so very large and very small values do not negatively affect the performance of a learning algorithm. Using a logarithmic transformation significantly reduces the range of values caused by outliers. Care must be taken when applying this transformation however: The logarithm of 0 is undefined, so we must translate the values by a small amount above 0 to apply the the logarithm successfully.

Question 102

Support Vector Machines - Describe the photo below in relation to gamma

Answer 102

Large Gamma = trying to classify every point Small Gamma = Clusters