Chapter 5 : Machine Learning

Question 1

What is machine learning?

Answer 1

Computer algorithms that can learn from data to make determinations or predictions on new data without being explicitly programmed

Question 2

Why isn’t search always effective

Answer 2

1. Can’t deal with new data

2. Deal with unforeseen circumstances

Question 3

What are different types of ML models

Answer 3

1. Classifier (Chooses an output)

2. Regressor (Generates an output)

Question 4

What is difference between discriminative and generative models?

Answer 4

Discriminative models draw boundaries in data space, while generative models attempt to map out the distribution of the model.

Question 5

What are evaluation metrics? Which one is often best to use?

Answer 5

Precision, Accuracy, Recall are main ones but can sometimes give misleading results. F1-Score which is a mix of precision and recall often gives the most meaningful results

Question 6

What is ROC curve?

Answer 6

Can tell how well classifier is working by compare true positive and false positive

Question 7

What is k-cross fold validation?

Answer 7

Split data into N-1 folds for training/validation and 1 fold for testing

Question 8

What is leave-one-out validation?

Answer 8

Pick N-1 data points for training/validation and 1 last point for testing

Question 9

What is the no free lunch theroem?

Answer 9

There is no one machine learning algorithm that can be applied to all problems, different models must be tested.

Question 10

What is regression?

Answer 10

Fitting data onto a polynomial curve

Question 11

How do we train regression?

Answer 11

Minimizing error function

Question 12

What is univariate linear regression?

Answer 12

A regression model with N = 1 and with 1 variable

Question 13

What is the time complexity of gradient descent?

Answer 13

O(n^3)

Question 14

Possible outcomes of gradient descent?

Answer 14

1. Converges
2. Diverges
3. Oscillates

Question 15

Logistic Regression

Answer 15

Used for classification and giving probability

Question 16

What is Naive Bayes

Answer 16

A generative classification model

Question 17

Can Naive bayes be used in regression?

Answer 17

True

Question 18

What is key assumption for Naive Bayes to work?

Answer 18

All features are independent of one another

Question 19

What is Overfitting?

Answer 19

When a model has a low error rate in training but then a high error rate in testing.

Question 20

What is generalization?

Answer 20

Notion of learning from some data to make conclusions based on unseen/excluded/new data

Question 21

How to prevent overfitting?

Answer 21

1. Lower order models
2. More data
3. Regularization

Question 22

What is bias-variance tradeoff?

Answer 22

Variability: Variability of model with respect to inputs
Bias: Difference between average model and average of the target data

Question 23

Can K-nearest neighbor be used as a regressor?

Answer 23

True

Question 24

How should k be chosen in K-nearest neighbour?

Answer 24

1. K should be odd to avoid ties
2. Not too small or else overfit
3. Not too large or else underfit

Question 25

Advantages/Disadvantage of kNN algorithm?

Answer 25

Advantage: Easy to train, intuitive algorithm Disadvantage: Computationally expensive as dataset grows

Question 26

What is support vector machine?

Answer 26

Binary Classifier that splits feature space by hyperplane. Goal is to find kernelized max-margin hyperplane.

Question 27

What do we do if data is not linearly separable?

Answer 27

Use kernel trick to project data into higher space

Question 28

What is entropy?

Answer 28

The measurement of uncertainty (fair coin has high entropy, bias coin has low entropy)

Question 29

Does the order of features in binary tree make a different?

Answer 29

True

Question 30

What is ensemble learning?

Answer 30

Create several different classifiers for a problem. Samples (data) are run through all of them and outcome is determined by average or voting.

Question 31

What are two most common methods of ensemble learning?

Answer 31

Bagging and boosting

Question 32

What is bootstrapping in bagging method?

Answer 32

The concept of randomly sampling from a dataset with replacement, to increase size of dataset

Question 33

What is the assumption of classifiers using ensemble learning?

Answer 33

- Slightly better than chance - Somewhat different - High variance

Question 34

What is Boosting?

Answer 34

Does not use bootstrap sampling instead weak classifiers are trained based off of previous iterations of classifiers

Question 35

How does Adaboost prevent overfitting?

Answer 35

Gives more weights to classifications that were done incorrectly.

Question 36

What are random forests?

Answer 36

Use many decision trees to make a decision

Question 37

What is bagged tree ensemble?

Answer 37

A number of trees are created through bootstrapped-sampled data. The final classification is based on a vote or average.

Question 38

Do boosted trees sample data?

Answer 38

While rare, it is possible using scholastic gradient boosted trees or XGBoosted trees

Question 39

What are the main fusion strategies of datasets?

Answer 39

1. Input level fusion 2. Feature-level fusion 3. Score-level fusion

Question 40

How is complexity of an ANN (Artificial Neural Network) Determined?

Answer 40

Hidden layers

Question 41

What are some of the parameters trained in regular ANN?

Answer 41

1. Type of network 2. Number of layers 3. Transfer functions

Question 42

What is Regularization?

Answer 42

Technique to avoid overfitting

Question 43

What is the lambda symbol do in regularization?

Answer 43

Reduces overfitting (variance) but increases bias (underfitting)

Question 44

What does p symbol mean in regularization?

Answer 44

Determines which type of regularization is done (p = 1, p = 2 most common)

Question 45

What is K-means?

Answer 45

Unsupervised machine learning algorithm that clusters data

Question 46

How do we train K-means?

Answer 46

1. Select k 2. Randomly select centroids 3. Assign data point to closest centroid 4. Re-calculate center of centroid 5. Repeat 3,4 until convergence

Question 47

How do we choose K in K-means?

Answer 47

1. Stop after # of iterations | 2. When the algorithm change in error becomes very small

Question 48

What is CNN?

Answer 48

Convolution Neural Network with many hidden layers. Requires large datasets and large computing resources.

Question 49

Other than convolution layer, what are other types of layer in CNN?

Answer 49

1. Pooling layer (max pooling, average pooling) | 2. Fully connected layers

Question 50

What are some parameters that can be trained in CNN?

Answer 50

1. Number of filters 2. Stride 3. Linear Spatial Extent 4. Batch size

Question 51

Is filter proffered to be odd size or even size?

Answer 51

Odd size

Question 52

If filter size is N how much does the size of output decrease?

Answer 52

By N/2 on each side of output. Output can retain it's size if it uses zero-padding.