ML-01 - ML-01-Introduction and linear regression

Question 1

ML-01 - Introduction and linear regression

When did Arthur Samuel come up with his definition of machine learning?

Answer 1

The 1950s.

Question 2

ML-01 - Introduction and linear regression

What as Arthur Samuel’s definition of machine learning?

Answer 2

“[…] the field of study that gives computers the ability to learn without being explicitly learned/programmed.”

Question 3

ML-01 - Introduction and linear regression

How did Tom Mitchell define machine learning?

Answer 3

Machine learning is a field of study which enables a computer program
learn from experience 𝑬 with respect to some task 𝑻 in a well-posed problem
and some performance measure 𝑷, and improves the performance 𝑷 with experience 𝑬.

Question 4

ML-01 - Introduction and linear regression

What are the 3 broad types of machine learning?

Answer 4

• Supervised learning
• Unsupervised learning
• Reinforcement learning

Question 5

ML-01 - Introduction and linear regression

What are the two big types of supervised learning?

Answer 5

• Classification
• Regression

Question 6

ML-01 - Introduction and linear regression

What are the two big types of unsupervised learning?

Answer 6

• Clustering
• Dimensionality reduction

Question 7

ML-01 - Introduction and linear regression

Describe the difference between regression and classification.

Answer 7

Regression predicts continuous values, while classification predicts discrete categories.

Question 8

ML-01 - Introduction and linear regression

What is Semi-supervised learning?

Answer 8

A type of ML approach where you have some labeled data, but lots of unlabeled data.

Question 9

ML-01 - Introduction and linear regression

What is reinforcement learning?

Answer 9

Learning by interacting with the environment.

Question 10

ML-01 - Introduction and linear regression

What are the 5 steps for a supervised learning workflow?

Answer 10

1) Get data
2) Clean, prepare, manipulate
3) Train the model
4) Test data
5) Improve

Question 11

ML-01 - Introduction and linear regression

What are the two most common optimization methods?

Answer 11

• Iterative methods, like gradient descent.
• Non-iterative methods, like the least squares method.

Question 12

ML-01 - Introduction and linear regression

Describe gradient descent.

Answer 12

Gradient descent works by following the gradient of a function to reach a minimum.

Question 13

ML-01 - Introduction and linear regression

What is the formula for gradient descent?

Answer 13

(See image)

Question 14

ML-01 - Introduction and linear regression

What are the 3 typical variants of gradient descent?

Answer 14

• (Batch) gradient descent
• Mini-batch gradient descent
• Stochastic gradient descent

Question 15

ML-01 - Introduction and linear regression

Describe (batch) gradient descent.

Answer 15

use the entire training samples in each iteration (called epoch) of gradient descent.

Question 16

ML-01 - Introduction and linear regression

Describe mini-batch gradient descent.

Answer 16

Instead of learning on all data per epoch, learning happens on subsets of the data. During each epoch, N / batch_size samples are selected without replacement. For each batch, the model is updated.

Question 17

ML-01 - Introduction and linear regression

Describe stochastic gradient descent.

Answer 17

During each epochs, set the batch size to 1 and update on each training example.

Question 18

ML-01 - Introduction and linear regression

How do you make sure you set the learning rate correctly?

Answer 18

Plot loss vs. the number of epochs and make sure the loss converges after some number of iterations.

Question 19

ML-01 - Introduction and linear regression

What happens if the learning rate is too high?

Answer 19

Loss might not decrease on every iteration and the training won’t converge.

Question 20

ML-01 - Introduction and linear regression

What happens if the learning rate is too low?

Answer 20

The learning takes a long time to converge.

Question 21

ML-01 - Introduction and linear regression

Describe the training workflow steps.

Answer 21

(See image)

Question 22

ML-01 - Introduction and linear regression

What is feature scaling?

Answer 22

A transformation of some data to minimize the effects of different scales.

Learning rates are sensitive to unnormalized data.

E.g. house prices are a lot higher than the number of square meters, and the corresponding coefficients might be disproportional.

Question 23

ML-01 - Introduction and linear regression

Describe visually what happens in feature scaling.

Answer 23

(See image)

The unnormalized data makes the learning curve jump around and it doesn’t converge nicely.

Question 24

ML-01 - Introduction and linear regression

What are the two most commonly used normalization methods?

Answer 24

• Min-max normalization
• Standardization (z-score)

Question 25

ML-01 - Introduction and linear regression

What’s the formula for min-max normalization?

Answer 25

(See image)

Question 26

ML-01 - Introduction and linear regression

What’s the range of data after applying min-max normalization?

Answer 26

Between 0 and 1.

Question 27

ML-01 - Introduction and linear regression

What’s the formula for standardization normalization?

Answer 27

It’s not bounded and might have outliers.

Question 28

ML-01 - Introduction and linear regression

What are some reasons to apply min-max normalization rather than standardization?

Answer 28

• Data doesn’t follow a normal distribution
• Data must be in a specific range
• Need to keep the original shape of the data, just scaled down
• Outliers are not a significant concern

Question 29

ML-01 - Introduction and linear regression

What are some reasons to apply standardization rather than min-max normalization rather? (4)

Answer 29

• When the data follows a normal distribution
• When the data has (extreme) outliers
• When your algorithm assumes standardized data
• When the range of your data is unknown or changing over time

Question 30

ML-01 - Introduction and linear regression

What is polynomial regression?

Answer 30

Using linear regression (i.e. weights for parameters) where you feature engineer the data through polynomial transformations.

(See image)

Question 31

ML-01 - Introduction and linear regression

What’s the image an example of? (See image)

Answer 31

Polynomial regression.

Question 32

ML-01 - Introduction and linear regression

What is the “normal equation”?

Answer 32

A non-iterative optimization method for solving the parameters for linear regression directly.

Question 33

ML-01 - Introduction and linear regression

What is the formula for the “normal equation”?

Answer 33

(See image)

Question 34

ML-01 - Introduction and linear regression

What’s the solution to the “normal equation”?

Answer 34

(See image)

Question 35

ML-01 - Introduction and linear regression

What are the advantages of gradient descent over the normal equation?

Answer 35

• It works well when the number of features is large.

Question 36

ML-01 - Introduction and linear regression

What are the advantages of the normal equation over gradient descent?

Answer 36

• No need to choose learning rate.
• Doesn’t iterate, solves the inverse (X^T*x)^-1.

Question 37

ML-01 - Introduction and linear regression

What are the disadvantages of gradient descent over the normal equation?

Answer 37

• You need to choose the learning rate.
• GD might run for many iterations.

Question 38

ML-01 - Introduction and linear regression

What are the disadvantages of the normal equation over gradient descent?

Answer 38

• Can be slow if the number of features is high; the time complexity of calculating matrix inverses is O(n^3).