Supervised Machine Learning – Regression and Classification

Question 1

What are 2 main types of machine learning?

Answer 1

Supervised and unsupervised learning

Question 2

What is supervised learning and what are two main types of it?

Answer 2

Supervised learning as a type of machine learning where model is trained with input and output data (x, y) from which model is able to predict the output y based on input x that was never used in model training. Two main types are regression and classification.

Question 3

What is regression?

Answer 3

Regression is a form of supervised learning where the model is predicting a specific number (ex. predict weight based on mouse size)

Question 4

What is classification?

Answer 4

Classification is a type of supervised learning where the model is predicting a category for an input (small set of options) – ex. predict if on an image there is a dog or a cat.

Question 5

What is unsupervised learning?

Answer 5

Unsupervised learning is a type of machine learning that tries to identify clusters or structure in unstructured data. Compared to supervised learning, there are no labels that will mark the output, you only have a bunch of input data and algorithm is trying to identify clusters without really knowing what they mean (ex. customer segmentation or defining type of people based on their genome sequence)

Question 6

What types of unsupervised learning do we have?

Answer 6

Clustering – identify clusters
Anomaly detection – ex. used for fraud detection
Dimensionality reduction – reduce a big dataset to a smaller one (compress data)

Question 7

What is the most common supervised algorithm that is used worldwide and how does it work?

Answer 7

That algorithm is linear regression. It tries to fit a straight line through the data where prediction will be located there on the line.

Question 8

How do you mark a specific row in the training dataset?

Answer 8

Question 9

Write down the linear regression function with one variable (unilateral)

Answer 9

f(x(i)) = Wx(i) + b

Question 10

What is the most common error function that is used to calculate parameters of linear regression ?

Answer 10

Squared error function

Question 11

How do you find values W and b in the linear regression function?

Answer 11

You will find it when you find a minimum of squared error function J(W,b).

Question 12

What is the shape of a cost function with 2 parameters and how to visualize it in 2D?

Answer 12

It has a bowl shape. To visualize it in 2D, you can use a contour plot.

Question 13

What is a gradient descent?

Answer 13

Gradient descent is an algorithm that is providing a structured way to minimize the function to a local minimum, in the case of linear regression minimize J(W1, W2, … Wn, b) = W1x + W2x + … Wnx + b.
It starts with some random values of parameters, calculating value of the function for all input/output values, and goes into a direction of steepest descent (adapt parameters) until it gets to the local minima.
There can be multiple local minimas for some non-linear functions like neural networks and in that case it depends from which initial parameters gradient descent has started.

Question 14

What is a learning rate?

Answer 14

It is a simple constant that decides in gradient descent how big the step would be, or how big the changes of params W and b will be in each step. Bigger steps – faster convergence but higher possibility that the algorithm will overshoot the local minimum since the step is too big. It is marked with Greek letter α. The closer you are to the local minimum, slope is smaller which leads to smaller steps (smaller derivative of function J)

Question 15

How to implement gradient descent?

Answer 15

Important thing to note that values W and b are updated simultaneously (at the same time). Incorrect way would be to update first W and then b since b will use a new value of W instead of the old one

Question 16

How to implement gradient descent for linear regression?

Answer 16

Question 17

What is the difference between batch and stochastic gradient descent?

Answer 17

When calculating parameters batch gradient descent is taking the whole training set while stochastic gradient descent is just taking one random training example. SGD is faster and gives good solution but not optimal. Should be used on large datasets.

Question 18

What is feature scaling?

Answer 18

In most cases, values of input features can be quite different like number of bedrooms vs size in square meters. Due to this, contour plot is narrow compared to features that have similar values. That would mean that it will take much more time until the model converges. To avoid this, you should scale your features to the same number range like 0 to 1 or -1 to 1.

Question 19

What are different methods to scale features?

Answer 19

• You can scale features by dividing it with the max value for that feature so you get all features on a scale from 0 to 1
• Mean normalization to center all feature values around 0
• Z-score normalization is the same like mean normalization just the features are mainly between -3 and 3

Question 20

How do you speed up linear regression calculation for multiple features?

Answer 20

You can do it by introducing vectorization – translate all feature and weights to vectors and use a dot product.
f(x) = w*x + b

Question 21

How does a gradient descent algorithm look like in vectorized implementation?

Answer 21

Question 22

What is the alternative algorithm that can be used instead of gradient decent for linear regression?

Answer 22

Normal equation – used only for linear regression, it works much faster when the number of features in < 10.000

Question 23

What are the best practices when it comes to checking gradient descent for convergence?

Answer 23

There are 2 options:
- Plot a function that has values J and # of iterations – values of J should always be decreasing
- Automatic convergence test – Define a small number like 10-3 and if programmatically decide if iteration step is less than that number it means that gradient descent has converged

Question 24

What is a recommended approach to choose learning rate?

Answer 24

Start with 0.001 and increase it 3 times in the next experiment. Stop when you see it is too large!

Question 25

What is feature engineering?

Answer 25

It is a process of creating new features out of existing ones, for example creating a new feature x3 = w1w2 where w1 is longitude and w2 latitude of geolocation. Doing a proper feature engineering can increase performance of your model.

Question 26

What is a polynomial regression?

Answer 26

Polynomial regression is a method of introducing a new feature like x2, x3 or √x to curve a linear regression to fit the training data

Question 27

What is a binary classification?

Answer 27

It is a classification when only two values are possible, positive or negative.

Question 28

Which algorithm is used for classification problems?

Answer 28

Logistic regression

Question 29

What is a sigmoid function?

Answer 29

Sigmoid function is a non-linear function that passes through 0.5 y-axis when x is 0. It looks like:

Question 30

How does a logistic regression algorithm look like?

Answer 30

Question 31

What is a decision boundary?

Answer 31

It is a line that separates positive from negative examples. It can be linear or non-linear (if we adapt the function to be polynomial). Let’s say the value for a threshold is 0.5 where the example is positive if >= 0.5. That would mean that z is positive for positive numbers. On the other hand, negative values are < 0.5 and z is a negative number. To draw a line, since z is actually f(x), you should make linear regression f(x) equal to 0 and then draw a line from that function.

Question 32

Why can’t you use a regular squared error cost function to params value in logistic regression?

Answer 32

Since logistic regression algorithm is different than linear regression, using the same cost function will produce a non-convex function.

Question 33

How does a cost function for logistic regression look like?

Answer 33