Linear Regression

Question 1

What is gradient descent?

Answer 1

an algorithm that tweaks parameters iteratively (individually) in order to minimize a cost function

Question 2

What is batch gradient descent?

Answer 2

instead of computing gradients individually, it computes them all in one go by using the whole training set at each iteration

Question 3

what is the downsides to batch gradient descent?

Answer 3

because it uses the whole set at each step, it is very slow on large sets

Question 4

what does a higher learning rate mean with gradient descent?

Answer 4

fails to find a good solution

Question 5

what does a lower learning rate mean with gradient descent?

Answer 5

takes longer to compute

Question 6

how does gradient descent perform with features with different scales?

Answer 6

it takes longer to reach the minimum, making the algorithm slower

Question 7

how does gradient descent perform with features with same scales?

Answer 7

it goes directly to the minimum without jumping around, making the algorithm faster

Question 8

what algorithm is better to use with Linear Regression out of -> Gradient Descent or Normal Equation when you have a larger dataset? Why?

Answer 8

Gradient Descent because it is faster

Question 9

out of gradient descent and normal equation, which is faster, why?

Answer 9

Gradient descent, because it handles instances one at a time.

Question 10

what is a cost function?

Answer 10

?

Question 11

what is stochastic gradient descent?

Answer 11

as opposed to batch-gd which uses the whole set at each step, sgd picks a random instance and handles them one at a time, making it much faster and better for bigger sets

Question 12

which gd algorithm is better for large sets?

Answer 12

sgd because it handles instances one at a time, instead of using the whole set at each step like bgd

Question 13

what happens when you reduce the sgd’s learning rate slowly?

Answer 13

jumps around for ages

Question 14

what happens when you reduce the sgd’s learning rate quickly?

Answer 14

get stuck in local minimum or frozen

Question 15

what is mini-batch gradient descent?

Answer 15

computes gradient on small random set (both sgd and bgd)

Question 16

what is polynomial regression?

Answer 16

when the data is too complex for a straight line, you can use powers on every feature and add them as new features, then use linear regression

Question 17

what is good about polynomial regression?

Answer 17

it finds relationships between features, and can be used when a straight line wont fit the data

Question 18

what are some regularized linear regression models?

Answer 18

ridge, lasso, elastic net

Question 19

what is ridge regression?

Answer 19

a regularized version of linear regression that forces the learning algorithm to git the data, but also keep the model weights as small as possible

Question 20

what happens if the ridges a=0?

Answer 20

it is linear regression

Question 21

what happens if the ridges a=large?

Answer 21

all weights end up very close to zero, resulting in a flat line

Question 22

what is an important step before using ridge regression?

Answer 22

scale!

Question 23

what should you do before ridge regression?

Answer 23

scale!

Question 24

what is lasso regression?

Answer 24

a regularized version of linear regression that eliminates weights of least important features and automatically performs feature selection and outputs a sparse model

Question 25

what is good about lasso?

Answer 25

it automatically performs feature selection

Question 26

what is elastic net?

Answer 26

it is in between ridge and lasso, a mix of both

Question 27

what happens if elastic nets r = 0?

Answer 27

it is ridge

Question 28

what happens if elastic nets r =1?

Answer 28

it is lasso

Question 29

when should you use ridge?

Answer 29

as a default for linear regression

Question 30

when should you use lasso or elastic net?

Answer 30

when only few features are useful because they both try reduce useless features

Question 31

when should you choose elastic net over lasso?

Answer 31

when the number of features is larger than the number of instances, because lasso may behave erratically OR when several features are correlated

Question 32

why would you want to use ridge regression instead of linear regression?

Answer 32

a model with some regularization performs better than a one without, so you should use ridge as a default over linear regression

Question 33

why would you want to use lasso instead of ridge regression?

Answer 33

lasso leads to a sparse model, which is a way to perform feature selection automatically, which is good for you if you suspect that only a few features actually matter. When you are not sure, you should use ridge regression.

Question 34

why would you want to use elastic net instead of lasso?

Answer 34

elastic net is preferred over lasso, however lasso has a hyperparameter that you can use if you want to use it without the erratic behaviour. set it close to 1.

Question 35

when would you choose two logistic regression classifiers over one softmax regression classifier? and vice versa

Answer 35

two when they are not exclusive,

softmax when they are.

Question 36

what suffers from features having different scales?

Answer 36

GD algorithms

Question 37

Can GD get stuck in a local minimum when training a logistic regression model?

Answer 37

no because the cost function is convex

Question 38

what does it mean when there is a large gap the training error and the validation error?

Answer 38

if the validation error is much higher than the training error, it is because your model is over fitting

Question 39

what does it mean when your validation error is much higher than your training error?

Answer 39

the model is overfitting