5 - Fixing Problems

Question 1

What does regularization do?

Answer 1

Regularization makes optimization problems *better-behaved*, by adding something to the cost/loss function

L(theta) = L(theta) + lambda*omega(theta)

where:
L(theta) = loss function
lambda = value between (0,1) that tells us how much we should weight our Betas or Weights
omega(theta) = our regularization term (L1=MAE) or (L2=MSE)

Question 2

What does Normalization do?

Answer 2

Normalization makes optimization *easier* by decreasing the range of values that a feature can take.