Week 10

Question 1

What is the Loss Function?

Answer 1

loss(hw) = sum ((yj - hw(xi))^2)

Question 2

What is the Perceptron Learning Rule?

Answer 2

wi = wi + axi(y - hw(x))

Question 3

How are the weights changed for the Perceptron Learning Rule?

Answer 3

If y = 1 but hw(x) = 0:
Make wTx larger so hw(x) outputs 1.
wi increased when xi is positive.

If y = 0 but hw(x) = 1, do opposite.

Question 4

What is an epoch?

Answer 4

The time after which we update the weights of the Perceptron Learning Rule.

One example.

Question 5

What are some issues in Linear Classification?

Answer 5

Hard threshold.

Function is not differentiable - weights learning could be unpredictable.

Question 6

What is Under/Overfitting?

Answer 6

Underfitting: Model not doing well on seen data.

Overfitting: Model trained too specific on training data (not good for generalisation).

Question 7

What is Regularisation?

Answer 7

Achieve generalisation in ML.

Require loss and regularisation paramater lamda.

Question 8

What is Drop Out?

Answer 8

Encourages network to adapt, not memorise.

Drop a certain number of weights.

Question 9

What are the benefits of Stochastic gradient descent?

Answer 9

Quicker to converge in practice.

Helps to avoid overfitting.