Homework 2

Question 1

Nesterov`s accelerated gradient scheme is often more efficient than the simple momentum scheme because the weighting factor of the momentum term increases as a function of iteration number.

Answer 1

False

Question 2

Weight decay helps against overfitting.

Answer 2

True

Question 3

Pruning helps against overfitting.

Answer 3

True

Question 4

L_1 regularisation reduces small weights less than L_2​​ regularisation.

Answer 4

False

Question 5

For the parity problem with N inputs, one can construct a perceptron that solves the problem with less than
2^N​​ hidden neurons.

Answer 5

True

Question 6

Using a stochastic path through weight space in backpropagation assures that the energy either decreases or stays constant.

Answer 6

False

Question 7

The number of N-dimensional Boolean functions is

2^N.

Answer 7

False

Question 8

How the weights are initialised for backpropagation does not matter (provided that they are not all zero) because one usually iterates for many epochs so that the initial conditions are forgotten.

Answer 8

False

Question 9

When solving a t=+1/−1 problem in two dimensions using a decision boundary in the form of a convex polygon, the resulting output problem may sometimes not be linearly separable.

Answer 9

False