Past Papers

Question 1

What are hyper parameters?

Answer 1

Set before training

Be sure to specify that they DEFINE network’s architecture

Question 2

For an L¹ regularised neural network, write down how the regularisation term changes the way the parameters are modified during back prop

Answer 2

Question 3

For an L¹ regularised neural network, write the loss function expanded around θ* and show what the minimum is for θi

Answer 3

Question 4

For an L² regularised neural network:

Write down how regularisation term changes the way the parameters update

Expand around the min of the loss function

Write down an expression for the ith component of the minimum

Answer 4

Question 5

When doing questions with L¹ remember to

Answer 5

Mention that we are introducing sparsity to the solution -> some parameters will go to zero if they are not significant

Question 6

Describe what early stopping does

Answer 6

At each iteration of early stopping, we check how the validation or test set errors behave. After p (patience) consecutive iterations where the test error gets worse, the algorithm terminates

Question 7

GPT4o definition of Universal Approximation Property

Answer 7

A feedforward NN with a single hidden layer containing a finite number of neurons can approximate any continuous function cation on a compact subset of R^d, given an appropriate activation function

Question 8

What is the capacity of an infinitely wide neural network with a single hidden layer?

Answer 8

By the UAP, this network can approximate any function and therefore has infinite capacity

Question 9

When asked to compare loss functions?

Answer 9

Remember to check wether the functions are bounded & wether they are differentiable

Question 10

When discussing MSE?

Answer 10

Remember to mention that it is preferred for regression

Question 11

How does L² regularisation change the way the parameters are updated using backprop?

Answer 11

Question 12

When finishing values that minimise MSE for linear regression

Answer 12

Mention linear independence of system of derivatives with respect to β

Question 13

How many times are parameters updated ?

Answer 13

N * (1 - validation_split) = num of training samples (X)

X / batch size = num of batches (B)

B * epochs = number of parameter updates

Question 14

Total # of parameters in a NN

Answer 14

Question 15

UAP conditions on g

Answer 15

Maps R to R
Measurable
Non polynomial
Bounded on any finite interval
The closure of the set of all discontinuities of g in R has zero lebesgue measure

Question 16

UAP property 1

Answer 16

Question 17

UAP property 2

Answer 17