Deep Learning Flashcards
What is batch normalisation?
Normalising the output of each layer in order to speed up training
Remove the mean and scale by the standard deviation,
We don’t care about the magnitude of the output just the shape of it
What is dataset augmentation?
This is a regularisation technique
This is where we create fake data and add it to the training set
Why do we add a small amount of noise to the network weights?
It is a regularisation technique to prevent over-fitting
What is early stopping?
A regularisation technique where we stop training early
The number of training samples can then be used as a hyper parameter
What is Sparse Representation?
A regularisation technique where we force the hidden representation to have more zeroes
What are Ensemble Methods?
A regularisation technique, where we train multiple models at the same time and let them vote on the final output
What is dropout?
A regularisation technique, where we randomly remove a percentage of neurons during training
What is Local Connectivity in the context of CNNs?
The idea that each neuron inside a layer is only connected to a small region in the previous layer, called a receptive field
What is weight sharing in the context of a CNN?
The idea that a single filter (small number of weights) can be used to detect a feature in any part of the image
What is stride in a CNN?
The size of the step we take when sliding the filter across the previous layer
What is the size of the output after convolving a FxF filter on an NxN image using a stride of S?
A= (N-F)/S + 1
(A,A)
we need to choose values of N,F and S such that A is an integer
