Week 5 Flashcards
Why don’t we use regular NN for images?
Doesn’t scale:
100x100 pixels = 10k parameters per node
Not robust to small changes in input
Doesn’t take advantage of correlations between pixels
What are filters in CNN
Subset of the image which acts as the weights that will be learned by the NN via backpropagation
How do we apply filters from CNNs
Do a dot product between filter and original image, store the result with a bias term in a matrix called a feature map. Move the filter 1 pixel across and go again.
What is a feature map
Map of where the feature indicated by your filter appears
Values > 0: filter appears here
Values < 0: does not appear here
Limitations of deep learning
DL is data hungry: it needs a LOT of data to work
DL is heavy: you often need GPUs and cloud computing to train it and even to use it
DL is bad at representing uncertainty: it’s easy to trick a neural network into thinking it’s right
Hard to optimise: architecture, learning method…
Hard to interpret: neural networks are black boxes.
What do we do to the feature map once its made?
Pass the values throguh a ReLU function.
(e.g. its equal to x if x>=0 or its 0 if its less).
Finds all the filter matches.
relu(x) = max(0,x)
Why do we need to downsample
Our feature map is N-2 pixels when N is the width of our image. Does not scale
How do we downsample?
Pooling
What is Pooling?
Aggregating,
Summarising,
Downsampling the image
Max-pooling
Average-pooling
What is stride?
Step size in convolution
Whats a problem with relu?
Discards all negative values
What other activation functions do we have?
ReLU
tanh
Sigmoid
Leaky ReLU
Maxout
ELU
Softmax
different activation functions solve different problems
Whats a problem with tanh?
It’s derivatives go to 0 (which is bad for backpropagation)
What is softmax?
Uses exponents to normalise the output layer of NN into probabilities.
Issues with large learning rate
Overshoot the bottom of the curve for error
