CNN Flashcards
What is the problem with sigmoid activation function as we add more hidden layers? What is the solution to this?
Vanishing gradient: change in weights become infinitesimally small
ReLu - Rectified Linear Unit
What are sigmoids mostly used for in CNNs?
As output transformations if we want a probability
What is the ReLu function maths?
f(x) = max(0,x)
What is a deep NN
2+ layers
Why do we need DNNs?
To solve real world problems
What does the term “compositional structure of natural data” mean? How do DNNs use this?
Real world data is inherently hierarchical
DNNs use this to classify/generate content by setting up layers of feature extractors and filters
What is a CNN?
A network with 1+ convolutional layers
Convolutional layers consist of:
Trainable filters
Non-linear activation function
Pooling layer
What are filters?
Trainable feature extractors
N dimensional window
Pixels of this window have the activation function applied to it
Applies dot product
What is pooling?
Reduce size of feature map by filtering
Like filtering, uses a window to scan feature map
Types of pooling?
Average = calculate average of each patch Max = select maximum from each patch
Why pool?
makes feature map easier to process for future layers
CNN process
- Normalise input
- Filter input
- Apply activation function to filtered input
- Pool to reduce kernel size
- Repeat for every convolutional layer
