Deep Learning Flashcards
True or False: Deep learning models are prone to overfitting.
True.
Deep learning models can have millions of parameters, making them complex and prone to overfitting.
What is early stopping?
A method for avoiding overfitting.
It identifies the point during training when the error of the model on a validation dataset begins to increase while the training error decreases.
What is a patience parameter?
A predefined threshold that specifies the number of consecutive epochs for which the error on the validation set increases while the training error decreases before training is stopped.
What is dropout?
(don’t even think about it!)
Dropout is another method for avoiding overfitting.
For each feedforward phase, a random set of neurons is chosen from the input and hidden layers.
These selected nodes are dropped from the network and reinstated at the end of the subsequent backpropagation phase.
Essentially, training then runs on the smaller network after the neurons have been dropped.
It is a form of regularization. It keeps all the weights in a network small, thereby making a model’s predictions relatively stable with respect to small changes in the input.
Complete: Once training is stopped, the version of the model that produced the lowest _____ set error is selected.
validation
What does a traditional Convolutional Neural Network (CNN) consist of?
(Hint: there are five layers)
1) input layer
2) convolutional layer
3) flattened layer
4) fully connected layer
5) output layer
What is inverted dropout?
A neuron is dropped by multiplying the activation of the neurons during the forward pass by zero.
A drop-mask is created using a probability P that a node in the network will not be dropped.
The activations of the nodes that are not dropped are divided by P to preserve the magnitude of the weighted sum calculations feeding into the next layer.
What are the input(s) and output(s) of a CNN?
A CNN accepts an image as input.
It can have a variety of outputs, e.g. soft-max output or image.
Complete: Grayscale images are _-dimensional structures, where pixel values are gray level values in the range ___ (black) to ___ (white).
two-dimensional
0 to 255
Complete: Color images are normally represented using the ____ color scheme as __-dimensional structures where the __ dimension (depth) has three channels such that ? < Img(i,j,k) < ?
Here i is the ____ of the image, ____ is the width of the image, k = __, and Img(i,j,k) is a color pixel in the image.
RGB color scheme.
three-dimensional
third dimension
0 to 255
i is the height
j is the width
k = 3
True or False: Image normalization prior to convolution ensures that the extracted features are agnostic of specific image intensity values.
True.
What is convolution?
Image processing technique.
Filter/kernel used as sliding window that traverses similarly sized tiles of an input image, while generating pixel values for an output image.
Element-wise multiplication between input image tile and kernel pixel values.
Then a summation operation that includes a bias.
What is padding?
Extends image boundaries by appending rows/columns of zero values to solve border effect problem.
“valid” - no padding
“same” - sufficient padding to ensure input and output dimensions are the same.
What is CNN training?
Process of learning kernel and fully connected layer weights.
What is pooling in a CNN, and why is it used?
Pooling scales down feature maps by aggregating a window of pixels to reduce dimensions and capture important features.
