Lecture 3 - CNNs

Question 1

Artificial Intelligence

Answer 1

Technique to make computer intelligent

Contains Machine Learning

Question 2

Machine Learning

Answer 2

A technique that enables machines to learn from experience

Contains Deep Learning

Question 3

Deep Learning

Answer 3

A subset of machine learning based on artificial neural networksF

Question 4

Layer

Answer 4

A function.

Deep networks are stacked layers.

Question 5

Convolution

Answer 5

Take a kernel (key) matrix and scan it over the image by multiplying the elements in the key with the corresponding image value. Add all up.

Repeat for all possible locations

Question 6

Padding

Answer 6

Add numbers to the edge of the matrix to allow the same output dimensions to be retained in convolution.

Question 7

Stride

Answer 7

The number of pixels shifted over the input matrix

(Think like how far the matrix jumps)

Question 8

Output volume height/width calculation (Feature Maps)

Answer 8

( (I-F+2P)/S ) +1

I is an input height/width
F is the corresponding filter height/width
P is the padding
S is the stride

Question 9

Activation Layer

Answer 9

Operation to make model non-linear (increase representation ability). ReLU etc

ReLU = max(0,x)
Leaky ReLU = max(0.1x,x)
Sigmoid = 1/(1+e^-x)
ELU = x above 0 and learning rate*((e^x)-1) equal to and below 0
No learnable parameters

Question 10

Normalisation layer

Answer 10

Accelerate training through a step that fixes the means and variances of each layer’s inputs

No learnable parameters

Question 11

Pooling Layer

Answer 11

Reduces the spatial dimension.

Max pooling, for example.

Question 12

Fully Connected Layer

Answer 12

Connect every neuron in one layer to every neuron in another layer

Learnable parameters = Input dimensions * Output dimensions + output dimensions

Question 13

Convolutional Layer learnable parameters

Answer 13

Ci * Fh * Fw * Co

WHere F is a filter size dimension (height and width) and C is the channels (i for input o for out)