DL-05a - Convolutional neural network

Question 1

DL-05a - Convolutional neural network

How do you perform convolution on discrete data?

Answer 1

Slide a filter (kernel) over the input data matrix, applying element-wise multiplication, and sum the results to create a new output matrix.

Question 2

DL-05a - Convolutional neural network

Describe how to perform convolution for 1 output element.

Answer 2

(See image)

Question 3

DL-05a - Convolutional neural network

Describe what comes out of after one pass. (See image)

Answer 3

A WxHx1 tensor (1 channel) at a time. Can have multiple output channels.

Question 4

DL-05a - Convolutional neural network

What is padding?

Answer 4

The process of adding extra pixels or data around the input image or feature map to maintain the spatial dimensions during convolution.

Question 5

DL-05a - Convolutional neural network

What is stride?

Answer 5

The number of units by which the filter slides over the input during the convolution process. I.e. jumping factor.

Question 6

DL-05a - Convolutional neural network

Why are fully connected layers not sufficient for image data?

Answer 6

When flattening the image, you lose information about spatial correlation (adjacency information).

Question 7

DL-05a - Convolutional neural network

Why is the number of parameters in a CNN lower than in a fully connected network?

Answer 7

The number of parameters in a CNN is lower because it uses shared weights and local connectivity, reducing redundancy and computational complexity.

Question 8

DL-05a - Convolutional neural network

How many parameters are in a convolutional layer?

Answer 8

n(wh*ch) + n

w, h = x size
ch = number of channels
W = Kernel of size w * h * ch
b = bias term
Use n kernels per input, add n biases -> n(Wx + b) -> n((wh) + 1) -> nw*h + n

(See image)

Question 9

DL-05a - Convolutional neural network

Is a smaller or larger kernel preferred?

Answer 9

Depends.
- Large captures global features
- Small captures local features

Question 10

DL-05a - Convolutional neural network

What kernel size captures global features?

Answer 10

Large kernels.

Question 11

DL-05a - Convolutional neural network

What kernel size captures local features?

Answer 11

Small kernels.

Question 12

DL-05a - Convolutional neural network

What does the pooling layer do? (3)

Answer 12

• Downsample spatial dimensions.
• Reduces number of params
• Helps control overfitting

Question 13

DL-05a - Convolutional neural network

What are the common pooling layer types? (2)

Answer 13

• Max pooling
• Average pooling

Question 14

DL-05a - Convolutional neural network

Is average or max pooling better?

Answer 14

Max pooling has been shown to work better in practice.

Question 15

DL-05a - Convolutional neural network

What is an alternative to pooling?

Answer 15

Using conv layers with stride >1.