09_convolutional neural networks and computer vision

Question 1

Why are some layers called “hidden layers”?

Answer 1

because it’s difficult to tell what happens in them exactly

Question 2

What happens in different stages in the layers of a fully connected neural network?

Answer 2

input layer

1) feature extraction

2) classification

output layer

–> end-to-end learning (no feature engineering needed)

Question 3

What happens if we linearize/vectorize an input?

Answer 3

the network is unable to leverage spatial information from a linearized image

Question 4

What are convolutions?

Answer 4

a mathematical operation that operate on dense data (such as images)

we consider an image I and a “Kernel” K. the “convolution” of I and K will result
in an output feature map S

S is constrained by the size of the inputs and the kernel

–> convolutional kernels acts as feature filters and are sometimes called “filters” for this exact reason
(provide high output signals where the projections of the kernel resembles the input pattern)

Question 5

What is the sobel filter?

Answer 5

is constructed to extract edges
(only cares about the light changes that indicate edges)

Question 6

On what parameters are convolutional layers based on?

Answer 6

• number of input channels
• number of output channels

(affect output feature map size)
- kernel size
- stride
- padding

Question 7

What is a channel?

Answer 7

a two-dimensional feature map

Question 8

What is multi-channel data?

Answer 8

stacks of feature maps
(eg with different colors)

we use one kernel per channel (don’t have to be identical)

Question 9

What does the kernel size define?

Answer 9

the size of the kernel operator - affects the size of the output of the convolution

Question 10

How can we calculate the output feature map in regards to the kernel size?

Answer 10

O = I - K + 1

O = output feature map size
K = kernel size
I = input feature map size

Question 11

What is the receptive field of a kernel?

Answer 11

depends on their size and the features of different sizes they are therefore able to see

–> large kernels are able to identify larger and more complex features such as an eye: they have more CONTEXT

Question 12

What is a stride in CNNs?

Answer 12

step size of a kernel
(default is 1)

If we choose a stride >1, it reduces the size of the output feature map

–> O = (I - K) / S + 1
S= stride

Question 13

Why do we need padding and what is it?

Answer 13

convolutions shrink feature maps, padding counter-acts this

–> padding input map with rows and columns of zeros

O = (I - K + 2P) / S + 1

Question 14

What is a huge advantage of convolutional layers regarding parameters?

Answer 14

because the kernel parameters are learned by the model itself and shared over the entire input feature map,
the number of parameters of the network is vastly reduced

Question 15

How many parameters does a convolutional layer have?

Answer 15

K^2 * C(in) * C(out)

the number of parameters is therefore independent of the input feature map size as opposed to linear layers

Question 16

What does the combination of convolutional layers lead to?

Answer 16

a large receptive field: info that is seen by a single element in the output feature map covers a much larger area in previous feature maps

Question 17

How is a CNN for image classification build?

Answer 17

Input

1) convolutional layers = feature extraction

(linearization in between)

2) linear layers = classifier

output

(non-linearities after each layer)

–> decreasing feature map size: decreasing resolution
–> increasing number of channels: increasing semantic information

Question 18

How to implement a segmentation model?

Answer 18

1) replace linear layers with convolutional layers (output must be feature map)

2) adjust convolutions so that final output has the same size as input image (eg through padding)

3) apply SOFTMAX after final layer to get class probability for each pixel

Question 19

Why do we use U-Nets for segmentation models?

Answer 19

it is very expensive if we have large size of feature map after each layer

Question 20

£What does the U-Net do?

Answer 20

first downsampling - encode
then upsampling - decode

downsampling of feature map greatly reduces the feature map extents and improves the ability to generalize

–> much better performance with skip connections

Question 21

What are skip connections in U-Net?

Answer 21

supplement the semantically rich information from the layers near the bottleneck with high-resolution feature maps from the downsampling branch

Question 22

How do we do upsampling?

Answer 22

transported convolutions: using a stride <1 we can use convolutions as learnable upscaling operations

Question 23

What is panoptic segmentation?

Answer 23

Combi of semantic segmentation and instance segmentation

Question 24

What is object detection for images?

Answer 24

goal: find instances of a given class in an image (provide class, approximate location and extent (bounding box))

Question 25

What are complications with object detection? (5)

Answer 25

• scale variability
• object might be occulted
• lighting and shadows
• perspective affects the appearance of an object
• how do we define the location of the object?

Question 26

What are examples of multi-stage object detection?

Answer 26

region-based CNNs
- object detection process is split into different steps that require to process the image data multiple times

fast R-CNN
- more efficient

faster R-CNN
- proposal regions are extracted from CNN feature map

Question 27

What is single-stage object detection?

Answer 27

YOLO - you only look once