Image Segmentation

Question 1

Name 2 traditional image segmentation methods

Answer 1

Clustering: Agglomerative and Divisive
Region Growing

Question 2

Name 3 properties that can be used for clustering-based image segmentation

Answer 2

Intensity values, Color properties and Texture measurements

Question 3

Draw a typical NN for semantic segmentation. Make sure to explain how the output shape looks like. Input: HxW C classes

Answer 3

Input -> Fully convolutonal Layers (Sampling and Unsampling, to process the semantic segmentation and return the output in the original size) -> Output: HXWXC

Question 4

What is instance segmentation and how does it compare to semantic segmentation. Name 2 typical approaches for instance segmentation

Answer 4

Instance segmentation not only classifies each pixel to an object like semantic segmentation, but also marks the instance that belongs it.
2 approaches are Mask R-CNN and Multi-task network cascades.

Question 5

What is the difference between semantic segmentation and instance segmentation. Name two fields of application for segmentation tasks

Answer 5

Instance segmentation not only classifies each pixel to an object like semantic segmentation, but also marks the instance that belongs it. Application fields: Medical Images and image or video captioning.

Question 6

Name two segmentation paradigms that are designed in such a way that the number of objects can be obtained. Explain the difference between these paradigm

Answer 6

Instance segmentation focus on classifies each pixel to an object and also says at which instance the object belongs. The semantic segmentation classifies each pixel to an object, but without saying which instance belongs to.

Question 7

Insert the formula for the Pixel Accuracy, IoU and dice coefficient DSC

Answer 7

Pixel Accuracy = (TP + TN) / (TP + TN + FP + FN)
IoU = TP / (TP + FP + FN)
DSC = 2TP / (2TP + FP + FN)

Question 8

Atrous Spatial Pyramid Pooling is an es essential part of DeepLab-Segmentation Architecture. Describe or draw an ASPP-block

Answer 8

Input -> Convolutional Kernel 1x1 rate 6 // Convolutional Kernel 3x3 rate 12 // Convolutional Kernel 3x3 rate 18 // Convolutional Kernel 3x3 rate 24 -> Concatenation Output

Question 9

What problems might ASPP specifically face?

Answer 9

ASPP can face with problems based on degeneration of filters due to 0-padding. That occurs due to increasing rate during the process, and the information of the image is lost by 0-padding (the mean value will also be 0 and the information would be lost)

Question 10

Name potential solutions for the problems you mentioned above about ASPP

Answer 10

Solution:
- Incorporation of global context
- GAP-Features fed to 1x1 convolution and upsample

Question 11

How many pooling layers of extent 2x2 does a fully convolutional network need to produce afeature map of 1/32 the scale of the original image?

Answer 11

The pooling operation with a 2x2 kernel reduces the spatial dimensions of the feature map by half in each dimension (HxW → H/2 x W/2).

So, to get (H/32 X W/32) it is necessary 5 pooling layers

Question 12

If a 2x2 atrous (dilated) convolution kernel with a dilated rate = 1 has 4 parameters, how many parameters does a 3x3 atrous (dilated) convolution kernel with a dilated rate = 2 have?

Answer 12

9 parameters. The dilated rate determines the spacing between the values in the kernel, and does not change the number of parameters.

Question 13

Write down one advantage of atrous (dilated) convolutionlayers

Answer 13

Increase the image resolution

Question 14

How does convolution in DeepLab differ from conventional convolutions and what is its advantage?

Answer 14

DeepLab uses atrous (dilated) convolutions, enabling increase the receptive field and maintains the image resolution during the training.

Question 15

What technique does DeepLabV3 use to deal with degeneration of filters?

Answer 15

DeepLabV3 uses the “atrous spatial pyramid pooling” (ASPP), which incorporates a global context by concatenating parallel atrous convolutions

Question 16

State one difference that sets apart data augmentation for segmentation and for classification

Answer 16

Data augmentation for segmentation must preserve spatial correspondence between input images and their corresponding masks, while classification augmentation does not require such alignment.