Lecture 9 - CNNs

Question 1

What is a Convolutional Neural Network (CNN)?

Answer 1

A CNN is a type of deep learning model designed for processing structured grid data like images. It uses convolutional layers to automatically and adaptively learn spatial hierarchies of features from input images.

Question 2

What is the purpose of convolutional layers in CNNs?

Answer 2

Convolutional layers apply a set of learnable filters (kernels) to the input image to extract local features, such as edges, textures, and patterns, reducing the number of parameters and computations required.

Question 3

Define “filter” or “kernel” in the context of CNNs.

Answer 3

A filter or kernel is a small matrix used to convolve with the input data to produce feature maps. It slides over the input and performs element-wise multiplication and summation.

Question 4

What is the role of pooling layers in CNNs?

Answer 4

Pooling layers reduce the spatial dimensions of the feature maps, retaining the most important information and making the network invariant to small translations of the input.

Question 5

Describe “ReLU” activation function.

Answer 5

The ReLU (Rectified Linear Unit) activation function applies the function
f(x)=max(0,x) to each element of the input, introducing non-linearity and helping to prevent vanishing gradients.

Question 6

What is a fully connected layer in a CNN?

Answer 6

A fully connected layer connects every neuron in one layer to every neuron in the next layer, typically used at the end of the network to combine the extracted features for classification or regression.

Question 7

Explain the concept of “stride” in convolution operations.

Answer 7

Stride refers to the number of pixels by which the filter matrix shifts over the input matrix. A larger stride reduces the spatial dimensions of the output feature map.

Question 8

What is padding in CNNs?

Answer 8

Padding involves adding extra pixels around the border of the input image to control the spatial dimensions of the output feature maps and preserve important information at the edges.

Question 9

Define “feature map” in the context of CNNs.

Answer 9

A feature map is the output of a convolutional layer after applying filters to the input image, representing the activation of specific features detected by the filters.

Question 10

What is batch normalization?

Answer 10

Batch normalization is a technique to stabilize and accelerate the training of deep neural networks by normalizing the inputs of each layer to have a mean of zero and variance of one.

Question 11

Describe the concept of “dropout” in CNNs.

Answer 11

Dropout is a regularization technique where randomly selected neurons are ignored (dropped out) during training, reducing overfitting by preventing neurons from co-adapting too much.

Question 12

What are some common architectures of CNNs?

Answer 12

Common CNN architectures include LeNet, AlexNet, VGGNet, GoogLeNet (Inception), and ResNet, each with different configurations of convolutional and pooling layers.

Question 13

Write the formula for a 2D convolution operation.

Answer 13

Question 14

Provide the formula for the ReLU activation function.

Answer 14

Question 15

What is the formula for the output size of a convolutional layer?

Answer 15

Question 16

How does a convolutional layer help in reducing the number of parameters in a CNN?

Answer 16

Convolutional layers use shared weights in the form of filters that slide across the input image, significantly reducing the number of parameters compared to fully connected layers.

Question 17

Explain the importance of non-linearity in neural networks.

Answer 17

Non-linearity introduced by activation functions like ReLU allows the network to learn complex patterns and functions that linear models cannot, enabling the network to capture intricate relationships in the data.

Question 18

What is the significance of using pooling layers in CNNs?

Answer 18

Pooling layers reduce the spatial dimensions of the feature maps, lowering the computational complexity, and making the network invariant to small translations, rotations, and distortions in the input image.

Question 19

Describe how dropout helps in preventing overfitting in CNNs.

Answer 19

Dropout randomly drops neurons during training, forcing the network to learn redundant representations and preventing overfitting by ensuring that the network does not rely too heavily on any single neuron.

Question 20

How does batch normalization improve the training of CNNs?

Answer 20

Batch normalization normalizes the inputs of each layer to have a mean of zero and variance of one, stabilizing and accelerating the training process by reducing internal covariate shift.

Question 21

What is the difference between valid and same padding in CNNs?

Answer 21

Valid padding means no padding is added, resulting in a smaller output feature map, while same padding adds zeros around the input to ensure the output feature map has the same spatial dimensions as the input.

Question 22

Explain the architecture of the AlexNet model.

Answer 22

AlexNet consists of five convolutional layers followed by three fully connected layers, with ReLU activations, max-pooling, dropout, and local response normalization to improve performance on the ImageNet dataset.

Question 23

What is the purpose of using multiple convolutional layers in a CNN?

Answer 23

Multiple convolutional layers allow the network to learn hierarchical representations, with lower layers capturing basic features like edges and textures, and higher layers capturing more complex features like shapes and objects.

Question 24

How does the Inception module in GoogLeNet differ from traditional convolutional layers?

Answer 24

The Inception module uses multiple parallel convolutional layers with different filter sizes and pooling operations, allowing the network to capture features at various scales and reducing the computational cost.

Question 25

Describe the concept of residual learning in ResNet.

Answer 25

Residual learning in ResNet introduces shortcut connections that skip one or more layers, enabling the training of very deep networks by addressing the vanishing gradient problem and making it easier to learn identity mappings.

Question 26

What is the role of data augmentation in training CNNs?

Answer 26

Data augmentation artificially increases the size of the training dataset by applying random transformations (e.g., rotations, flips, cropping) to the input images, helping to improve the generalization of the model.

Question 27

How does transfer learning benefit CNN training?

Answer 27

Transfer learning involves using a pre-trained network on a large dataset and fine-tuning it on a smaller, task-specific dataset, leveraging the learned features to improve performance and reduce training time.