Lecture 10 - Recognition

Question 1

What is image classification?

Answer 1

Image classification is the task of assigning a label from a predefined set of categories to an entire image.

Question 2

Describe the architecture of LeNet.

Answer 2

LeNet is one of the first convolutional networks designed for handwritten digit recognition, featuring layers of convolutional filters followed by subsampling (pooling) and fully connected layers.

Question 3

What is AlexNet and its significance?

Answer 3

AlexNet is a convolutional network that won the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) in 2012, significantly outperforming previous models with its deep architecture and use of ReLU activation and dropout.

Question 4

Explain dropout in neural networks.

Answer 4

Dropout is a regularization technique where a random subset of neurons is ignored during training, preventing overfitting by ensuring the model does not rely too heavily on any single neuron.

Question 5

What is data augmentation and its purpose?

Answer 5

Data augmentation artificially enlarges the training dataset by applying label-preserving transformations such as cropping, flipping, and color jittering to improve model generalization.

Question 6

Describe the purpose of batch normalization.

Answer 6

Batch normalization normalizes the inputs of each layer to have a mean of zero and variance of one, accelerating training and improving model performance by reducing internal covariate shift.

Question 7

What are the key features of VGGNet?

Answer 7

VGGNet uses small 3x3 convolutional filters stacked together, leading to deep networks with more non-linearities, which improve performance while maintaining computational efficiency.

Question 8

Explain the Inception module in GoogLeNet.

Answer 8

The Inception module in GoogLeNet combines multiple convolutional filters of different sizes and pooling operations within the same layer, allowing the network to capture features at various scales.

Question 9

What is a Residual Network (ResNet)?

Answer 9

ResNet introduces residual blocks with shortcut connections that bypass one or more layers, allowing very deep networks to be trained by mitigating the vanishing gradient problem.

Question 10

Describe the concept of feature learning in CNNs.

Answer 10

Feature learning in CNNs refers to the automatic extraction and learning of hierarchical features from raw input images, replacing the need for manual feature engineering.

Question 11

What is the role of the softmax function in CNNs?

Answer 11

The softmax function is used in the output layer of CNNs for multi-class classification, converting raw output scores into probabilities by normalizing them.

Question 12

Explain the concept of transfer learning in CNNs.

Answer 12

Transfer learning involves using a pre-trained network on a large dataset and fine-tuning it on a smaller, task-specific dataset, leveraging the learned features to improve performance and reduce training time.

Question 13

Write the formula for the softmax function.

Answer 13

Question 14

Provide the formula for the ReLU activation function.

Answer 14

Question 15

What is the formula for updating weights using gradient descent in neural networks?

Answer 15

Question 16

How did AlexNet overcome overfitting?

Answer 16

AlexNet used dropout on the fully connected layers and data augmentation techniques to overcome overfitting.

Question 17

What are the benefits of using small convolutional filters in VGGNet?

Answer 17

Small convolutional filters (3x3) in VGGNet allow for deeper networks with more non-linearities, improving feature learning while maintaining computational efficiency.

Question 18

Explain how the Inception module improves performance in GoogLeNet.

Answer 18

The Inception module improves performance by combining multiple convolutional filters and pooling operations, capturing features at various scales and reducing computational complexity.

Question 19

Why are residual connections important in ResNet?

Answer 19

Residual connections in ResNet allow the network to bypass certain layers, helping to prevent the vanishing gradient problem and enabling the training of very deep networks.

Question 20

How does batch normalization accelerate training?

Answer 20

Batch normalization accelerates training by normalizing the inputs of each layer, reducing internal covariate shift, and allowing for higher learning rates.

Question 21

Describe the role of data augmentation in improving model generalization.

Answer 21

Data augmentation improves model generalization by increasing the diversity of the training data through label-preserving transformations, helping the model learn to be invariant to various image distortions.

Question 22

What is the significance of using multiple GPUs in training AlexNet?

Answer 22

Using multiple GPUs allowed AlexNet to train faster and handle larger models by dividing the workload, making it feasible to train on large datasets like ImageNet.

Question 23

How does dropout prevent overfitting in neural networks?

Answer 23

Dropout prevents overfitting by randomly dropping neurons during training, ensuring the network does not rely too heavily on any single neuron and promoting redundancy.

Question 24

What are the main contributions of the LeNet architecture?

Answer 24

LeNet was one of the first successful applications of convolutional networks, demonstrating their effectiveness in recognizing handwritten digits with high accuracy and efficiency.

Question 25

Describe the process of transfer learning in the context of CNNs.

Answer 25

Transfer learning involves taking a pre-trained CNN on a large dataset, such as ImageNet, and fine-tuning it on a smaller, task-specific dataset to leverage learned features and improve performance.

Question 26

Explain the concept of feature learning in CNNs.

Answer 26

Feature learning in CNNs refers to the automatic extraction and learning of hierarchical features from raw input images through convolutional layers, enabling the network to learn relevant patterns for classification tasks.

Question 27

What challenges do deeper networks face and how are they addressed?

Answer 27

Deeper networks face challenges like vanishing/exploding gradients, slower training, and overfitting. These are addressed with techniques such as residual connections, batch normalization, dropout, and data augmentation.