Lecture 8 - Deep Learning Basics

Question 1

What is machine learning for visual perception?

Answer 1

Machine learning for visual perception involves discovering and leveraging patterns in images to make predictions or classifications based on visible patterns. It includes tasks like segmentation, detection, and recognition.

Question 2

Describe supervised learning.

Answer 2

Supervised learning involves training a model with labeled examples, where each example has both features (images) and corresponding labels related to the task, such as segmentation or recognition.

Question 3

What is unsupervised learning?

Answer 3

Unsupervised learning involves modeling, representing, or describing the variability within data without specific labels. It is used for tasks like noise removal, super-resolution, and deconvolution.

Question 4

Explain the concept of a cost function in machine learning.

Answer 4

A cost function is used to define “best” parameters for a model by measuring the error between the model’s predictions and the ground truth labels. Examples include cross-entropy for classification and mean squared error for regression.

Question 5

What is a perceptron?

Answer 5

A perceptron is a simple linear classifier used for binary classification. It consists of an activation function (linear transformation of inputs) and a nonlinearity (e.g., sigmoid function) to produce the output.

Question 6

Describe the architecture of a multilayer perceptron (MLP).

Answer 6

An MLP consists of an input layer, one or more hidden layers, and an output layer. Each layer is composed of neurons that apply a linear transformation followed by a non-linear activation function, enabling the network to model complex functions.

Question 7

What is the purpose of backpropagation in training neural networks?

Answer 7

Backpropagation is an algorithm used to train neural networks by computing the gradient of the cost function with respect to each weight and updating the weights to minimize the cost function.

Question 8

Explain the role of the activation function in neural networks.

Answer 8

The activation function introduces non-linearity into the network, allowing it to learn and model complex relationships between inputs and outputs. Common activation functions include sigmoid, tanh, and ReLU.

Question 9

What is a fully connected layer in a neural network?

Answer 9

A fully connected layer is a layer where each neuron is connected to every neuron in the previous layer, enabling complex interactions between features and contributing to the network’s ability to learn representations.

Question 10

Describe the softmax function used in neural networks.

Answer 10

The softmax function is used in the output layer of a neural network for multi-class classification. It converts raw output scores (logits) into probabilities by exponentiating the scores and normalizing them.

Question 11

What is the difference between a training set, validation set, and test set?

Answer 11

The training set is used to learn model parameters, the validation set is used to tune hyperparameters and prevent overfitting, and the test set is used to evaluate the model’s generalization performance on unseen data.

Question 12

Explain the concept of overfitting in machine learning.

Answer 12

Overfitting occurs when a model learns to perform well on the training data, including its noise and outliers, but fails to generalize to new, unseen data, resulting in poor performance on the test set.

Question 13

Provide the formula for the gradient descent update rule.

Answer 13

Question 14

Write the cost function for logistic regression.

Answer 14

Question 15

Write the formula for the output of a neuron in an MLP.

Answer 15

Question 16

How does supervised learning differ from unsupervised learning?

Answer 16

Supervised learning uses labeled data to train models for specific tasks, whereas unsupervised learning uses unlabeled data to find patterns or representations without a predefined task.

Question 17

What are some common activation functions used in neural networks?

Answer 17

Common activation functions include sigmoid, tanh, and ReLU (Rectified Linear Unit).

Question 18

Explain the purpose of the softmax function in a neural network.

Answer 18

The softmax function converts the raw output scores of a neural network into probabilities, making it suitable for multi-class classification tasks.

Question 19

Why is backpropagation important in training neural networks?

Answer 19

Backpropagation efficiently computes the gradient of the cost function with respect to each weight, allowing for effective optimization and training of deep neural networks.

Question 20

What is the role of the cost function in training neural networks?

Answer 20

The cost function measures the error between the predicted outputs and the actual labels, guiding the optimization process to find the best model parameters.

Question 21

How do you prevent overfitting in a neural network?

Answer 21

Overfitting can be prevented by using techniques such as cross-validation, regularization, dropout, early stopping, and increasing the size of the training set.

Question 22

Describe the process of gradient descent in training neural networks.

Answer 22

Gradient descent involves iteratively updating the model parameters in the direction of the negative gradient of the cost function to minimize the error.

Question 23

What is the significance of using multiple layers in a neural network?

Answer 23

Multiple layers allow the network to learn hierarchical representations, enabling it to model complex patterns and functions that a single layer cannot capture.

Question 24

How does a validation set differ from a test set?

Answer 24

The validation set is used to tune hyperparameters and select the best model during training, while the test set is used to evaluate the final model’s performance on unseen data.

Question 25

Explain the concept of “dropout” in neural networks.

Answer 25

Dropout is a regularization technique that randomly sets a fraction of the neurons to zero during training, preventing the network from becoming too reliant on specific neurons and reducing overfitting.

Question 26

What are the challenges of training deep neural networks?

Answer 26

Challenges include vanishing and exploding gradients, overfitting, high computational cost, and the need for large amounts of labeled training data.

Question 27

Describe the role of the learning rate in gradient descent.

Answer 27

The learning rate determines the step size for each iteration of gradient descent. A too high learning rate can cause divergence, while a too low learning rate can lead to slow convergence.