Introduction to Deep Learning

Question 1

In a single perceptron(neuron), what is the formula for calculating a single output?

Answer 1

y = g(W0 + XW)
1. Dot product (XW)
2. Add bias (W0)
3. Apply non-linearity (g)

Question 2

What are the 3 layers of NN?

Answer 2

Input layer, hidden layers, output layer.
The input layer receives data, hidden layers process it, and the output layer produces the final predictions.

Question 3

How to determine how well the model perform?

__ Function

Answer 3

Using Loss Function

Question 4

What are the 2 types of loss functions?

1. Binary __ loss
2. Mean __

Answer 4

1. Binary cross entropy loss (Classification)
2. Mean Squared Error (MSE) (Regression)

Question 5

What is the problem of training NN?

Finding the vector __. The __ determine the performance, because they…

Answer 5

Finding the vector weights(W). The weights determine the performance, because they directly influence the output and thus the loss.

Question 6

What is backpropagation in simple sense?

Backpropa uses the __ rule to compute the gradient of the loss …

Answer 6

Backpropagation uses the chain rule to compute the gradient of the loss with respect to the weights (W). This allows us to update the weights in the direction that minimizes the loss.

Question 7

In summary, what is forwardpropagation?

Answer 7

FP refers to the computation of outputs from the inputs in NN.

Question 8

What is the optimization method we can use during backpropagation?

__ (GD)

Answer 8

The method is gradient descent.

Question 9

How does GD works?

It sets the ___ __ (LR) …

Answer 9

It sets the learning rates by designing an adaptive LR that adapts to the landscape.

Question 10

What are 2 tips to train NN?

1. S…
2. B…

Answer 10

Stochastic GD (SGD) and batching

Question 11

Batching is better than SGD. Why?

It uses ___ batch of data to compute.

Answer 11

Instead of using single training point, batching uses small batch of data to compute. It is more accurate and faster to train.

Question 12

How to deal with overfitting in NN?

Re___

Answer 12

Regularization

Question 13

What is the 2 regularization technique?

1. D…
2. E… S…

Answer 13

1. Dropout
2. Early Stopping

Question 14

How does Dropout work?

1. Randonly set some ___ to 0.
2. Drop __% of __ in layers.

Answer 14

• Randomly set some activations to 0.
• Drop 50% of activation in layers

Question 15

How does Early Stopping work?

Stop the model from ___ before ___

Answer 15

Stop the model from training before overfitting.