Neural Networks

Question 1

What is a feedforward network?

Answer 1

A network where the inputs are put through multiple layers, every layer doing calculations before sending the input to the next layer

Question 2

How do backpropagation work?

Answer 2

• You run one feedforward cycle
• compare expected output to the output (y-hat)
• calculate the error between them
• Run backprop = see which neurons are doing a bad job predicting and decrease their weights and vice versa for good neurons
• repeat

Question 3

What is overfitting?

Answer 3

Your model is performing well on the training data but poorly on test data

Question 4

What is underfitting?

Answer 4

Your model is performing poorly on train data

Question 5

When do you want to use L1 regularization?

Answer 5

L1 is better when you want to find the best features with fewer weights - it is more sparse

Question 6

When do you want to use L2 regularization?

Answer 6

L2 is better for training neural models since it tries to optimize and keep all weights

Question 7

Why use dropout?

Answer 7

Because dropout prevents some neurons to become overly decisive in the decision-making.

Question 8

What is Stochastic Gradient Descent?

Answer 8

SGD allows you to divide the train data in each epoch into batches. You run each batch through the network, update the weights (backprop) and then repeat until all batches has run - you then run the next epoch