Deep Learning

Question 1

How do NN capture interactions?

Answer 1

By using hidden layers in which nodes are the result of the dot products of n-1 layer nodes.

Question 2

What is forward propagation?

Answer 2

Multiply input node values by weights specified in edge between those nodes and next layer’s node and ADD..

Question 3

How to compute final NN output value using np?

Answer 3

output = (hidden_layer_values * weights[‘output’]).sum()

Question 4

What is an activation function?

Answer 4

A function applied to node inputs to produce node output.

Question 5

What is RELU?

Answer 5

Rectified Linear Activation. 0 if x < 0 else x.

Question 6

Why is there less of a need for feature engineering with DL?

Answer 6

Deep networks internally build representations of patterns in the data. Subsequent layers build increasingly sophisticated representations of raw data.

Question 7

What are the steps of Gradient Descent?

Answer 7

Start at random point
Until you are somewhere flat:
● Find the slope
● Take a step downhill

Question 8

How to avoid big steps with GD?

Answer 8

Using a learning rate: Update each weight by subtracting

learning rate * slope

Question 9

How to calculate new weight for a current weight of 2 connecting Node with value of 3 to Node with predicted Valued 6 and actual Value of 10 and learning rate 0.01?

Answer 9

Multiply the gradient with the learning rate:
* Slope of the loss function (Error) w.r.t value at the node we feed into: 2* (Predicted Value [6] - Actual Value [10]): -8
* The value of the node that feeds into our weight: 3
* Slope of activation function at the node it feeds into. None here.
* Learning rate: 0.01
Result: 2 - 0.01(-24) = 2.24

Question 10

What is backpropagation?

Answer 10

It allows gradient descent to update all weights in neural network (by getting gradients for all weights). It first tries to estimate the slope of the loss function w.r.t each weight and then uses forward propagation to calculate predictions and errors.

Question 11

What is stochastic gradient descent?

Answer 11

When slopes are calculated on one batch at a time.

Question 12

What is an epoch?

Answer 12

When all batches have been used to update the weights.

Question 13

What is the Adam optimizer?

Answer 13

An algorithm for first-order gradient-based optimization of stochastic objective functions.

Question 14

How to load csv data with np?

Answer 14

predictors = np.loadtxt(‘predictors_data.csv’, delimiter=’,’)

Question 15

How to ease optimization?

Answer 15

Scaling data before fitting can ease optimization.

Question 16

What is a good objective function for regression?

Answer 16

Mean squared error

Question 17

What is a good objective function for classification?

Answer 17

categorical cross entropy

Question 18

What is softmax?

Answer 18

A generalization of the logistic function that “squashes” a K-dimensional vector z of arbitrary real values to a K-dimensional vector s(z) of real values in the range [0, 1] that add up to 1.

Question 19

What is the first argument to keras.layers.Dense?

Answer 19

The number of units, representing the dimensionality of the output space.

Question 20

How to do one-hot encoding with keras for classification?

Answer 20

keras.utils.to_categorical(list)

Question 21

Why is k-fold validation not used frequently for DL?

Answer 21

Too long too repeat; data sets are large anyway so trust single validation.

Question 22

What is early stopping?

Answer 22

How many epochs we are willing to see without improvements.