Chapter 3 Getting started with neural networks

Question 1

relu function vs. sigmoid

Answer 1

relu (rectified linear unit) zeroes out negative values, sigmoid “squashes” arbitrary values into the [0, 1] interval

Question 2

Advantages of larger layers?

Answer 2

smaller layers can act as information bottlenecks permanently dropping important information that other layers won’t have access to.

Question 3

softmax activation

Answer 3

network will output a probability distribution over the different output classes that sums to 1

Question 4

Categorical_crossentropy

Answer 4

loss function which measures the distance between two probability distributions.

Question 5

network overfitting

Answer 5

when the network starts to get trained to specific features of a data set rather than learning overall trends

Question 6

what loss function should you use for single-label, multiclass classification problems?

Answer 6

categorical crossentropy

Question 7

feature-wise normalization

Answer 7

best practice for data pre-processing. For each feature in the input data you subtract by the mean and divide by the standard deviation

Question 8

amount of data and overfitting

Answer 8

the less data you have, the worse overfitting is

Question 9

what is one way to mitigate overfitting?

Answer 9

use smaller networks

Question 10

widely used loss function for regression problems?

Answer 10

Mean Squared Error

Question 11

Mean Squared Error

Answer 11

loss function: the square of difference between the predictions and the targets

Question 12

Mean Absolute Error (MAE)

Answer 12

metric for monitoring model performance in regression problems