Chapter 2. Before we begin: the mathematical building blocks of neural networks

Question 1

What is a class ?

Answer 1

It’s a category in a classification problem

Question 2

What is a sample ?

Answer 2

samples are data points

Question 3

What is a label ?

Answer 3

The class associated with a specific sample

Question 4

What is a loss function ?

Answer 4

How the network will be able to measure its performance on the training data, and thus how it will be able to steer itself in the right direction.

Question 5

What is an optimizer ?

Answer 5

The mechanism through which the network will update itself based on the data it sees and its loss function.

Question 6

What are the metrics to monitor during the training and testing process ?

Answer 6

Here, we’ll only care about accuracy (the fraction of the images that were correctly classified).

Question 7

What is a tensor ?

Answer 7

Data stored in multidimensional Numpy arrays.

At its core, a tensor is a container for data—almost always numerical data. So, it’s a container for numbers. You may be already familiar with matrices, which are 2D tensors: tensors are a generalization of matrices to an arbitrary number of dimensions (note that in the context of tensors, a dimension is often called an axis).

Question 8

What is a scalar ?

Answer 8

A tensor that contains only one number

Question 9

How many number of axis a scalar tensor has ?

Answer 9

0

Question 10

Which attribute do you use to display number of axis of a tensor ?

Answer 10

.ndim

Question 11

What is called an array of numbers ?

Answer 11

An array of numbers is called a vector, or 1D tensor. A 1D tensor is said to have exactly one axis.

Question 12

What is a tensor rank ?

Answer 12

the number of axes

Question 13

What is a matrix ?

Answer 13

An array of vectors or 2D tensors

Question 14

A tensor is defined by which three attributes ?

Answer 14

• Number of Axes
• Shape
• Data Type

Question 15

What is a tensor shape ?

Answer 15

It is a tuple of integers that describes how many dimensions the tensor has along each axis. For instance, the previous matrix example has shape (3, 5), and the 3D tensor example has shape (3, 3, 5). A vector has a shape with a single element, such as (5,), whereas a scalar has an empty shape, ().

Question 16

What is tensor slicing ?

Answer 16

Selecting a specific item in a tensor

Question 17

What is a batch axis or batch dimension ?

Answer 17

It’s the first axis ( axis 0 )

Question 18

What to steps broadcasting consist of:

Answer 18

1. Axes (called broadcast axes) are added to the smaller tensor to match the ndim of the larger tensor.
2. The smaller tensor is repeated alongside these new axes to match the full shape of the larger tensor.

Question 19

What is tensor reshaping ?

Answer 19

Reshaping a tensor means rearranging its rows and columns to match a target shape

Question 20

What is Transposing a Matrix ?

Answer 20

It means exchanging its rows and its columns so that x[i, :] becomes x[:, i]:

Question 21

Summarise steps for a training loop:

Answer 21

1. Draw a batch of training samples x and corresponding targets y.
2. Run the network on x (a step called the forward pass) to obtain predictions y_pred.
3. Compute the loss of the network on the batch, a measure of the mismatch between y_pred and y.
4. Update all weights of the network in a way that slightly reduces the loss on this batch.

Question 22

What is a function’s minimum ?

Answer 22

it is a point where the derivate is 0
Applied to a neural network, that means finding analytically the combination of weight values that yields the smallest possible loss function

Question 23

What is stochastic ?

Answer 23

Stochastic is a scientific synonym of random

Question 24

Where the knowledge of the network persists ?

Answer 24

In the weights tensors

Question 25

What is an epoch ?

Answer 25

Each iteration over all the training data

Question 26

What does learning mean ?

Answer 26

Learning means finding a combination of model parameters that minimizes a loss function for a given set of training data samples and their corresponding targets.

Question 27

How the learning process happen ?

Answer 27

Learning happens by drawing random batches of data samples and their targets, and computing the gradient of the network parameters with respect to the loss on the batch. The network parameters are then moved a bit (the magnitude of the move is defined by the learning rate) in the opposite direction from the gradient.

Question 28

What is the loss ?

Answer 28

The loss is the quantity you’ll attempt to minimize during training, so it should represent a measure of success for the task you’re trying to solve.

Question 29

What is an optimizer ?

Answer 29

The optimizer specifies the exact way in which the gradient of the loss will be used to update parameters: for instance, it could be the RMSProp optimizer, SGD with momentum, and so on.