Your Deep Learning Journey

Question 1

Label

Answer 1

The data that we’re trying to predict, such as “dog” or “cat”

Question 2

Architecture

Answer 2

The template of the model that we’re trying to fit; i.e., the actual mathematical function that we’re passing the input data and parameters to

Question 3

Model

Answer 3

The combination of the architecture with a particular set of parameters

Question 4

Parameters

Answer 4

The values in the model that change what task it can do and that are updated through model training

Question 5

Fit / Train

Answer 5

(1) Update the parameters of the model such that the predictions of the model using the input data (2) match the target labels

Question 6

Pretrained model

Answer 6

A model that has already been trained, generally using a large dataset, and will be fine-tuned

Question 7

Fine-tune

Answer 7

Update a pretrained model for a different task

Question 8

Epoch

Answer 8

One complete pass through the input data; the model has seen every item in the training set.

Question 9

Loss

Answer 9

A measure of how good the model is, chosen to drive training via SGD (Stochastic Gradient Descent)

Question 10

Metric

Answer 10

A measurement of how good the model is using the validation set, chosen for human consumption

Question 11

Validation set

Answer 11

A set of data held out from training, used only for measuring how good the model is

Question 12

Training set

Answer 12

The data used for fitting the model; does not include any data from the validation set

Question 13

Overfitting

Answer 13

Training a model in such a way that it remembers specific features of the input data, rather than generalizing well to data not seen during training

Question 14

CNN

Answer 14

Convolutional neural network; a type of neural network that works particularly well for computer vision tasks

Question 15

Deep learning is…

Answer 15

…a specialty within machine learning that uses neural networks with multiple layers.

Question 16

Machine learning is…

Answer 16

…a discipline in which we define a program not by writing it entirely ourselves, but by learning from data.

Question 17

What is distinctive about Deep Learning architectures?

Answer 17

They are based on neural networks. (e.g.: CNN, RNN, Transformers)

Question 18

What is segmentation?

Answer 18

At its core, segmentation is a pixelwise classification problem. We attempt to predict a label for every single pixel in the image. This provides a mask for which parts of the image correspond to the given label.

Question 19

Tabular Data

Answer 19

Data that is in the form of a table, such as from a spreadsheet, data‐base, or a comma-separated values (CSV) file.

Question 20

Categorical Value

Answer 20

contain values that are one of a discrete set of choices, such as occupation

Question 21

Continuous Value

Answer 21

contain a number that represents a quantity, such as age

Question 22

What are “hyperparameters”?

Answer 22

choices regarding network architecture, learning rates, data augmentation strategies, and other factors […] Training models require various other parameters that define how the model is trained. For example, we need to define how long we train for, or what learning rate (how fast the model parameters are allowed to change) is used. These sorts of parameters are hyperparameters.

Question 23

What was the name of the first device that was based on the principle of the artificial neuron?

Answer 23

Mark I perceptron built by Frank Rosenblatt

Question 24

Why is it hard to understand why a deep learning model makes a particular prediction?

Answer 24

Deep neural networks have thousands of layers. It is hard to determine which factors are important in determining the final output. The neurons in the network interact with each other. All of this makes it very difficult to understand why a neural network makes a given prediction.

Question 25

What is the name of the theorem that a neural network can solve any mathematical problem to any level of accuracy?

Answer 25

The universal approximation theorem states that neural networks can theoretically represent any mathematical function. However, it is important to realize that practically, due to the limits of available data and computer hardware, it is impossible to practically train a model to do so. But we can get very close!

Question 26

What do you need in order to train a model?

Answer 26

You will need (1) an architecture for the given problem. You will need (2) data to input to your model. For most use-cases of deep learning, you will need (3) labels for your data to compare your model predictions to. You will need (4) a loss function that will quantitatively measure the performance of your model. And you need a way to update the parameters of the model in order to improve its performance (this is known as an [5] optimizer).

Question 27

How could a feedback loop impact the rollout of a predictive policing model?

Answer 27

If a model is retrained based on the policing decisions taken based on the previous model, this action will only add bias to the model.

Question 28

Do we always have to use 224x224 pixel images with the cat recognition model?

Answer 28

No we do not. 224x224 is commonly used for historical reasons. You can increase the size and get better performance, but at the price of speed and memory consumption.

Question 29

What is a validation set? What is a test set? Why do we need them?

Answer 29

Validation Set: Used to ‘validate’ the model’s prediction, it uses data the model didn’t use to train itself; it prevents overfitting.
Test Set: Tests the model with data neither the model nor the modeller had seen before; the modeller can adjust hyperparameters to make the validation set ‘fit’. That’s why it’s important to have an unseen test set.

Question 30

What is a metric? How does it differ to “loss”?

Answer 30

A metric is a function that measures quality of the model’s predictions using the validation set. This is similar to the ­ loss , which is also a measure of performance of the model. However, loss is meant for the optimization algorithm (like SGD) to efficiently update the model parameters, while metrics are human-interpretable measures of performance. Sometimes, a metric may also be a good choice for the loss.