Neural Networks

Question 1

process input signals and can be activated

Answer 1

Neurons

Question 2

_____ are connected to and receive electrical signals from other ____

Answer 2

Neurons

Question 3

a mathematical model for learning inspired by biological neural networks

Answer 3

Artificial Neural Network

Question 4

How do Artificial Neural Networks work?

Answer 4

Artificial neural networks model mathematical functions that map inputs to outputs based on the structure and parameters of the network. In artificial neural networks, the structure of the network is shaped through training on data.

Question 5

Different activation functions

Answer 5

• step function
• logistic function
• rectified linear unit (ReLU)

Question 6

gives 0 before a certain threshold is reached and 1 after the threshold is reached.

Answer 6

step function

Question 7

outputs any real number from 0 to 1, thus expressing graded confidence in its judgment.

Answer 7

logistic function

Question 8

allows the output to be any positive value. If the value is negative, the function sets it to 0.

Answer 8

rectified linear unit (ReLU)

Question 9

an algorithm for minimizing loss when training neural networks

Answer 9

gradient descent

Question 10

Algorithm of gradient descent

Answer 10

• Start with a random choice of weights. This is our naive starting place, where we don’t know how much we should weight each input.
  Repeat:
• Calculate the gradient based on all data points that will lead to decreasing loss. Ultimately, the gradient is a vector (a sequence of numbers).
• Update weights according to the gradient.

Question 11

gradient is calculated based on one point chosen at random. This can be quite inaccurate.

Answer 11

Stochastic Gradient Descent

Question 12

computes the gradient based on on a few points selected at random, thus finding a compromise between computation cost and accuracy

Answer 12

Mini-Batch Gradient Descent

Question 13

Only capable of learning linearly separable decision boundary

Answer 13

perceptron

Question 14

Alternative to model data non-linearly

Answer 14

multilayer neural networks

Question 15

an artificial neural network with an input layer, an output layer, and at least one hidden layer

Answer 15

multilayer neural networks

Question 16

main algorithm used for training neural networks with hidden layers.

Answer 16

Backpropagation

Question 17

neural networks that have more than one hidden layer.

Answer 17

deep neural networks

Question 18

algorithm of backpropagation

Answer 18

• Calculate error for output layer
• For each layer, starting with output layer and moving inwards towards earliest hidden layer:
  * Propagate error back one layer. In other words, the current layer that’s being considered sends the errors to the preceding layer.
  * Update weights.

Question 19

combatting overfitting in neural networks

Answer 19

dropout

Question 20

In this technique, we temporarily remove units that we select at random during the learning phase.

Answer 20

dropout

Question 21

encompasses the different computational methods for analyzing and understanding digital images

Answer 21

Computer Vision

Question 22

applying a filter that adds each pixel value of an image to its neighbors, weighted according to a kernel matrix.

Answer 22

image convolution

Question 23

neural network that uses convolution, usually for analyzing images.

Answer 23

convolutional neural network

Question 24

reducing the size of an input by sampling from regions in the input

Answer 24

pooling

Question 25

pooling by choosing the maximum value in each region

Answer 25

max-pooling

Question 26

neural network that has connections only in one direction

Answer 26

feed-forward neural network

Question 27

neural network that generates output that feeds back into its own inputs

Answer 27

recurrent neural network