Perceptrons and FeedForward ANN's

Question 1

What is a perceptron?

Answer 1

An equation that takes a number of inputs, and gives a single output value

Question 2

What are the inputs to a perceptron?

Answer 2

The value of the input, multiplied by its weight (how valuable the input is).

Question 3

What does an activation function do?

Answer 3

It decides whether the output of one perceptron is enough to give as input to the next layer of perceptrons

Question 4

What are some issues with training a perceptron on some data?

Answer 4

There may be overfitting. This is when the model has perfectly separated the classes, meaning that it cannot be generalised (used on other data - it is useless)

Question 5

How do we estimate weights?

Answer 5

Initialise weights. Can be all 0, or a random number

Iterate through each sample in the training dataset
- Calculate the predicted output
- Update the weights based on this

Question 6

What is the relationship between input magnitude and ∆ weight?

Answer 6

The larger the input magnitude, the larger effect it has on ∆ weight

Question 7

What is learning rate?

Answer 7

It is a measure that controls how fast a network learns
It can be constant or adaptive

Question 8

What is the issue with the learning rate being too big?

Answer 8

The network may take a long time to reach the global minimum or oscillate around it forever

Question 9

What is the issue with the learning rate being too small?

Answer 9

The network may become trapped in a local minimum

Question 10

When do we stop training a network?

Answer 10

When the error falls below an acceptable threshold

When the change in instantaneous error falls below an acceptable threshold

When it has completed a predetermined amount of iterations