12-perceptron

Question 1

What is the perceptron?

Answer 1

The perceptron is a minimal neural network. Neural networks are composed of neurons

Question 2

What is a neuron?

Answer 2

Neuron is defined as:
- Input: a vector of numeric inputs
- Output: a scalar
- Hyper-parameter: an activation function
- Parameters: set of parameters per input

Question 3

What is the perceptron model?

Answer 3

f(theta^Tx) = 1, if theta^tx >= 0, else -1

f is a step function

Question 4

What does learning mean in the context of the perceptron model?

Answer 4

Learning means to modify the parameters of the perceptron so that performance is optimised

Question 5

What does the percepton algorithm do?

Answer 5

Iterate over the training data:
- if label is actual do nothing
- if true label is 1 and modelled is -1, then decrease weights
- if true label is -1 and modelled is 1, then increase weights

Question 6

How does the update formula work for the perceptron algorithm?

Answer 6

theta_j = theta_j + eta(y-y^)*x_j

Question 7

What does the perceptron algorithm guarantee?

Answer 7

The perceptron algorithm guarantees convergence for linearly separable data

Question 8

What is an online algorithm?

Answer 8

Updates the weights after each training example. The perceptron is an online algorithm

Question 9

What is a batch algorithm?

Answer 9

Computes statistics of the whole training data set and updates all parameters at once