8+9

Question 1

What is the main limit of perceptrons? What is a way to fix this?

Answer 1

A perceptron splits data into two half spaces, if the boundary needs to be non linear this is impossible. If we make the perceptron multi layered we can get around this.

Question 2

What is a layer? What is a single layer network good for?

Answer 2

A number of neurons connected to the same inputs(with different weights) forms a layer. A single layer network is good for multi-class classification.

Question 3

How does error backpropagation work?

Answer 3

Take the error associated with output and distribute amongst units which provided input, the strongest connected inputs take the most blame. This continues until at the input layer. Once all blame has been distributed each neuron will change its weights and biases to reduce its share of blame.

Question 4

Can multi layer networks be done with a hard limiting function?

Answer 4

No they need a sigmoid activation function(or other continuous function with a non zero derivative).

Question 5

What are all the steps in the backpropagation algorithm?

Answer 5

1. Initial guess all parameter values.
2. forward phase, make forward computation to determine output, save output at each layer.
3. Compute cost based on network output and desired output.
4. backpropagate blame for error in network and update parameters.
5. go back to step 2.

Question 6

What are the main activation functions?

Answer 6

Sigmoid(0,1) , tanh(-1, 1), linear, rectified linear unit(linear if value is not less than 0. 0 in this case.)

Question 7

Which output neuron should be chosen for classification problems?

Answer 7

The one with the greatest confidence.

Question 8

What is the cross-entropy cost?

Answer 8

A cost system better than MSE for classification problems.

Question 9

What is the softmax activation function?

Answer 9

Neuron output from activity is normalised by total activity of all output neurons, producing a probability distribution over classes?

Question 10

What is a feed forward neural network What about a recurrent neural network?

Answer 10

Feed forward is a directed acyclic graph, meaning the internal state only depends on the current input. A recurrent neural network includes cycles, allowing the model to show a dynamic temporal behaviour.

Question 11

What does a SRN involve?

Answer 11

Tanh activity function in hidden layer, softmax in output. Is a recurrent network, hidden layer feeds back in on self.