L16 - Neural Networks 2: Learning in ANNs

Question 1

Define supervised learning and unsupervised learning in context of a neural network. What’s the difference?

Answer 1

Supervised learning -> The output of the neural network is compared against known correct data. The accuracy of the model can then be determined and fine tuned.

Unsupervised Learning -> Used to establish patterns in data. No comparator output data. Output of the neural network is analysed to identify trends.

• The difference is that supervised is used mainly for prediction and classification purposes whereas unsupervised is used do identifying patterns in data.

Question 2

What are the components of a Perceptron?

Answer 2

1. A set of weighted connections that input data and associated weights into the perceptron.
   2. The activation function that operates the input data.
   3. The output axon, that outputs the function result.

Question 3

What does the weights of each input data indicate?

Answer 3

The importance of that input

Question 4

What type of learning is a perceptron for?

Answer 4

Supervised

Question 5

Is a Perceptron an algorithm?

Answer 5

Yes. It’s a supervised learning algorithm mainly used for classification.

Question 6

What does the threshold function do?

Answer 6

Gives a binary output

Question 7

What does the threshold function not do?

Answer 7

Give any information regarding error.

Question 8

What are the 4 steps of learning in the Perceptron?

Answer 8

1. Initialise input weights and the threshold function.
2. Present the input data as well as the expected output data.
3. Calculate the output of the neural network…
   1. Multiply each X by the corresponding weight W.
   2. Sum all XW’s
   3. Feed sum into activation function
   4. Get output
4. Adjust weights based on amount of error between output and expected…
   * Lots of error → Make big changes.
   * Small error → Small incremental changes.

Question 9

What are the formats of the weight update function?

Answer 9

1. If the output Y is correct -> Wi(t+1) = Wi(t)
2. If Y == 0, but should == 1 -> Wi(t+1) = Wi(t) + Xi(t)
3. If Y == 1, but should == 0 -> Wi(t+1) = Wi(t) - Xi(t)

Question 10

If the threshold function is OR, do we always floor the sum of the XW’s?

Answer 10

Yes… See letter notes for examples

Question 11

See lecture notes for examples of perceptron learning with XOR and OR activation functions…

Answer 11

…

Question 12

What parameter is used to control the learning rate of the weight update function?

Answer 12

n

E.g. Wi(t+1) = Wi(t) + nXi(t)

Question 13

Define the Widrow-Hoff learning rule…

Answer 13

Weight updates of n are proportional to the error made.

Question 14

What are some limitations of the single-layer perceptron? What is the solution to this?

Answer 14

• Can only solve linearly separable problems i.e Can’t solve XOR classification
• Multi-layer Perceptron

Question 15

How does the multi-layer perceptron solve the issues of the single-layer perceptron?

Answer 15

1. By adding hidden perceptron layers between input and output layer
2. Activation function used provides information about error
3. Error can be minimised as data is being passed through the algorithm

Question 16

What is the sigmoid function?

Answer 16

An activation function that can be used in MLP

Question 17

What are the 2 learning algorithms that can be utilised in MLP?

Answer 17

Back propagation

Feed forward

Question 18

How does the feed forward learning algorithm work?

Answer 18

1. Initialise weights and thresholds to small random values.
2. Feed data
3. Multiple XiWi for all X’s
4. Feed sum of XiWi into signed
   activation function
5. Pass output onto next layer

Question 19

How does the back propagation learning algorithm work?

Answer 19

An optimisation algorithm in which weights are adapted starting from the output layer and working backwards using the sigmoid derivative.

Question 20

What are the 2 types of weight updating?

Answer 20

Batch -> Weightsare updated after a complete iteration of the entire dataset. Calculating all errors.

Online -> Weights are updated after each data point has been processed, based on the error of that data point.