Perceptron

Question 1

What is the learning rate?

Answer 1

This is how large of a step the AI takes each iteration

Question 2

What is the effect of slower learning rate?

Answer 2

It takes longer for the data to converge

Question 3

What is the effect of a higher learning rate?

Answer 3

> The AI learns faster

> It may become unstables

Question 4

How do human neurons fire, why?

Answer 4

> Neurons accumilate stimulus

> Neurons fire when the stimulous given to them is higher than a certain threshold

> Neurons are binary, they either fire or dont fire

Question 5

What is the model for the McCulloch and Pitts neuron?

Answer 5

> Several inputs

> Each input has a weight associated with it which indicates the importance of that input

> If the sum of weights and inputs is greater than a threshold, then it fires

Question 6

What is the equation for the weights and inputs of a neuron for the McCulloch and Pitts neuron?

Answer 6

∑ w_ix_i = w∙x

Question 7

What is the threshold equation for the McCulloch and Pitts neuron?

Answer 7

o(h_w) =

{ 1 if h_w > θ

{ 0 if h_w ≤ θ

Question 8

What is the decision boundary of the McCulloch and Pitts neuron?

Answer 8

h_w = θ

Question 9

What are the issues with the McCulloch and Pitts model of a neuron?

Answer 9

> Other neuron characteristics are not taken into account

> The sum of weights and inputs is not neccissarily what the neuron is doing

> Actual neurons do not fire in a single spike, instead they fire in a spike train

> Artificial networks are synchronous where real biological netowrks are asynchronous

Question 10

What is a functional aproximators and why is a neuron called a functional aproximator?

Answer 10

> It is something that can aproximate any function

> They are called it because they produce an aproximate function that is a boundary between classes

Question 11

What are the two approaches of AI models?

Answer 11

> Generative model

> Discriminative model

Question 12

What is a generative model?

Answer 12

A model that learns the underlying distribution of the data so it can generate more points

Question 13

What are examples (2) of using a generative model?

Answer 13

> 4k Upscaling

> Generating faces of not real people

Question 14

What is a discriminative model?

Answer 14

A model that only learns the decision boundary and not the distribution that underlays the data points

Question 15

What is an example of using a discriminative model?

Answer 15

> Image recognition

Question 16

What is a perceptron?

Answer 16

This is a collection (array) of neurons and input variables

Question 17

When should a perceptron be used?

Answer 17

When data is linearly separable by a hyper plane in n dimensions

Question 18

How is the number of neurons decided?

Answer 18

The number of classes being classified

Question 19

How do we calculate the number of parameters for a perceptron with multiple neurons?

Answer 19

Number of inputs × Number of neurons = number of connections

number of connections + number of outputs = number of parameters

Question 20

What is the basic error function?

Answer 20

> The total number of mistakes

> E(X) = ∑ | y_n - t_n |

• E(X) = Error
• y_n = Result of perceptron
• t_n = target of perceptron

Question 21

How does the basic error function work for each example?

Answer 21

> E(X) = ∑ | y_n - t_n |

> When they are different:

• y_n - t_n = 1 - 0 = 1
• y_n - t_n = 0 - 1 = -1
• | y_n - t_n | = 1

> When the same:

• y_n - t_n = 0 - 0 = 0
• y_n - t_n = 1 - 1 = 0
• | y_n - t_n | = 0

Question 22

What do we want to do with the weights to optimise the perceptron?

Answer 22

Adjust the weights until the error is minimised

Question 23

How do we use biased input?

Answer 23

Instead of having the threshold associated with each neuron, we can instead include this as an input to the neuron. This would make the equation for the sum:

h_w = ∑ w_ix_i - θ

and the output:

o(h_w) =

{ 1 if h_w > 0

{ 0 if h_w ≤ 0