Lesson 1 - Perceptron Learning

Question 1

What is Intelligence?

Answer 1

The ability to achieve goals in a wide range of environments

Question 2

Give a definition of machine learning.
The definition I’m searching for is the one from Arthur Samuel (Stanford, 1959)

Answer 2

“Field of study that gives computers the ability to learn without being explicitly programmed”

Question 3

What are some characteristics of real neurons (in the human brain)

Answer 3

• Basic Computation
• Highly interconnected
• Has inhibition/excitation connections
• Possesses a state
• Outputs spikes

Question 4

Explain the components of a (real) neuron

Answer 4

Dendrite takes input
Cell body does some aggregation on input
Then outputted thru axon terminal

Question 5

What are some characteristics of artificial neurons?

Answer 5

• Basic Computation
• Has inhibition/excitation connections
• Building Block
• Time-independent state
• Outputs real values

Question 6

What are some differences between artificial neurons and biological ones?

Answer 6

Biological ones have a state and output a spike
Artificial ones have a time-independent state (so no state) and output a real value

Question 7

What was the first artificial neuron?

Answer 7

The McCulloch-Pitts Neuron, otherwise known as the MP-neuron

Question 8

Give some characteristics of the MP-neuron

Answer 8

• Inputs are boolean values (inhibition/excitation)
• Two internal functions: g for aggregation, f for threshold
• Threshold parameter theta
• Output is binary

Question 9

What are some limitations of the McCulloh-Pitts neuron? The first neuron artificial neuron

Answer 9

• Only boolean inputs which is most likely not the case in real life
• The threshold is manually indicated
• Input is not weighted –> everything has same relevance
• What about functions that are not linearly separable?

Question 10

After the McCulloh-Pitts (MP) neuron, came another one. Which one and what were the extensions?

Answer 10

The Perceptron Model:

• Weights indicate relevance
• Real inputs are supported
• Threshold (bias) is learnable

Question 11

How do we learn the weights in the perceptron model?

Answer 11

There is an algorithm for it.
- weights are initialized randomly
- iterate over the training data and adjust the weights when needed
- convergence proof

Question 12

What are the task (T), experience (E) and metric (M) for the perceptron learning algorithm?

Answer 12

Task: find the weights –> get the best weights

Experience: the while loop

Metric: convergence check –> both if statements