L2_NCC_Perceptron

Question 1

Cognitive functions have to be investigated on 3 levels?

Answer 1

• Computational Level
What does a cognitive function do?
• Algorithmic Level
What is the functional organization within a cognitive module?
• Implementational Level
What is the physical/physiological realization of this algorithm?

Question 2

Explain the 3 Steps of Nearest Centroid Classifier (NCC)

Answer 2

1. Calculate the class means of the Prototypes w1 and w2
2. find the linear classification boundary: w.Tx-ß = 0
   with w = w1 - w2 and ß =1/2 * (w1.Tw1 -w2.T*w2)
3. Classify the class label sign f(x)= +1 / -1

Question 3

Basic Artificial Neural Networks model (4 point)

Answer 3

• Input nodes xi receive information
• Inputs are multiplied with a weighting factor wi and summed up
• Integrated input is mapped through some (non-linear) function f (·)
• f(x)= +1 if x is preferred stimulus / -1 if x is any other stimulus

Question 4

Goal of the The Perceptron Learning Algorithm

Answer 4

Binary classification of multivariate data x ∈ RD

Question 5

Input of the The Perceptron Learning Algorithm

Answer 5

Learning rate η and N tupels (xn,yn) where

• xn ∈ RD is the D-dimensional data
• yn ∈ {−1, +1} is the corresponding label

Question 6

Output of the The Perceptron Learning Algorithm

Answer 6

Weight vector w ∈ RD such that w.T*xN =

≥0 if yn = +1 / < 0 if yn = −1

Question 7

What is a good w?

Answer 7

We need an error function
that tells us how good w is.
Then we chose w such that the error function is minimized.
Perceptron error EP is a function of the weights w, where M denotes the index set of all misclassified data xm

Question 8

Explain the Perceptron Learning Algorithm (2 steps)

Answer 8

1. Initialize wOLD (randomly, 1/n, …)
2. While there are misclassified data points
   Pick a random misclassified data point xm
   Descent in direction of the gradient at single data point xM:
   Em(w) = −w.⊤ * xM * yM
   ∇Em (w) = −xm ym

wNEW ← wOLD-η∇Em(wOLD) = wOLD + η * xM * yM

Question 9

[Novikoff, 1962; Rosenblatt, 1962] about The Perceptron Learning Algorithm

Answer 9

If there is a solution, the perceptron algorithm will find it in a finite number of steps

Question 10

Convergence on non-linearably-separable sets:

Answer 10

wnew ← wold + ηxmym
t→∞
• Proven for variable learning rate η(t), with η(t) → 0
• Best convergence speed is achieved for η(t) ∼ 1

Question 11

Problems with Nearest Centroid Classification

Answer 11

Non-linear Data

Correlated Data

Question 12

High accuracy but ”lousy” weight vector - why?

Answer 12

Misconception: signal channels (here: pixels) with large classifier weights are strongly related to the class label

The purpose of the weight vector is two-fold: amplify the signal of interest, while at the same time surpress signals of no interest.