Hoorcollege 2 Regression

Question 1

Machine learning problems

Answer 1

• Regression: from data to scores:
• x = [x1, x2, x3] assign it to a score z = wx + b

learning: minimizing loss on training data

Question 2

Gradient (descent)

Answer 2

wat als parameter w veranderd bij dw, hoeveel veranderd de loss functie?
* n is the learning rate

−𝜂 𝑑/𝑑𝑤 𝐿(𝑦̂,𝑦)

Loss function: squared error

Question 3

Logistic regression

Answer 3

Score is a label (good / bad)
* Classification is regression

use probabilities for training
* Pgood([23,5,1]) ≈ 0
From a score to probability:
* sigmoid = 𝜎(𝑧) = 1 / (1 + 𝑒 ^(−𝑧) )

Question 4

Other error functions

Answer 4

• Squared error: 𝐿_𝑀𝑆𝐸 (𝑦̂,𝑦)=(𝑦̂−𝑦)^2
• Mean squared error: error high for big distances
• Surprisal: S(p) = - log(p)
  Error is high for is low probabilities
• Cross entropy loss

Extending to mulitple classes:
* softmax(𝑧_𝑖 )=𝑒^(𝑧_𝑖 )/(Σ_(𝑗=1)^𝐾 𝑒^(𝑧_𝑗 ) )

In practice: Pytorch