Theory of Learning from Data

Question 1

Was ist eine Risk Function?

Answer 1

Question 2

Wie unterscheiden sich true (expected) risk und empirical risk?

Answer 2

• true expected risk ist das Risiko/der Verlust auf unbekannten Datan
• empiricak risk ist das Risiko/der Verlust auf bekannten Daten

Question 3

Was bedeutet VC(H)?

Answer 3

Question 4

Gib VC(H) an: Linear classifiers for d features plus a constant term b

Answer 4

d + 1

Question 5

Gib VC(H) an: Decision tree of rank r that defines Boolean functions
on n boolean variables

Answer 5

Question 6

Gib VC(H) an: Neural networks

Answer 6

VC(H) ≈ #parameters

Question 7

Gib VC(H) an: Linear classifier in 2D mit drei Punkten

Answer 7

VC(H) = 3

Question 8

Was ist Structural Risk Minimization?

Answer 8

Risk Calculation of different Models

Question 9

Wie geht der Satz von Bayes?

Answer 9

Question 10

Wie unterscheiden sich bayesian view und cost funtion view?

Answer 10

Question 11

Gib die Bayesian probabilistic formulation

Answer 11

Question 12

Wie hängen bayesian view und cost function view zusammen?

Answer 12

Question 13

Wie kann man Modellkomplexität verringern?

Answer 13

Question 14

Wie kann man Parametergrößen “restricten” (beschränken)

Answer 14

Question 15

Beschreib Regularizer (L2 norm)

Answer 15

Question 16

Beschreib Regularizer (L1 norm)

Answer 16

Question 17

Erkläre Cross Validation

Answer 17

Question 18

Which of the following statements on the different kinds of cross-validation are correct?
1. The leave-one-out method is a special form of k-fold cross-validation.
2. Cross-validation is used to find the best training data to train a model.
3. The bootstrap resampling technique involves dividing the dataset into multiple partitions, evaluating each subset individually as test data after training on the rest.
4. A major advantage of k-fold cross-validation is that it is a fast method to test the quality of the chosen model.

Answer 18

1

Question 19

The Vapnik Chervonenkis (VC) dimension of a classifier H is the cardinality of the smallest set that can be fully represented by H.
Ist das Wahr?

Answer 19

Nein, actually it is the largest set a classifier H can fully represent.

Question 20

Which of the following statements on VC theory are correct?
1. A larger model complexity implies a smaller empirical risk.
2. The effective model complexity is fixed during the course of training.
3. The empirical risk is a good measure for the generalization capabilities of a model.
4. Structural risk minimization balances empirical risk and VC dimension.

Answer 20

1 und 4