Theory of Learning from Data Flashcards
Was ist eine Risk Function?
Wie unterscheiden sich true (expected) risk und empirical risk?
- true expected risk ist das Risiko/der Verlust auf unbekannten Datan
- empiricak risk ist das Risiko/der Verlust auf bekannten Daten
Was bedeutet VC(H)?
Gib VC(H) an: Linear classifiers for d features plus a constant term b
d + 1
Gib VC(H) an: Decision tree of rank r that defines Boolean functions
on n boolean variables
Gib VC(H) an: Neural networks
VC(H) ≈ #parameters
Gib VC(H) an: Linear classifier in 2D mit drei Punkten
VC(H) = 3
Was ist Structural Risk Minimization?
Risk Calculation of different Models
Wie geht der Satz von Bayes?
Wie unterscheiden sich bayesian view und cost funtion view?
Gib die Bayesian probabilistic formulation
Wie hängen bayesian view und cost function view zusammen?
Wie kann man Modellkomplexität verringern?
Wie kann man Parametergrößen “restricten” (beschränken)
Beschreib Regularizer (L2 norm)
Beschreib Regularizer (L1 norm)
Erkläre Cross Validation
Which of the following statements on the different kinds of cross-validation are correct?
1. The leave-one-out method is a special form of k-fold cross-validation.
2. Cross-validation is used to find the best training data to train a model.
3. The bootstrap resampling technique involves dividing the dataset into multiple partitions, evaluating each subset individually as test data after training on the rest.
4. A major advantage of k-fold cross-validation is that it is a fast method to test the quality of the chosen model.
1
The Vapnik Chervonenkis (VC) dimension of a classifier H is the cardinality of the smallest set that can be fully represented by H.
Ist das Wahr?
Nein, actually it is the largest set a classifier H can fully represent.
Which of the following statements on VC theory are correct?
1. A larger model complexity implies a smaller empirical risk.
2. The effective model complexity is fixed during the course of training.
3. The empirical risk is a good measure for the generalization capabilities of a model.
4. Structural risk minimization balances empirical risk and VC dimension.
1 und 4
