Models

Question 1

What do we model with supervised learning?

Answer 1

The relationship between independent and the dependent variables.

Question 2

When do we use regression to learn?

Answer 2

When the response (or DV’s) are numerical in supervised learning

Question 3

When do we use classification to learn?

Answer 3

When the response (or DV’s) is categorial in supervised learning
.

Question 4

What do we model with unsupervised learning?

Answer 4

The relationship between observed IV’s and some unobserved (latent) variables. (such that we can estimate the IV’s based on the laten variables.)

Question 5

When do we use clustering to learn?

Answer 5

When we want to find latent structure in unsupervised learning

Question 6

When do we use dimensionality reduction to learn?

Answer 6

To simplify datasets while retaining information in unsupervised learning.

Question 7

When asuming a model for our data, we disregard the error.

Answer 7

False, we asume that the results (DV’s) are dependent on error, whilst error is independent of the input variables (IV’s)

Question 8

What are reducible errors?

Answer 8

These are error margins in a model that can be reduced by choosing a different model or better predictors.

Question 9

What are irreducible errors?

Answer 9

These errors are always there as an upper bound on model quality. They exist due to the fact that we can never have a perfect model.

Question 10

How do we estimate the correct model for our data?

Answer 10

We use training data and an assumption of the class of models in order to estimate best model that fits.

Question 11

What are parametirc models?

Answer 11

Models where we asume a class of models by a fixed set of parameters, and we select the parameters such that it best fit the training data.

Question 12

What are non-parametric models?

Answer 12

We asume a class of models, but we dont asume their form. Therefore the training data can be seen as the parameters of the model.

Question 13

What is meant with simple models?

Answer 13

A parametric model: easy to interpret, often not flexible enough.

Question 14

What is meant with an complex models

Answer 14

A non-parametric model: complex to interpret, fits the training data quite well.

Question 15

What is overfitting in modelling?

Answer 15

When there were too many parameters in a non-parametric model, so the model perfectly fits the training data but produces very bad estimates with new data.

Question 16

Why does a model overfit?

Answer 16

When given enough parameters, the model will fit so it counters all error present in the training data. It has no knowledge of underlying processes, it only knows how to model the training data.

Question 17

How do we determine the accyracy of our model?

Answer 17

By testing our model with new data; that is different from the training data of the model, but comes from the same dataset as our training data!

Question 18

What is the purpose of a test data set?

Answer 18

In order to check if our estimated model based on the training set is still accurate when evaluated with an new data set.

Question 19

In modeling, do we want our training accuracy as low as possible?

Answer 19

No, this means the data is overfitted

Question 20

In modeling, do we want our test accuracy as low as possible?

Answer 20

Yes, this means the current model best reflects the data.

Question 21

What is model bias?

Answer 21

Bias refers to the inability of the estimated model to represent the underlying process.

Question 22

How can you reduce model bias?

Answer 22

By increasing the model complexity.

Question 23

What is model variance?

Answer 23

Variance refers to how much the estimated model varies with respect to the training data. (an different training set would result in a very different model when of equal complexity).

Question 24

How can we reduce model variance?

Answer 24

It is reduced by decreasing model complexity or increasing data size and diversity.

Question 25

Why would we regard to a bias-variance trade-off?

Answer 25

Because the optimal complexity for the estimated model lies (closely) on the intersection of the model’s bias and variance.

Question 26

Which colour represents the model’s Bias?

Answer 26

Purple: Bias decreases with complexity.

Question 27

Which colour represents the model’s Variance?

Answer 27

Green, the variance increases with complexity.

Question 28

In terms of bias and variance, when is the data underfitted?

Answer 28

high bias, and low variance

Question 29

In terms of bias and variance, when is the data overfitted?

Answer 29

low bias, and high variance

Question 30

What is the main difference between Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC)?

Answer 30

AIC penalizes only on model complexity, where BIC also acounts for the size of the data set

Question 31

Which is better AIC or BIC?

Answer 31

Neither. The term ‘better’ is always relative to the dataset and the goal of the study. It is always relative and very dependent on domain knowledge.

Question 32

What is cross validation?

Answer 32

A modelling technique that is used when there is no seperate test set available for a trained model. It allows us to estimate if an model would still be accurate against new data.

Question 33

How does k-fold cross validation work?

Answer 33

We randomly parition the training data into k folds of the same size: the k-th fold is a validation set, the rest is used for training. When all k iterations have been done, the cross validation accuracy of the

Question 34

What is the accuracy of the (k fold) cross validation?

Answer 34

The cross validation accuracy is the mean accuracy over all k folds.

Question 35

How can we use cross validation to find an optimal model?

Answer 35

By plotting the k-fold cross validation accuracy for one certain value of k, against the complexity of the model. At the minimum, take the top of the standard error and go straight back to decrease complexity without compromising performance.