Data Science MODULE 3

Question 1

What does it mean when a statistical model has been overfit?

Answer 1

It mirrors the training data extremely well, but doesn’t perform well with unseen data

Question 2

Hoekom is overfitting so n groot probleem

Answer 2

So die doelwit van n data scientist is om responses te predict met n geleerde model. As j overfit, het dit n direkte negatiewe impak op jou vermoe om te predict

Question 3

In tree-based models, the algorithm is programmed to?(2)

Answer 3

• reduce the impurity of partitions in a classification problem
• reduce the mean squared error in regression problems

Question 4

When fitting data we need to find the balance between?

Answer 4

Bias and variance

Question 5

What is variance?

Answer 5

It indicates an error related to how robust the model is when unseen data is used

Question 6

What does it mean when a model has high variance?

Answer 6

Performs well on training data, but not the test set - i.e the model is overfitted

Question 7

The ideal model in terms of variance and bias

Answer 7

Low variance and low bias

Question 8

Models with a high bias, tends to be?

Answer 8

Too simple, they don’t capture the shape of the data

Question 9

Kan jy die variance en bias meet?

Answer 9

Nope, omdat ons nie weet hoe die onderliggende data werklik lyk nie

Question 10

Wat is n validation error?

Answer 10

Soos ek verstaan, train jy die model. Maar jy hou ook test data eenkant. As hy klaar is, toets jy dit met die unseen data en bepaal dan die fout. Hierdie staan bekend as die validation set approach

Question 11

As die validation error hoog is, maar die training error is small?

Answer 11

You have done it, you have overfitted the data

Question 12

So daar is eintlik drie stelle data waarmee ons train

Answer 12

Initial training data, validation data en dan test data

Question 13

So met fitting, hoe word die validation data actually gebruik?

Answer 13

Ons probeer die beste waardenkry vir die hyperparameter (denoted as alpha)

Question 14

Different hyperparameters are assessed to determine the one that results in?

Answer 14

The lowest validation error

Question 15

So wanneer sal ons dan nou die test data begin gebruik?

Answer 15

Wanneer die validation error geminimise is, deur die hyperparameter aanpassings

Question 16

Two of the most common types of validation used?

Answer 16

K-fold cross validation
Standard

Question 17

Hoe lyk die data split vir standard validation

Answer 17

70% op die training, 20% op die validation, 10% op die test set

Question 18

Oorhoofs, hoe werk k-fold cross validation

Answer 18

Aanvanklik word data net gesplit in training data en n test set. Die training data word dan verdeel in k hoeveelheid “folds”. Een word gebruik vir die validasie. Die data word randomly assign aan elke fold, so die oorhoofse verspreiding behoort redelik eweredig te wees

Question 19

So as jy five folds het, hoe werk die training?

Answer 19

Elke fold kry n kans om die validation set te wees. As die data klaar gebruik was vir die validasie, word dit weer gebruik vir training. So die model word vyfkeer getrain. Die final validation error is dan die gemiddeld van die 5 errors.

Question 20

2 Voordele kan k-fold validation?

Answer 20

1. Jy kry n beter idee van die final error
2. Gee jou n beter idee van die variasie in die validation error

Question 21

So we do we manage complexity with tree-based models?

Answer 21

Via pruning and validation sets

Question 22

Wat is die hoof doel van cost complexity pruning?

Answer 22

To reduce overfitting by penalising large trees

Question 23

How does the penalising work with cost complexity pruning

Answer 23

So n penalty term word ingesluit. Hierdie penalty term sluit in die hyperparameter, sowel as |T|, wat die aantal nodes is op die terminale vlak

Question 24

Oorhoofs, wat is die drie stappe wat gevolg word om n decision tree te prune?

Answer 24

1. Generate die groot boom
2. K-fold cross validation to find the optimum hyperparameter
3. Apply cost complexity pruning to the large tree with the optimum alpha value

Question 25

As the hyper parameter is increased?

Answer 25

Branches are pruned from the tree

Question 26

So die tree wat oorbly na pruning, staan bekend as die

Answer 26

Subtree, en dit is afhanklik van die alpha waarde

Question 27

For a given alpha,

Answer 27

The subtree is found that minimises the penalised cost function

Question 28

So hoe vind ons nou die beste alpha waarde?

Answer 28

Deur k-fold validation te gebruik, per alpha waarde, en kyk waar ons die laagste gemiddelde error kry

Question 29

Met k-fold validation, hoe lyk die modelle wat ontwikkel word met elke stap?

Answer 29

Verskillend, omdat ons elke keer n ander subset van die data gebruik

Question 30

So as ons nou klaar die k-fold validation gedoen het om die optimum alpha te bepaal, hoe kry mens die finale decision tree?

Answer 30

Deur die hele training set te retrain met die optimum alpha

Question 31

Kan j cost-complexity pruning doen in Python

Answer 31

Nee, so dis bietjie pointless eintlik. In python kan ons pre-pruning doen, of early stopping om kleiner bome te kry

Question 32

Jy kry ook pruning by depth (deur in python die max_depth parameter te verander met scikit-learn)

Answer 32

Jip, die akkuraatheid word brpaal as n funksie van diepte, die diepte met die hoogste score tydens validasie word gespesifiseer as die maksimum diepte in die finale model

Question 33

Wat is van die ander pre-pruning methods?

Answer 33

Minimum samples per node, maximum number of terminal nodes, minimum purity increase, minimum samples per split

Question 34

The resubstitution validation technique

Answer 34

Niks snaaks. Basies, die fout, of die error wat bereken is vir daardie datastel

Question 35

Wat is hold-out validation

Answer 35

Gebruik stratification, so n seker gedeelte van die data word uitgehou vir validasie doeleindes (soos dit is die standard validation wat vroeer bespreek is)

Question 36

Met k-folds validation, hoeveel folds wors gebruik vir training?

Answer 36

K-1, die laaste een is dan die toets data. Proses word dan herhaal

Question 37

Wat is LOOCV

Answer 37

Leave one out cross validation Basies k-folds op steriods, ipv n fold gebruik, word daar n iterasie gedoen vir elke record

Question 38

Wat is random subsampling validation technique?

Answer 38

Soortgelyk aan k-folds en LOOCV, in hierdie geval doen jy net n aantal iterasies, maar training set is random data test set os wat oorbly

Question 39

Bootstrapping validation technique?

Answer 39

Bietjie chaos, basies dieselfde as random subsampling, maar daar kan ook herhalings wees van sekere rekords