Lecture 2 - Binary Classification

Question 1

Tasks, Instances, Instance Space, Label Space, Output space, model

What is the difference between instances and instance space

Answer 1

Instances: the objects of interest in machine learning described with features
Instance Space: the space of all possible instances

Question 2

Tasks, Instances, Instance Space, Label Space, Output space, model

What is the difference between Label Space and Output Space

Answer 2

Label Space: labels of example instances
Output space: the space of the outcomes (or targets) of the task

Question 3

Tasks, Instances, Instance Space, Label Space, Output space, model

What is a Model?

Answer 3

A mapping from the input space to the output space potentially using labelled examples

Question 4

What is a binary classifier?

Answer 4

Binary classifier has only two class labels (example: True or False)

Question 5

What is the term “learning a classifier”?

Answer 5

Learning a classifier involves constructing the function c^ such that it matched c as closely as possible

Question 6

What are the two ways of assessing performance of a binary classifier?

Answer 6

• Contigency table
• ill get back

Question 7

What is a contingency table?

Answer 7

A matrix that displays the (multivariate) frequency distribution of the variables. c(x) and c^(x)

Question 8

table looks like?

Answer 8

predicted + predicted -
actual + True positive False negative
actual - False positive True negative

Question 9

Contingency table

Accuracy?

Answer 9

acc = (TP+TN)/Total

Question 10

Contingency table

Error rate?

Answer 10

err = (FP + FN)/total

Question 11

Contingency table

What is an unbalanced dataset?

Answer 11

a dataset within which one or some of the classes have a much greater number of examples than the other

Question 12

Contingency table

What do we use in an unballanced dataset?

name 5 >:)

Answer 12

• True Positive rate, sensitivity, recall
• True Negative rate, specificity
• False Positive rate, false alarm rate
• Flase Negative rate
• Precision, confidence

Question 13

What is F-measure or F1 score?

Answer 13

F-measure is a performance metric that is not affected by negatives as a weighted harmonic mean of precision and recall

Question 14

what is harmonic mean used for

Answer 14

helps to find multiplicative or divisor relationships between fractions without worrying about common denominators. Harmonic means are often used in averaging things like rates (e.g the average travel speed given a duration of several trips).

Question 15

What is the F-measure formula?

Answer 15

F1 = 2(prec x rec)/(prec + rec)

Question 16

What are train/test splits?

Answer 16

The test set is used to make predictions during the inference phase on data not used during the trainig phase

Question 17

What are folds

Answer 17

a process when all data is randomly split into k folds
train, train,train,train,test
train, train, train,test,train
train, train,test,train,train,

Question 18

How can we aggregate the folds?

Answer 18

1/N * sum(scores)

Question 19

What if validation set for?

Answer 19

To test hyperparameters.
To decrease bias from train/test.
train, train,tarin,valid,test
Cross-validation

Question 20

define train set error and test set error

Answer 20

• Train set error is measured on the validation set
• Test set error is measured on the test set

Comparing these metrics helps knowing if a model is over-fitting on the training set

Question 21

What is overfitting and underfitting

Answer 21

1. Over-fitting: where a model performed well on trainig data but poorly on test data
2. Under-fitting: models poorly bad on both

Question 22

What is generalisability in terms of overfitting and underfitting

Answer 22

both over-fitting and under-fitting presence shows low generalisability

Question 23

What is ROC and coverage plot for?

Answer 23

A coverage plot and a Reciever Operation Curve (ROC) help summarize all different confusion matrices.

Question 24

what are the axis for coverage plot and roc

Answer 24

coverage plot: positives/negatives
roc: True pos ratio/False pos ratio
or sensetivity/(1-specificity)

Question 25

What is AUC Area under the curve

Answer 25

Area under the curve used as summary of the model skills

Question 26

What is a scoring classifier

Answer 26

a classification score is any score or metric the algorithm is using (or the user has set) that is used in order to compute the performance of the classification. Ie how well it works and its predictive power.

Question 27

what is the line seperating correct and wrong classifications called?

Answer 27

the margin

Question 28

What is the name of the function that maps example’s margin to an associated loss. Rewards large posistive margins and penalizes large negative values

Answer 28

Loss function

Question 29

Error rate (ERR) ?

Answer 29

Error rate (ERR) is calculated as the number of all incorrect predictions divided by the total number of the dataset.

Question 30

What is a probability estimator

Answer 30

A class probability estimator is a scoring classifier that outputs probability vectors over classes.
The output of the classifier shows how likely it is that the instancebelongs to a specific class rather than which class it will belong to.

Question 31

Squared Error (SE)?

Answer 31

look it up

Question 32

MSE?

Answer 32

look it up