Lecture 1

Question 1

Key aspects of Data Mining

What is data mining, see last card

Answer 1

Trade-off processing time and memory

Computers as tool and with growing data

From unstructured data to structured knowledge

Question 2

What is large amount of big data?

Answer 2

volume
variety
velocity

Question 3

3 characteristics of volume (big data)

Answer 3

too big for manual analysis
too big to store in RAM
too big to store on disk

Question 4

3 characteristics of variety (big data)

Answer 4

variance
outliers, confounders, noise
different data types

Question 5

2 characteristics of velocity (big data)

Answer 5

results before data changes

streaming data

Question 6

What makes predictions possible?

Answer 6

associations between features/target

numerical: correlation coefficient
categorical: mutual infomation

Question 7

Supervised learning (2 types)

Answer 7

regression (predictior)

classification (classifiers)

Question 8

Unsupervised learning (2 types)

Answer 8

clustering

dimensionality reduction

Question 9

Learning

Answer 9

A program is said to learn from experience E on task T and a performance measure P if its performance at task T as measured by P improves with E.

Suppose your email program watches which emails you do or do not mark as spam and based on that learns how to better filter spam. What is E, T and P?

E = Watching you label email 
T =  Classifying emails spam/ham
P = The number (or fraction) of emails correctly classified as spam/ham

Question 10

characteristics supervised learning

Answer 10

trying to predict a specific quantity (like Dow Jones of tomorrow, is a e-mail spam or ham)

have training examples with labels

can measure accuracy directly

Question 11

characteristics unsupervised learning

Answer 11

not looking for something specific, you want to ‘understand the data’

looking for structure (or unstructured) patterns

does not require labeled data

evaluation usually indirect or qualitative

Question 12

description supervised learning

Answer 12

wa are giving labels to the data manually and it are the labels we want to predict as good as possible. The algorithm is giving supervision, examples, of what you want to see come out of it

Question 13

workflow supervised learning

Answer 13

collect data

label the data manually (target variable)

choose representatation

train the model to learn

evaluate

Question 14

what is meant by ‘representatation’ (workflow SL)

Answer 14

feature selection

possibly) convert to feature vector

Question 15

Split the set in …. and …. for ‘train model’ (workflow SL)

Answer 15

train set for learning

validation set for hyperparamater tuning

Question 16

what is meant by ‘evaluation’ (workflow SL) (2)

Answer 16

check performance of tuned model (/validated model) on test set

estimate how well model will do in the real world

Question 17

parameter or model tuning;

what is it?

for each value of hyperparameters you… (3)?

Answer 17

1. learning algorithms typically have settings (aka hyperparameters)

2a. apply algorithm to training set to learn
2b. check performance on validation set
2c. find/choose bes-performing setting (aka hyperparameter)

Question 18

label examples (3)

Answer 18

annotation guidelines

measure inter-annotator agreement

crowdsourcing

Question 19

persons’s r = correlation coefficient

See below for difference covariance and correlation

Answer 19

measures strength of a LINEAR and LINEAR relationship only (dependency)

correlation does never imply causation, discovery of corrleation can only suggest a causal relationship

Question 20

what does the values of pearson’s r mean?

Note
In statistics, when we talk about dependency, we are referring to any statistical relationship between two random variables or two sets of data.

Answer 20

1 = A value of 1 implies that a linear equation describes the relationship between X and Y perfectly, with all data points lying on a line for which Y increases as X increases.

1 / 0 = positive (higher values of x means tend to have higher values of y) linear correlation

0 = A value of 0 implies that there is no linear correlation between the variables.
NOTE that if X,Y are independent the correlation coefficient between X and Y is zero (X,Y uncorrelated). BUT, if the correlation coefficient between X and Y is zero (X, Y uncorrelated), that does not mean that X and Y are independent.
E.g. suppose Y = X^2. Then Y is completely determined by X, so that X and Y are perfectly dependent, so there is some statistical relationship, but just no linear one.

• 1 / 0 = negative (lower values of x tend to have lower values of y) linear correlation
• 1 = A value of −1 implies that all data points lie on a line for which Y decreases as X increases

Question 21

pearson’s r visually

Answer 21

-1 or 1 = a line pointing up or down, does not matter how steep (r= <= 1), as long as it is not horizontal.

values between -1 / 0 and 1 / 0 = some cloud of dots where you can draw a line in

0 = a round cloud of dots, point really far apart or some figure that does not make sense at all

https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Question 22

formula pearson’s r

Answer 22

Pearson’s correlation coefficient is the covariance of the two variables divided by the product of their standard deviations;

covariance / product of standard diviations

s. d. is the same as variance. We take the squared root of variance, because using the standard diviation makes correleation independent units aka not sensitive to scaling)
http: //www.datasciencemadesimple.com/pearson-function-in-excel/

Question 23

covariance;

meaning
formula (if sample)

Answer 23

measure of joint variability of two variables.

Sum(variance of X * variance of Y ) / N - 1

Question 24

variance

Answer 24

distance of a datapoint from it’s mean.

To calculate variance you need to sum and square al the variance (spread from its mean) / N - 1

Question 25

anscombe’s quartet

Answer 25

even if 4 datasets have the same summary statistics (mean, variance, correlation coefficient and best fit-line) they look the same visually.
But this is not the case. Effect of curvature and outliers can be huge.

Statistical descriptors are incomplete descriptors of the underlying data!

Question 26

regression models can be used to (2)

Answer 26

describe the relationship between random variables

predict the value of one variable based on another variable

Question 27

dependent variable

Answer 27

outcome, response, target (on y-axis)

Question 28

independent variable

Answer 28

input, predictors, features (on x-axis)

Question 29

a regression model models the relationship as a…..

Answer 29

parametrized function y = F(x)

x –> f() —> F(x)
you just put x in some function and you get the function of x.

this is the learning task, could be predictor or classifier

Question 30

classification

Answer 30

The model learns from the data input given to it and then uses this learning to classify new observation.

Each instance (point) has a class label, can be represented by feature vectors, e.g. [X1X2] (in case classes are defined by 2 features)

The classes are devided by a descision bounary

Question 31

Decision boundaries

Answer 31

A descision bounary is nothing more than the function of x ( so f(x), the learning task, the classifier, the model).

This function, this classifier, is trained on the dataset (labelled data points) to ‘draw’ a division between classes. ITERATIVE PROCESS

The decision boundary is considered to be a model of the separation between classes.

NOTE
There is a difference between the terms algorithm and a model.
Algorithm = a mathematical technique or equation (that is, a framework) with parameters.
Model = equation that is formed by using data to find the parameters in the equation of an algorithm.

Question 32

Decision boundaries can be 2 types;

Answer 32

linear (stright line)

non linear (wiggly line), depends on number of parameters and polynomials

Question 33

Dimensionality reduction

Answer 33

feature selection (select relevant features)

feature extraction (define relevant features) e.g. PCA, FA for
• Image Processing: edge detection
• From pixels to reduced set of features

https://www.youtube.com/watch?v=LDhqqxOVqV0

Question 34

feature selection (4) -dimensionality reduction

Answer 34

reduce complexity and easier interpretation

reduce demand on resources (computation/RAM)

reduce ‘curse of dimensionality’

reduce change of overfitting

Question 35

Pattern mining

Answer 35

Identifying rules that describe specific patterns within the data. For example, supermarkets used market-basket analysis to identify items that were often purchased together—for instance, a store featuring a fish sale would also stock up on tartar sauce.

Question 36

Difference Correlation and covariance

Answer 36

Covariance

• Tries to look into and measure how X and Y change together.
• tells us direction in which two variables vary with each other.
• Covariance can be classified as positive covariance (that large values of one variable are associated with big values from the other) and negative covariance (large values of one variable are associated with small values of the other one) + no trend

========================
Correlation.

• Both, the direction and magnitude of how X and Y vary with each other.
• Serves as a scaled version of a covariance, make covariance unitless, sort of normalise (same idea as variance and standard deviation)
• three categories: positive, negative, or zero

Question 37

What is Data Mining

Answer 37

Process of using computer science (algorithms and database methods) and statistics, to find applicable, usable knowledge (like patterns, trends and relationships) in raw (big) data.

We apply an algorithm that “learns” something about the data. These algorithms are machine learning algorithms.

NOTE
There is a difference between the terms algorithm and a model.
Algorithm = a mathematical technique or equation (that is, a framework) with parameters.
Model = equation that is formed by using data to find the parameters in the equation of an algorithm.