Data Types

Question 1

3 types of data

Answer 1

quantitative, qualitative, textual

Question 2

Quantitative data (1 word)

Answer 2

numerical

Question 3

Qualitative data (1 words)

Answer 3

categorical

Question 4

2 types of quantitative data

Answer 4

continuous and discrete

Question 5

numerical qualitative data

Answer 5

not mathematical, category numbers

Question 6

2 types of qualitative data

Answer 6

ordinal (ordered), norminal (un ordered)

Question 7

1st step of investigating data

Answer 7

examine the univariate statistics

Question 8

purposes (3) of starting with univariate stats

Answer 8

1) detect distribution anomalies 2) get an idea of some orders of magnitude 3) see how to discretize the continuous variables, if needed

Question 9

type of plot for univariate review of discrete or qualitative data

Answer 9

frequency table

Question 10

type of plot for univariate review of continuous

Answer 10

box blot, taking note of extreme percentiles

Question 11

purposes (3) of bivariate analysis

Answer 11

1) incompatible variables 2) links between dependent (target) variable and independent 3) links between independent variables

Question 12

simple table for bivariate analysis

Answer 12

contingency table

Question 13

2nd step of investigating data

Answer 13

Rare or Missing values

Question 14

problem with rare values

Answer 14

can create bias in factor analysis or skew in measures of center

Question 15

dealing with rare values

Answer 15

remove or replace with more frequent value

Question 16

problem with missing values

Answer 16

1) may not be random, skewing data 2) aggregates over multiple variables

Question 17

dealing with missing values (4 options)

Answer 17

1) remove records 2) remove/replace variable 3) replace value 4) treat ‘missing’ qualitative data as it’s own value

Question 18

when missing values >= 15-20% of values

Answer 18

cannot use replace values or treat missing data as it’s own value

Question 19

Statistical replacement of the missing values uses a process called

Answer 19

imputation

Question 20

simplest method of imputation

Answer 20

replace missing value with most frequent value or mean/median

Question 21

most widespread imputation model (simple imputation)

Answer 21

each missing value is replaced with an assumed value

Question 22

multiple imputation

Answer 22

missing vales are replaced with multiple plausible values creating several complete data tables

Question 23

3rd step of investigating data

Answer 23

Aberrant Values

Question 24

define aberrant value

Answer 24

erroneous value: can be caused by incorrect measurement, calculation error, input error, false declaration.

Question 25

extreme values and aberrant values relationship

Answer 25

extreme values not always aberrant, aberrant values not always extreme.

Question 26

tools for detecting aberrant values

Answer 26

frequency tables, univariant statistics

Question 27

dealing with aberrant values (4)

Answer 27

1) delete records 2) delete/replace variable 3) replace value 4) tolerate small margin of error and keep values as are.

Question 28

4th step of investigating data

Answer 28

extreme value

Question 29

what situations are tolerant of extreme values

Answer 29

1) decision trees 2) where rare profiles are the subject of study (fraud prediction, etc)

Question 30

what situations are especially intolerant of extreme values

Answer 30

continuous variables used in logistic regression, PCA, and variance.

Question 31

dealing with extreme values (#1)

Answer 31

exclude outliers from model learning sample (ensuring not more than 1-2% are excluded)

Question 32

dealing with extreme values (#2)

Answer 32

divide continuous variable into classes

Question 33

dealing with extreme values (#3)

Answer 33

Winsorize the variable- values beyond 1st or 99th percentile are given 1st and 99th percentile values.

Question 34

Step 5 of investigating data

Answer 34

tests of normality

Question 35

why normality matters

Answer 35

normal distribution is required for all parametric and linear regression methods.

Question 36

Tests of normality: when to use Shapiro–Wilk

Answer 36

uses a P-P plot with a diagonal line that represents normality. small samples (under 2000)

Question 37

Tests of normality: when to use Kolmogorov–Smirnov

Answer 37

very general, typically will use variants

Question 38

Tests of normality: when to use Anderson–Darling test

Answer 38

Variant of Kolmogorov-Smirnov that corrects less-sensitive tails

Question 39

Tests of normality: when to use Lilliefors test

Answer 39

Variant of Kolmogoro-Smirnov for cases when mean and variance are estimated from sample data

Question 40

define homoscedasticity

Answer 40

the property of having equal statistical variances

Question 41

homoscedasticity with a single independent variable in a discrimination model

Answer 41

equality of the variances of the variable in a number of samples, for example in the different groups of a population

Question 42

homoscedasticity with more than one independent variable in a discrimination model

Answer 42

equality of the covariance matrices of the variables in a number of samples

Question 43

homoscedasticity in linear regression

Answer 43

variance of the residuals does not depend on the value of the predictors