Lecture 4 - Data Understanding II

Question 1

What does data preparation do with the information provided by data understanding?

Answer 1

• Selects attributes
• Reduces the dimension of the data set
• Selects records
• Treats missing values
• Treats outliers
• Improves data quality
• Unify and transform data

Question 2

What is feature extraction?

Answer 2

The construction of new features from the given attributes

For example instead of tasks finished, hours worked, number of hours needed usually for task -> creating new attribute “efficiency”

Question 3

What can be used for simple models feature extraction?

Answer 3

Non-linear functions like x^p, divided by x, log(x), sin(x) etc.

Question 4

How to predict y from x?

Answer 4

Prior knowledge, is the y dependant on x, visualization, trial and error

Question 5

What’s the disadvantage of methods like PCA for feature extraction?

Answer 5

Dimensionality reduction techniques like PCA lead to features that can no longer be interpreted in a meaningful way, how to understand a feature that is a linear combination of 10 attributes?

Question 6

What are some complex data type feature extractions?

Answer 6

Text data analysis -> frequency of keywords
Time series data analysis -> fourier or wavelet coefficients
Graph data-analysis -> number of vertices, edges

Question 7

What does feature selection refer to?

Answer 7

Techniques used to choose a subset of the features that is as small as possible and sufficient for the data analysis

Question 8

What are the reasons for feature selection?

Answer 8

• Prior knowledge: we know something is irrelevant
• Quality control: majority of values missing or bad
• Non-informative: eg all values same
• Redundancy: Identical or correlated values

Question 9

What does record selection refer to? Why is it done?

Answer 9

Selecting only some rows of the data.
- Timeliness: older data might be outdated
- Representativeness: The sample in the database might not be representative for the whole population
- Rare events: Useful for something like stock market crashes

Question 10

How to choose records for rare events?

Answer 10

• Artificially increase the proportion of the rare events by adding copies
• Choose only subset of the data

Question 11

What does data cleansing refer to?

Answer 11

Detecting and correcting/removing inaccurate, incorrect or incomplete records from the data set

Question 12

How to improve data quality?

Answer 12

• Turn all characters same sensitivity
• Remove spaces etc.
• Fix the format of numbers
• Split the fields “Chocolate, 100g” -> “chocolate” “100.0”
• Normalize the writing

Question 13

What are the four discretizations?

Answer 13

1. Equi-width discretization: Splits range into same length intervals [0-20, 20-40, 40-60]
2. Equi-frequency discretization: Splits range into intervals with roughly the same number of records [4,4,4,4]
3. V-optimal discretization: Minimizes the sum of n*V, where n is the number of data objects and V is the sample variance
4. Minimal entropy discretization: minimizes the uncertainty

Question 14

Why should data sometimes be normalized?

Answer 14

To guarantee impartiality for models that use distances

Question 15

What is min-max normalization?

Answer 15

All the values are scaled between 0 and 1, outliers affect a lot.
x = x-min_x / (max_x - min_x)

Question 16

What is z-score standardization?

Answer 16

scales the data to have a mean of 0 and deviation of 1
x = x - mean(x) / variance(x)

Question 17

What is robust z-score standardization?

Answer 17

x = x - median(x) / IQR(x)

Question 18

What is decimal scaling?

Answer 18

For attribute X and the smallest integer value s larger than log_10(max(x))

x = x/10^s

Question 19

What does centering the data matrix mean?

Answer 19

Removing the mean from all the rows of matrrix X, it moves the data to the center.

Question 20

What the number of possible 2D scatter plots for attributes m?

Answer 20

m(m-1), so for 50 50*49 = 2450

Question 21

Why do we want to change data to lower dimensional?

Answer 21

There could be hundreds of thousands of attributes, to include them all in a plot we need to define a measure that evaluates lower-dimensional plots of daata in term of how well the plot preserves the original structure

Question 22

What are parallel coordinates?

Answer 22

They draw the coordinate axes parallel to each other, so that there is no limitation for the number of axes to be displayed.

Aka plot for multiple attributes like \/_/_

Question 23

What is the basic idea for dimensionality reduction?

Answer 23

Change the data from n-dimensional space to q-dimensional space (q= 2 or 3)

R^n -> R^q

Question 24

What is a linear map?

Answer 24

New attributes are linear combinations of old ones.

new_feature = 0.5feature_1 + 0.3feature_2

Question 25

How does PCA work?

Answer 25

PCA uses the variance in the data as the structure preservation criterion? It then tries to preserve as much of the original variance of the data when projected to a lower-dimensional space.

It uses an orthogonal transformation to convert a set of observation of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components.

Question 26

How does the principal components get determined?

Answer 26

The first PC has the largest possible variance, and each succeedin component in turn has the highest variance under the constraint that it is orthogonal to the preceding components

Question 27

Is PCA sensitive to the relative scaling of the original variables?

Answer 27

Yes, usually Z-score standardized

Question 28

What is an eigenvector?

Answer 28

The number of principle components.

Question 29

what is t-SNE?

Answer 29

t-distributed stochastic neighbor embedding. Non-linear dimensionality reduction method

Question 30

How does t-SNE work?

Answer 30

Similar items end up close together points and dissimilar at distant points
Generates clusters even when the data doesn’t support this

Question 31

What are the two stages of t-SNE?

Answer 31

1. A probability distribution over pairs of high-dimensional objects is constructed so that similar objects receive higher probability while dissimilar points receive lower probability
2. A similar probability distribution is generated for the points in the low-dimensional map

Question 32

What are some dimensionality reduction methods?

Answer 32

1. PCA
2. t-SNE
3. Kernel PCA (non-linear)
4. Linear discriminant analysis (used in classification, finds low dimensional reprsentation of the data such that separates classes well)