03_data and features

Question 1

What is data?

Answer 1

-Info output by sensing devide or organ
-includes both useful and irrelevant or redundant info
-must be processed to be meaningful

• info in digital form that can be transmitted or processed

-factual information (eg measurements or statistics)
-used as a basis for reasoning, discussion or calculation

Question 2

What is the pipeline/process associated with data? (3 steps)

Answer 2

1) data acquisition
2) data storage (used to be a bottleneck)
3) data analysis

Question 3

Is all existing data technically accessible for analysis?

Answer 3

No, most of it is privately owned

Question 4

What are two types of data?

Answer 4

1) structured data
2) unstructured data

Question 5

What is structured data?

Answer 5

preprocessed and formatted data that is easily queryable

eg quantitative data in a table

most data analysis techniques require data to be available in a structured form for easier processing

Question 6

How is structured data represented?

Answer 6

Always in a database schema
(eg a table in 2 dimensions)

Question 7

What is unstructured data?

Answer 7

Unprocessed and unformatted data is not easily queryable

eg qualitative data, textual data, image data, data stream, audio data, video data (with increasing data complexity)

Question 8

What is quantitative data?

Answer 8

can be measured,
distances can be defined

Question 9

What are two kinds of quantitative data?

Answer 9

1) continuous data
2) discrete data

Question 10

What is continuous data?

Answer 10

real-valued numbers;
potentially within a given range

eg
- temperatures
- a person’s height
- prices

Question 11

What is discrete data?

Answer 11

discrete numbers;
whole numbers or real numbers;
potentially within a given range

eg
- number of people in a room
- inventory counts

Question 12

What is qualitative (categorical) data?

Answer 12

cannot be measure,
distances not defined

Question 13

What are two types of qualitative data?

Answer 13

1) nominal data
2) ordinal data

Question 14

What is nominal data?

Answer 14

Labels for different categories
without ordering

eg
- color of hair
- names of persons
- types of fruit

Question 15

What is ordinal data?

Answer 15

Labels for different categories
following an inherent ranking scheme

eg
- rank in a competition
- grades
- day of the week

Question 16

What is feature engineering?

Answer 16

Turning unstructured data into structured data

Question 17

Why do we need feature engineering?

Answer 17

Before ML methods can be applied to unstructured data, we have to process those and extract useful features from them

Question 18

What are features?

Answer 18

features are quantitative and independent variables
based on which our ML model learns

Question 19

What is the process of data analysis?

Answer 19

1) raw data (qualitative)
- feature engineering
2) features (quantitative), x
- input
3) ML model, f(x)
- output
4) target (supervised setup), y

Question 20

What does feature engineering do?

Answer 20

extract or create features that may provide a ML model with rich info on its task based on domain knowledge

can be applied to raw data, resulting in quantitative data that can be directly fed into the ML model (features)

Question 21

What is feature engineering for quantitative data?

Answer 21

create meaningful features through mathematical transformations

Question 22

What are examples for mathematical transformations for feature engineering with quantitative data?

Answer 22

• arithmetic:
  eg for difference of two variables
• aggregation of features:
  eg for aggregation of two business units to one overall result
• geometric transformations:
  eg to identify common wind speed patterns, vector-calculations

Question 23

What is feature engineering for qualitative data?

Answer 23

since qualitative/categorical data cannot be fed into ML models directly, they have to be turned into quantitative data first

Question 24

What are two methods to feature engineer qualitative data?

Answer 24

• label encoding
• one-hot encoding

Question 25

What is label encoding?

Answer 25

type of feature engineering (qualitative data)

ordinal (ranked) data –> discrete quantitative data

ranking/order of the classes is conserved in a discrete numerical schema and a “distance” can be defined

Question 26

What is one-hot encoding?

Answer 26

type of feature engineering (qualitative data)

nominal (unranked) data –> binary coding of labels

for each possible class in a feature, a binary feature is introduced, only those features that match have a value of one

–> multi-class features are also possible

Question 27

Can one-hot encoding be used with a huge number of classes?

Answer 27

if too many classes present, use label encoding instead

–> curse of dimensionality

Question 28

How can feature engineering be done to image data?

Answer 28

all pixels are considered in a greyscale way with
0 being a black pixel
maximum value being a white pixel

or 8-bit encoding: 0 is black, 256 is white

Question 29

What is special for color-images with feature engineering?

Answer 29

RGB: three channels (red green blue)

each channel is a grayscale image in itself

Question 30

What are methods for feature engineering of color-images? (5)

Answer 30

1) concatenate all channels (RGB) and feed stack into the model
(only works for eg CNNs because they can deal with 2D data)

2) linearize channels and concatenate vectors
(spatial info is somewhat lost, works for models that expect linear input data eg MLPs, K-NNs etc)

3) build a histogram for each channel
(spatial info is fully lost)

4) Visual bag-of-words
(use clusters as features and count their frequencies)

5) histogram of oriented gradients (HOG)
(for each cell, create histogram of gradients as feature)

Question 31

Which of the methods for feature engineering of image data works best?

Answer 31

depends on task and data set

Question 32

What is the final data set nomenclature?

Answer 32

Features/attributes (x)

f(x) = y

Targets/Labels, (y) - Ground-Truth

Question 33

What type of data are weight and height?

Answer 33

continuous

Question 34

What type of data is “wings”, true or false?

Answer 34

binary

Question 35

What type of data is number of legs?

Answer 35

discrete

Question 36

What type of data is “cuteness”?

Answer 36

ordinal

Question 37

What type of data is “type = bird/cat/dog etc”?

Answer 37

categorical (multi-class)

Question 38

What are “bird/cat/dog” labels for the ground-truth?

Answer 38

Classes of label “type”

Question 39

What is data scaling?

Answer 39

to linearly transform your data in order to normalize them

Question 40

Why do we need to scale data?

Answer 40

1) many ML models are based on a notion of “distance” between samples;
improperly scaled data may jeopardize the learning capability of such models

2) some ML models intrinsically presume that data are distributed following a Gaussian fashion with similar variances along all features;
high variance along one feature leads to bias

Question 41

How to we scale data? (2)

Answer 41

• normalize feature variance
  (to give similar weights to the different features)
• normalize feature mean values
  (assumed by a number of ML models)

Question 42

What is the MinMax scaler?

Answer 42

scale every feature onto a range from 0 to 1 based on the minimum and maximum of the underlying distribution

xi’ = (xi - min(Xi)) / (max (Xi) - min (Xi))

Question 43

What is a disadvantage of the MinMax scaler?

Answer 43

is prone to outliers and
does not center the distribution in the origin

Question 44

What are different scalers?

Answer 44

• MinMax Scaler
• standard Scaler
• robust scaler

Question 45

What is the Standard scaler?

Answer 45

scale every feature onto a range from -1 to 1 based on the mean and standard deviation of the underlying distribution

xi’ = (xi - mean(Xi)) / standarddeviation(Xi)

this scaling is centered onto the origin but still prone to outliers to some extent