Session 2.1

Question 1

Data Structure (Volume/Velocity)

Answer 1

• Cross sectional
• Transactional
• Panel

Question 2

Cross sectional

Answer 2

Data that (almost) never changes. (e.g. city names and locations, customer’s birth date, etc.)

Question 3

Transactional

Answer 3

one observation represents one transaction (e.g. a website visit, or a purchase)

Question 4

Panel

Answer 4

one observation represents one individual during a time period (e.g., monthly bill, website visits per week)

Question 5

Data Structure: Tidy Data

Rules?

Answer 5

Put your data on a single table according to the following rules:

1 Each variable must have its own column.
2 Each observation must have its own row.
3 Each value must have its own cell.

Question 6

Structured Data (Variety)

• Qualitative/Categorical Data

Answer 6

Nominal categories have no natural order
-> e.g., race, gender, country

Ordinal there is a natural ordering of the categories
-> e.g., age bracket, satisfaction level

Question 7

Structured Data (Variety)

• Quantitative Data

Answer 7

Discrete countable number of distinct values
-> e.g., age, number of kids

Continuous any value within an interval
-> e.g., wage, temperature

Question 8

Unstructured Data (Variety)

Answer 8

Text-based documents (e.g., tweets, webpages, complaints, emails, etc.)
Images / Videos

Question 9

Unstructured Data (Variety)

Methods to transform unstructured data into structured data:

Answer 9

• Topic Modeling (text)
• Sentiment Analysis (text)
• Feature extraction (image / video / sound)

Question 10

Data Quality (Veracity)

Data quality can be affected in two major ways:

Answer 10

1. Missing Data

2. Measurement Error

Question 11

Missing Data (Veracity)

Answer 11

• Missing observations

- Missing values in some observations

Question 12

Reasons for missing data (Veracity)

Answer 12

1. Missing at random

If data are missing at random, the remaining observations are still a representative sample of the population

Simplest Solution: listwise deletion, i.e., delete all observations that do not have values for all variables in the analysis

2. Missing not at random

If data are missing not at random, then the remaining observations are not a representative sample of the population

No Simple Solution

Question 13

Selection Bias (Veracity) occurs…

Answer 13

when the sampling procedure is not random, and thus the sample is not representative of the population

Question 14

Selection Bias (Veracity)

Answer 14

1. Self-selection

some members of the population are more likely to be included in the sample because of their characteristics

e. g., participants in a voluntary insurance program
2. Attrition

some observations may be less likely to be present in the sample due to time constraints

e.g., tendency to look only at firms that survive

Question 15

Measurement Error (Veracity) occurs…

Answer 15

when the data collected contains errors that are non random

Question 16

Measurement Error (Veracity)

Answer 16

1. Recall bias

respondents recall some events more vividly than others

e. g., child deaths by guns vs swimming pools
2. Sensitive questions

respondents may not report data accurately

e. g., wages, health conditions
3. Faulty equipment

equipment that exhibits systematic measurement errors

e.g., a thermometer that is off by 1 degree Celsius