Data collection and preparation

Question 1

ways to get data from online sources

Answer 1

1. web scrapping
2. api access
   2.1 Rest API (low-frequency information
   2.2 Streaming APIs

Question 2

Types and properties of Data

Answer 2

1. volume
   - data at rest (space)
2. velocity
   - data in motion (how fast)
3. variety
   - data in many forms
4. veracity
   - data in doubt (uncertainty)

Question 3

Data Structure: cross-sectional vs transactional vs panel data

Answer 3

cross-sectional: data that almost never changes
transactional: one observation represents one transaction
panel: one observation represents one individual during a time period (etc. monthly bill)

Question 4

Structured data vs Unstructured data

Answer 4

Structure data:
- qualitative/categorical data
- quantitative data
Unstructured data:
- text-based documents
- images/videos/sound

Question 5

measurement bias

Answer 5

1 recall bias: respondents recall some events more vividly than others
2 sensitive questions: respondent may not report data accurately
3 faulty equipment

Question 6

two groups

Answer 6

1. unsupervised methods (no specific target variable)
   - affinity grouping
   - similarity matching
   - clustering
   - sentiment analysis
2. supervised methods (specific target variable)
   - predictive modelling
   - causal modeling

Question 7

types of predictive modeling

Answer 7

1. regression
   predict the numerical value of some variable
2. classification
   predict which of a small set of classes an individual belongs to