Data Analysis

Question 1

Exploratory Data Analysis

Answer 1

the process of examining
a dataset to find insights and patterns.

Question 2

Data Cleaning

Answer 2

EDA helps in identifying errors, missing values, and outliers that need to be addressed before deeper analysis.

Question 3

Insight Generation

Answer 3

It provides a first look at the data, allowing analysts to develop insights and hypotheses for testing.

Question 4

Model Preparation

Answer 4

By understanding relationships and patterns, EDA informs feature selection and model building strategies.

Question 5

Assumption Validation

Answer 5

Many statistical models and machine learning algorithms require assumptions about data distribution or variable relationships. EDA validates these assumptions upfront.

Question 6

Communication

Answer 6

EDA techniques generate visualizations and summaries that communicate findings effectively to stakeholders.

Question 7

Common Data formats

Answer 7

CSV/TSV : tabular data, easy to work with
JSON : Nested dictionary format, ideal for hierarchical data.
XML/HTML : Nested data, used for web scraping.
Log data : unstructured text, parsed with Regular Expressions.

Question 8

Granularity

Answer 8

The level of detail represented by each record in a dataset.

Question 9

Scope

Answer 9

The coverage of the dataset in relation to what we are interested in analyzing.

Question 10

Temporality

Answer 10

Refers to the timing aspects of data, crucial for understanding when events occurred,

Question 11

Two different types of scope

Answer 11

Temporal Scope : dataset spans a concise period
Geographical scope: dataset geographically focused on certain hotspots

Question 12

Four aspects of temporality

Answer 12

Date and time fields - in the calls and stops datasets, datetime fields mark when police interactions were reported.
Timezone awareness - Important to consider timezone and daylight savings for accurate temporal analysis.
Data format - US datetime format (MM/DD/YYYY) used, essential for correct interpretation.
Placeholder dates - watch for default timestamps that may indicate missing values.

Question 13

Faithfulness

Answer 13

A dataset is considered faithful if it accurately reflects reality, crucial for reliable analysis.

Question 14

Common faithfulness issues

Answer 14

Unrealistic values - Future dates, nonexistent locations, negative counts, or significant outliers.
Dependency violations - mismatch between related fields, like age and birthdate.
Manual data entry errors - prone to spelling mistakes and inconsistencies
Data falsification indicators - repeated unusual names or email addresses, suggesting fabricated entries.

Question 15

Quantiative Data Plots

Answer 15

Histograms, box plots, scatter plots

Question 16

Qualitative data plots

Answer 16

Bar plots, dot plots, mosaic plots

Question 17

Mixed data type plots

Answer 17

Overlaid density curves, side-by-side box plots

Question 18

Rug plots

Answer 18

These plots show individual data points as marks along an axis, like threads on a rug. They’re helpful for a small number of observations but become cluttered with larger datasets

Question 19

Two quantitative features in a relationship

Answer 19

Use scatterplots to explore relationships, looking for linear or nonlinear patterns

Question 20

One qualitative and one quantitative variable in a relationship

Answer 20

Divide data into groups based on the qualitative feature and compare the quantitative distribution across these groups.
Density curves, box plots, and violin plots help compare distributions, highlighting differences in spread, central tendency and outliers

Question 21

Two qualitative features in a relationship

Answer 21

Compare the distribution of one feature across subgroups defined by another feature, focusing on proportions.

Question 22

Scatter matrix

Answer 22

Shows the correlations between each variable. They can also help reveal non linear correlations