Data Wrangling

Question 1

What is Data Wrangling?

Answer 1

It is the data exploration, transformation and validation process that involves preparing raw data for a clearly defined purpose, where raw data at this stage is data that has been collated through various data sources in a data repository.

Question 2

4 Step Process for Data Wrangling

Answer 2

• Discovery.
• Transformation.
• Validation.
• Publishing.

Question 3

Discovery Phase

Answer 3

Is about examining and understanding your data, base on your use case and creating a plan for cleaning, structuring, organizing, and mapping your data.

Question 4

Transformation Phase

Answer 4

Forms the whole of data wrangling process. It involves the tasks you undertake to transform the data, such as structuring, normalizing, denormalizing, cleaning, and enriching the data.

Question 5

(Part of Transformation Phase) Structuring Data

Answer 5

This task includes actions that change the form and schema of your data, which can be in Joins and Unions.

Question 6

Joins

Answer 6

Combine columns. When columns from the first source table are combined with columns from the second source table, coming up with 1 same column row.

Question 7

Unions

Answer 7

Combine rows. Rows of data from the first source table are combined with rows of data from the second source table, turning it into a single table.

Question 8

(Part of Transformation Phase) Normalizing Data

Answer 8

includes cleaning not used data, reducing redundancy, and reducing inconsistency.

Question 9

(Part of Transformation Phase) Denormalization Data

Answer 9

Is the process of combine data from multiple tables into a single table for faster querying of data for reports and analysis.

Question 10

(Part of Transformation Phase) Cleaning Data

Answer 10

are actions that fix irregularities in data in order to produce a credible and accurate analysis.

Question 11

are actions that fix irregularities in data in order to produce a credible and accurate analysis.

Answer 11

is the adding of data points to make your analysis more meaningful.

Question 12

Validation Phase

Answer 12

Validation rules refer to programming steps used to verify the consistency, quality, and security of the data we have after being structured, normalized, denormalized cleaned, and enriched.

Question 13

Publishing Phase

Answer 13

Is the transformed and validated version of the dataset along with the metadata about the dataset that would be delivered for downstream project needs.

Question 14

Some popular Data Wrangling software and tools:

Answer 14

• Excel Spreadsheets.
• OpenRefine.
• Google DataPrep.
• Watson Studio Refinery.
• Trifacta Wrangler.
• Python.
• R.

Question 15

Excel Spreadsheets

Answer 15

Microsoft Excel and Google Sheets have features and in-built formulas that can help identity issues, clean, and transform data. They allow you to import data from several different sources, cleaning, and transforming data as needed.

Question 16

OpenRefine

Answer 16

It is an Open-source tool that allows to clean, transform, import and export data in a variety of formats, such as TSV, CSV, XLS, XML, and JSON. It contains a menu-based operations (we don’t nee to memorize commands or syntax’s).

Question 17

Google DataPrep

Answer 17

Is an intelligent cloud data service that allows to visually explore, clean, and prepare structured and unstructured data for analysis. This system offers suggestion on ideal next steps and automatically detects schemas, data types, and anomalies.

Question 18

Watson Studio Refinery

Answer 18

This software its able to detect data types, it auto-enforces data governance policies, transforms large amounts of raw data into consumable for analysis, and cleanses.

Question 19

Trifacta Wrangler

Answer 19

Its a cloud-based service that cleans messy real-world data, arranges data into data tables, transform data, and allows multiple members to work simultaneously.

Question 20

Python

Answer 20

Contains a huge library and set of packages that offer powerful data manipulation capabilities, such as:
- Jupyter.
- NumPy.
- Pandas.

Question 21

R

Answer 21

Offers a series of libraries and packages that are explicitly created for wrangling messy data. Using these libraries you can investigate, manipulate, and analyze data. Some of the libraries are:
- Dplyr.
- Data.Table.
- Jsonlite.

Question 22

Cleaning workflow includes

Answer 22

• Inspection.
• Cleaning.
• Verification.

Question 23

(Cleaning workflow) Inspection

Answer 23

Its the process of detect different types of errors that the database may have, figure out structure and content of the data, visualize data, and use scrips to validate data.

Question 24

(Cleaning workflow) Cleaning

Answer 24

The techniques use for cleaning datasets will depend on the case and the type of issues encounter. Some of the common data issues are:
- Missing Values.
- Duplicate Data.
- Data type conversion.
- Irrelevant Data.
- Standardizing Data.
- Syntax Errors.
- Outliers.

Question 25

Data type conversion

Answer 25

needed to ensure that values in a field are stored as the data type of that field

Question 26

Standardizing Data

Answer 26

needed to ensure data-time formats are standard across the dataset.

Question 27

Syntax errors

Answer 27

Syntax errors

Question 28

Outtliers

Answer 28

are values that are vastly different from other observations in the data set, they need to be examined for accuracy in the dataset because they may be correct or incorrect.

Question 29

(Cleaning workflow) Verification

Answer 29

In this step we inspect results to establish effectiveness and accuracy achieved as a result of data cleaning. We need to re-inspect the data to make sure the rules and constraints applicable on the data still hold after the corrections you made.

Question 30

Documentation

Answer 30

Its the process of write down all changes and reasons behind making changes into a document for referral in the future with similar cases.