1 - Introduction

Question 1

What is the growth rate of job openings in data science from 2012 to 2017?

Answer 1

6.5 times as many job openings in 2017 as compared to 2012

This indicates a significant increase in demand for data science professionals.

Question 2

What was IBM’s projection for the yearly demand for data scientists by 2020?

Answer 2

Nearly 700,000 openings

This projection reflects the increasing need for data professionals in various industries.

Question 3

What is the primary reason for the high demand for data scientists in America?

Answer 3

There is a shortage of talent

This shortage has made data science one of the top jobs in the U.S.

Question 4

Define data science.

Answer 4

The systematic analysis of data within a scientific framework

This involves an adaptive, iterative, and phased approach to data analysis.

Question 5

What are the three main components that data science combines?

Answer 5

• Data-driven approach of statistical data analysis
• Computational power and programming acumen of computer science
• Domain-specific business intelligence

These components work together to uncover actionable insights from data.

Question 6

What is the Data Science Methodology (DSM)?

Answer 6

A framework that helps the analyst keep track of the analysis phases

DSM is adaptive and iterative, allowing for revisiting previous phases as needed.

Question 7

List the seven phases of the Data Science Methodology.

Answer 7

• Problem Understanding Phase
• Data Preparation Phase
• Exploratory Data Analysis Phase
• Setup Phase
• Modeling Phase
• Evaluation Phase
• Deployment Phase

Each phase plays a crucial role in the data science process.

Question 8

What is the focus of the Problem Understanding Phase?

Answer 8

Clearly enunciate project objectives and formulate a solvable problem

This phase aims to align teams on the problem to be addressed.

Question 9

What is the most labor-intensive phase of the data science process?

Answer 9

Data Preparation Phase

This phase involves cleaning and preparing raw data for analysis.

Question 10

What tasks are performed during the Exploratory Data Analysis Phase?

Answer 10

• Exploring univariate relationships
• Exploring multivariate relationships
• Binning based on predictive value
• Deriving new variables

Simple exploratory methods are used to gain preliminary insights.

Question 11

What is the purpose of the Setup Phase?

Answer 11

To prepare for modeling by performing necessary tasks like cross-validation and baseline performance

This ensures data is ready for effective modeling.

Question 12

What does the Modeling Phase involve?

Answer 12

• Selecting and implementing modeling algorithms
• Ensuring models outperform baseline models
• Fine-tuning model algorithms

This phase is crucial for uncovering profitable relationships in the data.

Question 13

What is evaluated during the Evaluation Phase?

Answer 13

• Model performance against baseline measures
• Whether models solve the original problem
• Application of error costs intrinsic to the data

This phase determines the effectiveness of the models.

Question 14

What is the final phase of the Data Science Methodology?

Answer 14

Deployment Phase

This phase involves reporting results and adapting models for real-world use.

Question 15

What are the most common data science tasks?

Answer 15

• Description
• Estimation
• Classification
• Clustering
• Prediction
• Association

Each task serves a specific purpose in data analysis.

Question 16

Define the Description task in data science.

Answer 16

Describing patterns and trends within the data

This task is often used by both specialists and nonspecialists.

Question 17

What is Estimation in data science?

Answer 17

Approximating the value of a numeric target variable using predictor variables

Estimation models learn from known target values to predict unknowns.

Question 18

What distinguishes Classification from Estimation?

Answer 18

Classification deals with categorical target variables, while estimation deals with numeric target variables

This makes classification a crucial task for many applications.

Question 19

What is the goal of the Clustering task?

Answer 19

Identifying groups of records that are similar

Clusters can provide insights and serve as inputs for further analysis.

Question 20

What does the Prediction task involve?

Answer 20

Forecasting future outcomes based on current data

Prediction can relate to both numeric and categorical variables.

Question 21

What is the Association task in data science?

Answer 21

Determining which attributes are associated with each other

This helps in understanding relationships between different variables.

Question 22

What is data science?

Answer 22

Data science is an interdisciplinary field that uses scientific methods, processes, algorithms and systems to extract knowledge and insights from structured and unstructured data.

Question 23

Which areas of study does data science combine?

Answer 23

Data science combines statistics, computer science, and domain knowledge.

Question 24

What is the goal of data science?

Answer 24

The goal of data science is to extract meaningful insights and knowledge from data.

Question 25

Name the seven phases of the DSM.

Answer 25

* Problem Understanding Phase * Data Preparation Phase * Data Exploration Phase * Model Building Phase * Evaluation Phase * Deployment Phase * Monitoring Phase

Question 26

Why is it a good idea to have a Problem Understanding Phase?

Answer 26

The Problem Understanding Phase helps to clarify the objectives and scope of the data science project.

Question 27

Why do we need a Data Preparation Phase? Name three issues that are handled in this phase.

Answer 27

The Data Preparation Phase is needed to clean and format the data for analysis. Issues handled include: * Handling missing values * Removing duplicates * Data normalization

Question 28

In which phase does the data analyst begin to explore the data to learn some simple information?

Answer 28

Data Exploration Phase

Question 29

Explain in your own words why we need to establish baseline performance for our models. Which phase does this occur in?

Answer 29

Establishing baseline performance helps to measure the effectiveness of the model. This occurs in the Model Building Phase.

Question 30

Which phase represents the heart of your data scientific investigation? Why might we apply more than one algorithm to solve a problem?

Answer 30

Model Building Phase; more than one algorithm may be applied to find the best solution or to improve accuracy.

Question 31

How do we determine whether our predictions are any good? During which phase does this occur?

Answer 31

We determine the quality of predictions during the Evaluation Phase.

Question 32

True or false: The data scientist’s work is done with the Evaluation Phase. Explain.

Answer 32

False; the data scientist's work continues into the Deployment and Monitoring Phases.

Question 33

Explain how the DSM is adaptive.

Answer 33

The DSM is adaptive as it can adjust to new information and changes in project requirements.

Question 34

Describe how the DSM is iterative.

Answer 34

The DSM is iterative because it allows for revisiting and refining previous phases based on findings.

Question 35

List the most common data science tasks.

Answer 35

* Estimation * Prediction * Classification * Clustering * Association

Question 36

Which of these tasks have many nonspecialists been doing all along?

Answer 36

Estimation and Prediction

Question 37

What is estimation? In estimation, what must be true of the target variable?

Answer 37

Estimation is predicting a numeric variable; the target variable must be continuous.

Question 38

What is the most widespread task in data science? For this task, what must be true of the target variable?

Answer 38

Prediction; the target variable must be categorical.

Question 39

What are cluster profiles?

Answer 39

Cluster profiles are descriptions of the characteristics of each cluster formed during clustering.

Question 40

True or false: Prediction can only be used for categorical target variables. Explain.

Answer 40

False; prediction can also be used for continuous target variables.

Question 41

For an association rule, what do we mean by support?

Answer 41

Support refers to the proportion of records the rule applies to.