Definitions

Question 1

Data Science is the art of turning data into actions.

Answer 1

Combines: Domain Expertise, Statistics, and Computing Skills
Flows back and forth between deductive and inductive reasoning
Relatively new discipline in which methodologies and frameworks are still being solidified

Question 2

Inter-related concepts

of Data Science

Answer 2

Analytics, Business Analytics, Data Science, Business Intelligence, Data Analytics, Big Data, Statistical Learning.

Question 3

Deductive Reasoning

Answer 3

Theory Driven, Hypothesis —-> To Analytics.

Question 4

Inductive Reasoning

Answer 4

Empirically Drive, Analytics —–> Hypothesis

Question 5

Big Data:

Answer 5

Data in which the volume, variety, or velocity of information prohibits analysis via conventional desktop or server scale tools.

Question 6

Distributed Processing (or computing):

Answer 6

A solution to the big data problem. Platforms which allow the power of individual machines to be simultaneously utilized to solve big data problems (e.g. Hadoop)

Question 7

Machine Learning:

Answer 7

Most closely associated with Inductive reasoning. Algorithms that allow computers to learn from data without explicit instructions from the operator.

Question 8

Supervised Learning:

Answer 8

Machine learning in which the outcome is defined by the operator. Can think of predicting outcomes.

Question 9

Unsupervised Learning:

Answer 9

Machine learning in which the outcome is not defined. Can think of classifying observations or dimensions.

Question 10

Regression:

Answer 10

A class of problems in which the objective is to predict the value of an outcome.

Question 11

Classification:

Answer 11

A class of problems in which the objective is to predict which group or “class” of an observation is likely to belong to.

Question 12

Parametric Techniques:

Answer 12

Techniques in which there are specific assumptions about the nature and/or shape of relationships between variables. E.g. in linear regression the slope of a line is being fit.

Question 13

Non-parametric Techniques:

Answer 13

Techniques in which there are not specific assumptions about the nature and/or shape of the relationships between variables. E.g. decision trees.

Question 14

Un-Structured Data:

Answer 14

Data that has no easily identified structure (e.g. free-form text responses)

Question 15

Types of Analytics

Answer 15

Descriptive Analytics: What is or has been?

Predictive Analytics: What is likely to happen?

Prescriptive Analytics: What should you do?

Question 16

Good Analytics

Answer 16

Creates Action: What will be different?

Understands context: What are the physics of the problem?

Avoids Bias: In the model and in the setup

Focuses on Impact: What value is generated?

Question 17

Data Science is a response and solution to the data deluge

Answer 17

Tools and process to deal with “Big Data”

Creates advantage to companies that use it effectively

Question 18

Data Science can handle a breadth of problems

Answer 18

Different domains
Different outcomes
Different purposes (Descriptive, Predictive, Prescriptive)

Question 19

CRISP-DM Definition

Answer 19

Cross-Industry Standard Process for Data Mining

Question 20

CRISP-DM

Components

Answer 20

Business Understanding
Data Understanding
Data Preparation
Modeling
Evaluation
Deployment

Question 21

Crisp-DM

Business Understanding

Answer 21

Determine Business Objectives:
Business background
Objectives and Success Criteria

Assess the situation:
Resource Inventory (e.g. budget, people)
Requirements, Assumptions, Constraints, Risks, Contingencies
Cost/Benefit

Determine Data Mining Goals and Success Criteria

Produce a Project Plan

Question 22

Crisp-DM

Data Understanding

Answer 22

Collect Initial Data

Describe the Data

Explore the Data

Verify Data Quality

Question 23

Crisp-DM

Data Preparation

Answer 23

Select Data

Cleaning Data:
Missing, Invalids

Construct New Data:
Transformations, Structure Data

Integrate Data

Format Data

Question 24

Crisp-DM

Modeling

Answer 24

Select Modeling Technique

Generate a Test Design

Build the Model

Assess the Model

Revise the Model

Question 25

Crisp-DM

Evaluate the Results

Answer 25

Evaluate the Results Relative to Objectives

Review

Determine Next Steps

Question 26

Crisp-DM

Plan for Deployment

Answer 26

Deployment Plan

Monitoring and Maintenance Plan

Final Report

Final Review

Question 27

Common Pitfalls in Data Science Projects

Answer 27

Assume model build and evaluation are a linear process:
In reality, they are very iterative
Agile methodologies valuable here
Requires tight integration between data scientist and domain knowledge

Do not allocate enough time for data gathering, clean-up, and understanding
Often the longest poll in the tent
Often iterative as analysis leads to more questions requiring more data
Build solutions that are not compatible with infrastructure and implementation
Complexity of model overwhelms the ability to implement
Speed of execution not compatible with use case

Do not match monitoring and maintenance to the velocity of the problem
World is not static, just because a model works today does not mean it will work tomorrow
Need to update the model in a way that is consistent with the business problem.

Poorly defined business problem
Contextual differences between build and application
Analytics do not answer the core question

Human bias
Confirmation
Uncertainty

The CRISP-DM methodology is an attempt to define the common analytical process that occurs regardless of industry context

The CRISP-DM methodology lays out specific steps involved in analytics project

Even with the CRISP-DM methodology, there are a series of common pitfalls to watch out for