Lecture 5 (Data Mining)

Question 1

Why Data Mining?

Answer 1

More intense competition
Recognition of the value in data sources
Availability of quality data on customers, vendors, transactions

Question 2

Definition of Data Mining?

Answer 2

The nontrivial process of identifying valid, novel, potentially useful, and ultimately understandable patterns in data stored in structured databases.

Question 3

Data Mining Characteristics and Objectives?

Answer 3

Source of data for DM is often consolidated data warehouse
DM environment is usually a client-server or a Web-based information system architecture
Data is the most critical ingredient for DM which may include soft/unstructured data
The miner is the end user

Question 4

How data mining works?

Answer 4

DM extract patterns from data

Question 5

Types of patterns in data mining?

Answer 5

Association
Prediction
Cluster
Sequential

Question 6

Association methods?

Answer 6

Market-basket
Link analysis
Sequence analysis

Question 7

Prediction methods?

Answer 7

Classification
Regression
Time Series

Question 8

Segmentation methods?

Answer 8

Clustering

Outlier analysis

Question 9

Supervised Learning problems?

Answer 9

Classification

• The domain of the target is finite and categorical
• A classifier must assign a class to an unseen example

Regression

• The target attribute is formed by infinite values
• To fit a model to learn the output target attribute as a function of input attributes

Time Series Analysis
- Making predictions in time

Question 10

Unsupervised Learning Problems?

Answer 10

Clustering
Association Rules
Pattern Mining
- It is adopted as more general term than frequent pattern mining or association mining

Outlier Detection
- Ot is the process of finding data examples with behaviours that are very different from the expectation

Question 11

Data Mining Applications?

Answer 11

Customer Relationship Management
Banking and Other Financial
Retailing and Logistics
Manufacturing and Maintenance
Brokerage and Securities Trading
Insurance
Computer Hardware and Software
Science and Engineering
Government and Defense
Homeland security and law enforcement
Travel, entertainment, sports
Healthcare and medicine
Sports, virtually everywhere

Question 12

Customer Relationship Management?

Answer 12

Maximize return on marketing campaigns
Improve customer retention
Maximize customer value
Identify and treat most valued customers

Question 13

Banking and Other Financial?

Answer 13

Automate the loan application process
Detecting fraudulent transactions
Maximize customer value
Optimizing cash reserves with forecasting

Question 14

Retailing and Logistics?

Answer 14

Optimize inventory levels at different locations
Improve the store layout and sales promotions
Optimize logistics by predicting seasonal effects
Minimize losses due to limited shelf life

Question 15

Manufacturing and Maintenance?

Answer 15

Predict/prevent machinery failures
Identify anomalies in production systems to optimize the use manufacturing capacity
Discover novel patterns to improve product quality

Question 16

Brokerage and Securities Trading?

Answer 16

Predict changes on certain bond prices
Forecast the direction of stock fluctuations
Assess the effect of events of market movements
Identify and prevent fraudulent activities in trading

Question 17

Insurance?

Answer 17

Forecast claim costs for better business planning
Determine the optimal rate plans
Optimize marketing to specific customers
Identify and prevent fraudulent claim activities

Question 18

Data mining process?

Answer 18

A manifestation of best practices
A systematic way to conduct DM projects
Moving from Art to Science for DM project
Everybody has a different vision

Question 19

Most common standard processes of Data Mining?

Answer 19

CRISP-DM
SEMMA
KDD

Question 20

CRISP-DM?

Answer 20

Cross Industry Standard Process for Data Mining

Proposed in 1990s by European consortium

Question 21

Steps of CRISP-DM?

Answer 21

Business Understanding
Data Understanding
Data Preparation
Model Building
Testing and Evaluation
Deployment

Question 22

SEMMA?

Answer 22

Sample
Explore
Modify
Model
Assess

Question 23

KDD?

Answer 23

Knowledge Discovery in Databases

Question 24

Steps to KDD?

Answer 24

Data selection
Data cleaning
Data transformation
Data mining
Internalization

Question 25

Examples of Classification Task?

Answer 25

Predicting tumor cells as benign or malignant Classifying credit card transactions as legitimate or fraudulent Classifying secondary structures of protein as alpha-helix

Question 26

Classification Techniques?

Answer 26

``` Decision tree based methods Rule-based methods Neural Networks Naive Bayes and Bayesian Belief Networks Support Vector Machines ```

Question 27

Pros of KNN?

Answer 27

Simple Flexible Excellent performance on a wide range of tasks

Question 28

Cons of KNN?

Answer 28

Time consuming with n training points Memorization, not learning. No insight into the domain

Question 29

Assessment Methods for Classification?

Answer 29

Predictive accuracy - Hit rate ``` Speed - Model building versus predicting/usage speed Robustness Scalability Interpretability ```

Question 30

In classification problems, the primary source for accuracy estimation is the?

Answer 30

Confusion Matrix