Data Mining

Question 1

What is Data Mining

Answer 1

The process of extracting information from large databases and using it to make decisions

Question 2

Name two methods of Data Mining

Answer 2

Predictive and Description

Question 3

What is the prediction method

Answer 3

use some variables to predict unknown or future values of other variables

Question 4

what is the description method

Answer 4

find human-interpretable patterns that describe the da

Question 5

What are the 4 basic tasks of data mining

Answer 5

Classification, Regression, Clustering and Association rule discovery

Question 6

What is Classification

Answer 6

maps data into predefined groups or classes

Question 7

What is Regression

Answer 7

maps a data item to a real valued prediction variable

Question 8

What is Clustering

Answer 8

maps data into groups or classes which are defined by the data

Question 9

What is Association rule discovery

Answer 9

uncover relationships among data

Question 10

What are the 5 stages of the Data Mining Process

Answer 10

Data Gathering,
Data Preparation and Cleansing
Pattern Extraction and Discovery
Visualisation of the data
Analysis and Evaluation of Results

Question 11

Name two types of learning

Answer 11

deductive and inductive

Question 12

what is deductive learning

Answer 12

uses existing knowledge to deduce new knowledge. It is from general rules to special cases

Question 13

what is inductive learning

Answer 13

uses many examples to produce a generalisation of the examples that were given

Question 14

Name the 3 types of inductive learning

Answer 14

Supervised, Unsupervised and Reinforcement

Question 15

What is Supervised learning?

Answer 15

Training examples are input-output pairs with informative output

Classification learning is sometimes called supervised, because, in a sense, the
scheme operates under supervision by being provided with the actual outcome for
each of the training examples—the play or don’t play judgment, the lens recommendation, the type of iris, the acceptability of the labor contract.

Question 16

What is Unsupervised learning?

Answer 16

Training examples are input patterns with no associated output patterns

Question 17

What is Reinforcement learning?

Answer 17

Training examples are input-output pairs with evaluative output only

Question 18

Name two types of data values

Answer 18

nominal and real

Question 19

What is included in data preparation

Answer 19

data selection, data transformation

Question 20

What is included in data cleansing

Answer 20

Check if/for:
free from errors
missing data
outliers
duplicates

Question 21

Name two examples used for finding patterns in data

Answer 21

Classification and Association rules discovery

Question 22

Give an example of were classification is used

Answer 22

Fraud Detection

predict fraudulent cases in credit card transactions

Question 23

Describe the Classification steps

Answer 23

Given a collection of records (training set) - each record contains a set of attributes, one of the attributes is the class.

Find a model for class attribute as a function of the values of other attributes.

Goal: previously unseen records should be assigned a class as accurately as possible.

Question 24

Describe the Association rules discovery steps

Answer 24

Given a set of records each of which contain some number of items from a given collection;

Produce dependency rules which will predict occurrence of an item based on occurrences of other items.

Goal: previously unseen dependencies in a collection should be identified properly.

Question 25

Give an example of where Association is used

Answer 25

Supermarket shelf management

Goal: Identify items that are bought together by sufficiently many customers.

Method: process all the transaction data collected with barcode scanners to find dependencies among items.

Question 26

What is the minimum coverage

Answer 26

minimum number of instances

Question 27

What is the OneR(one attribute rule) algorithm

Answer 27

find one attribute to use that makes fewest prediction errors

generates rules that only include one attribute plus the class

Question 28

Describe the OneR algorithm

Answer 28

For each attribute A:
For each value V of that attribute, create a rule:
1. count how often each class appears
2. find the most frequent class, c
3. make a rule “if A=V then C=c”
Calculate the error rate of this rule. Pick the attribute whose rules produce the lowest error rate

Question 29

What is the Apriori algorithm

Answer 29

finds frequent itemsets using candidate generation. Finds associations between data items

generates association rules that involve several attributes and does not focus on any particular attribute

Question 30

Describe the Apriori algorithm

Answer 30

1. Set a minimum coverage
2. Find all one-attribute associations, which satisfy the minimum coverage;
3. Find all two-attribute associations, which satisfy the minimum coverage;
4. Until either reach a specified maximum number of attributes, Or can no longer generate associations that have the set minimum coverage
5. Set a minimum accuracy (confidence)
6. Generate rules from each association, which satisfy the minimum accuracy.

Question 31

What is the ID3 Algorithm?

Answer 31

used to generate a decision tree from a dataset. Several rules can be generated and it only accepts nominal values

Question 32

Describe the ID3 Algorithm

Answer 32

Classify examples by sorting them down the tree from the root node to some leaf notes

Learned function represented by tree

Each node in tree is tested on some attribute of an instance

Branches represent values of attributes

Follow tree from root to leaves for output value.

Question 33

For ID3, how do you determine which attribute best classifies data?

Answer 33

Entropy

Question 34

What is entropy?

Answer 34

Entropy is a measure of ‘degree of doubt’

The higher it is, the more doubt there is about the possible conclusions.

The attribute which has the lowest entropy is the most useful determiner.

Question 35

What are the advantages of ID3

Answer 35

It generates a detailed decision tree.

With training data provided, it is always able to generate a tree.
it is easily implemented

The output is easily to be understood and interpreted

The process is simple process

Its running time increases only linearly with the complexity of the problem

Question 36

What are the limitations of ID3

Answer 36

Wholly spurious correlations are possible, since the algorithm takes no account of any meaning that the data it works on may have.

The algorithm considers just one attribute at a time.

When inducing rules from large sets of examples in which there are a large number of possible outcomes, then the algorithm can be very sensitive to apparently trivial changes in the set of examples.

The algorithm cannot generate uncertain rules or handle uncertain data

Question 37

Give an example of an Association Rule

Answer 37

Transactions item set: {nappies, beer}

IF bought_nappies THEN bought_beer_likely

Question 38

Give an example of a classification rule

Answer 38

IF buy_time in December
and cost > 500
and type_of_item = electronics
and location = overseas
and ...etc...
THEN possibly_fraudulent = yes

Question 39

Name 3 examples of data mining applications

Answer 39

Retail/Marketing
ID buying patterns
associations among customer demographics

Banking
patterns of fraudulent use
id loyal customers
spending of customer groups

Insurance
Claims analysis

Medicine
successful therapies