Week 5

Question 1

What does it mean that the miner is often the end user?

Answer 1

Data mining is carried out by knowledge persons within the different business units.

Question 1

What is the output of regression data mining (belongs to predictions)

Answer 1

A number

Question 2

What does link analysis try to achieve?

Answer 2

Find patterns in relationship to each other.

Question 3

What does the robustness of data mining refer to?

Answer 3

Its ability to overcome noisy data to make somewhat accurate predictions.

Question 4

What does the accuracy of data mining refer to?

Answer 4

Its ability to predict the outcome of a previously unknown data set accurately.

Question 5

In estimating the accuracy of data mining (or other) classification models, the true positive rate is

Answer 5

the ratio of correctly classified positives divided by the sum of correctly classified positives and incorrectly classified negatives.

Question 6

When would the iteration of steps 3 and 4 stop in K-means clustering?

2 awnsers

Answer 6

1. When the recalculation of center points does not lead to a reassignment of data points anymore.
2. When a pre-defined number of iterations have been carried out.

Question 7

Would the algorithm always show the same results if we keep K the same and all other parameters the same?

Answer 7

No, because the initial selection of cluster center points is random.

Question 8

What is the output variable in Association?

Answer 8

There is no output variable.

Question 9

What is the Euclidian distance

Answer 9

Ordinary distance between two points that one would measure with a ruler.

Question 10

What is the manhattan distance?

Answer 10

rectilinear distance, or taxicab distance, between two points)

Its the total travel distance if one can only move along grid lines.

Question 11

What can cluster analysis be used for?

Answer 11

Cluster analysis can be used for automatic identification of natural groupings of things.

Question 12

What kind of learning does cluster analysis use?

Answer 12

Unsupervised learning

Question 13

What kind of data set does supervised learning use?

Answer 13

A labeled data set

Question 14

How does clustering work?

Answer 14

It works by learning the clusters of things form past data, then assigning new instances.

Question 15

What are some of the use cases of clustering?

Answer 15

• Identify natural groupings of customers;
• Identify rules for assigning new cases to classes for targeting/diagnostic purposes;
• Provide characterization, definition, labeling of populations,
• Decrease the size and complexity of problems for other data mining methods;
• Identify outliers in a specific domain.

Question 16

How is the optimal amount of clusters determined?

Answer 16

There is no optimal way to calculate the amount of clusters, hence heuristics are often used.

Question 17

What does K-means clustering mean?

Answer 17

K-means clustering means that there is a pre-determined number of clusters.

Question 18

What are the steps of K-means clustering?

Answer 18

1. Determine the value of k;
2. Randomly generate k random points as initial cluster centers;
3. Assign each point to the nearest cluster center;
4. Re-compute the new cluster centers;
5. Repeat steps 3 and 4 until some convergence criterion is met.

Question 19

When is the conversion criteria met?

Answer 19

The convergence criterion is usually met when the assignment of points to clusters becomes stable.

Question 20

What is association rule mining?

Answer 20

It finds interesting relationships (affinities) between variables (items or events).

Question 21

What kind of machine learning does association utilize?

Answer 21

Unsupervised.

Question 22

What is the input of association?

Answer 22

Point-of-sale transaction data

Question 23

What is the output of association?

Answer 23

Most frequent affinities among items.

Question 24

What are the use cases for association for business and medicine?

Answer 24

Business: cross-marketing, cross-selling, store design, catalog design, e-commerce, site design, optimization of online advertising, product pricing, and sales promotion/configuration.

Medicine: relationships between symptoms and illnesses, diagnosis and patient characteristics and treatments, and genes and their functions.

Question 25

What is support in association?

Answer 25

How often X and Y go together

Baskets with both X and Y divided by total baskets

Question 26

What is the confidence in association?

Answer 26

How often Y goes together with X

SuPP X–>Y divided by Supp X

Question 27

Why is text mining needed to stay competitive?

Answer 27

Because 85-90 percent of corporate data is unstructured.

Question 28

What is text mining?

Answer 28

A semi-automatic process of extracting knowledge from unstructured data sources.

Question 29

What is the difference between data mining and text mining?

Answer 29

Text mining uses unstructured data.

Question 30

What does text mining do?

Answer 30

Text mining imposes structure to the data and mines the structured data

Question 31

What is structured data?

Answer 31

Composed of clearly defined data types whose pattern makes them easily searchable.

Question 32

What is unstructured data?

Answer 32

Everything else from structured. There is some internal structure but are not structured via pre-defined models or schemas. Usually ont easily searchable.

Question 33

What is semi-structured data?

Answer 33

Data have some internal structure but also unstructured elements (emails, xml files)

Question 34

What is the corpus?

Answer 34

A large and structured set of texts prepared for the purpose of conducting knowledge discovery.

Question 35

What is tokenizing?

Answer 35

A token is categorized block of text in a sentence. The assignment of meaning to blocks of text is known as tokenizing.

Question 36

What are terms?

Answer 36

A term is a single word or multi word phrase extracted directly from the corpus of a specific domain by means of natural language processing (NLP) methods.

Question 37

What are concepts?

Answer 37

Concepts are features generated from a collection of documents by means of manual, statistical, rule-based, or hybrid categorization methodology. Compared to terms, concepts are the result of higher-level abtraction.

Question 38

What is stemming?

Answer 38

THe process of reducing inflected words to their stem (or base or root) form.

Question 39

What are stop words (and include words)

Answer 39

Words that are filtered out prior to or after processing of natural language data. Usually a list of verbs that are removed.

Question 40

What are synonyms?

Answer 40

Syntactically different words with identical or similar meanings.

Question 41

What are polysemes?

Answer 41

Syntactically identical words with different meanings.

Question 42

What is the term dictionary?

Answer 42

A collection of terms specific to a narrow field that can be used to restrict the extracted terms within a corpus.

Question 43

What is word frequency?

Answer 43

The number of times a word is found in a specific document.

Question 44

What is natural language processing (NLP)

Answer 44

A subfield of AI and computational linguistics that studies the understanding of the natural human language.

Question 45

What is bag-of-words?

Answer 45

Disregarding grammar or order of words?

Question 46

What are the applications of NLP?

Answer 46

• Language translation applications;
• Checking grammatical accuracy;
• Personal assistant applications.

Question 47

What are the steps in the text mining process?

Answer 47

1. Establish the corpus
2. Create the term-document matrix
3. Extract patterns/knowledge

Question 48

What is the term-by-document Matrix (TDM)?

Answer 48

This matrix describes how often terms are mentioned in each document. Decisions such as which words to include (stop words, synonyms/homonyms and stemming) and kind of indices to use.

The dimensionality of the TDM can be reduced manually by an export. The matrix is transformed using singular value decomposition (SVD).