Introduction

Question 1

What is Data Minig

Answer 1

• Large quantities of data
• data contains interesting patterns

Data Mining helps to

• discover patterns in data
• use the patterns for decision making

Question 2

Large data sources set the foundation for data mining

Answer 2

• Law enforcement agencies -> terrorist detection
• Facebook -> interest and behavior of its users
• Sloan Digital Sky Survey -> predict type of sky object

Question 3

What is the problem with the large amount of data available

Answer 3

• Many data is collected
• Only a small amount can be looked at by humans
• We are interested in the patterns, and not the data itself

Question 4

Definitions of Data Mining

Answer 4

Exploration & analysis of large quantities of data in order to discover meaningful patterns

Question 5

Data Mining methods

Answer 5

1. Detect interesting patterns
2. Support human decision making with patterns
3. Predict the outcome of a future observation based on patterns

Question 6

Origins of data mining - relation of data mining to other areas

Answer 6

Combination of those areas:

• Statistics
• Machine Learning AI
• Database Systems

Question 7

Origins of data mining - motivating challanges

Answer 7

• Large amount of data
• high dimensionality of data
• heterogenous and complex data
• explorative analysis > hypothesize-and-test paradigm

Question 8

What are the two data mining tasks and what is the ML terminology

Answer 8

Descriptive Tasks (Unsupervised)
Goal: Find patterns in the data

Predictive Tasks (Supervised)
Goal: Predict unknown values of a variable

Question 9

Data Mining Tasks

Answer 9

• Cluster Analysis (Descriptive)
• Classification (Predictive)
• Regression (Predictive)
• Association Analysis (Descriptive)

Question 10

The most used methods in practice

Answer 10

• Regression
• Decision Trees
• Clustering
• Random Forests

Question 11

The steps of the data mining process

Answer 11

1) Data selection
2) Data preprocessing
3) Data transformation
4) Data mining
5) Interpretation / Evaluation of patterns

Question 12

Questions that come up for data selection

Answer 12

• What data is useful for the task?
• What data is available?
• How is the data quality?

Question 13

What does exploration / profiling mean

Answer 13

• Develop initial understanding of the data
• Calculate basic summarization statistics
• Visualize the data
• Identify data problems (outliers, missing values, duplicate records)

Question 14

What does preprocessing and transformation mean

Answer 14

Transform data into a representation that is suitable for the chosen data mining method

Data integration and preperation takes 70 - 80 % of the time for a data mining project

Question 15

Important transformation aspects

Answer 15

• scale attributes (nominal, ordinal, numeric)
• number of dimensions (represent relevant information with less attributes)
• amount of data (determines hardware requirements)

Question 16

Transformation methods

Answer 16

• Discretization and binarization
• feature subset selection / dimensionality reduction
• attribute transformation
• aggregation, sampling
• integrate data from multiple sources

Question 17

Concept of Data mining

Answer 17

Input: Preprocessed Data
Output: Model / Patterns

1. Apply data mining method
2. Evaluate resulting model
3. Iterate (change the parameter settings / use other methods, improve preprocessing, increase quality of training data)

Question 18

Description of the deployment step

Answer 18

• Use the model in the business context

- Keep iterating to improve the model

Question 19

How do data scientists spend their days?

Answer 19

• 60% cleaning data
• 19 % collecting data sets
• 9 % mining data for patterns

Question 20

Common data mining software

Answer 20

• Python
• RapidMiner
• scikit-learn
• SQL
• Anaconda

Question 21

Advantages of RapidMiner

Answer 21

• Visual modeling of data mining pipelines

- Faster learning curve for applying data mining methods

Question 22

Different attribute types

Answer 22

Categorial (qualitative)

• Nominal : Values of the attributes can only be distinguished from another ( equal or unequal)
• Ordinal : Values of the attribute can be ordered (>,