lec 5(done)

Question 1

Measures for data quality:

Answer 1

• Accuracy
• Completeness
• Consistency
• Timeliness
• Believability
• Interpretability

Question 2

Major Tasks in Data Preprocessing:

Answer 2

• Data cleaning
• Data integration
• Data transformation
• Data reduction

Question 3

Data cleaning include:

Answer 3

1-Fill in missing values
2-Smothing noise
3-identify or remove outliers
4-resolve inconsistencies

Question 4

Data integration is:

Answer 4

Integration of multiple
1-databases
2- data cubes
3- files

Question 5

Data reduction include:

Answer 5

1-Dimensionality reduction

2-Numerosity reduction

Question 6

Data transformation include:

Answer 6

1-Normalization
2-Data discretization
3-Concept hierarchy generation

Question 7

Real-world data tend to be :

Answer 7

1-Incomplete (missing): missing attribute values
2-Inaccurate (noisy): containing errors, or outliers
3-Inconsistent: containing discrepancies

Question 8

Reasons for missing data

Answer 8

1-Information is not collected
2-Attributes may not be applicable to all cases
3-Inconsistent with other recorded data and thus deleted
4-Human/Hardware/Software problems

Question 9

How to Handle Missing Data?

Answer 9

1-Ignore the tuple
2-Fill in the missing value manually: tedious + infeasible
3-Fill in the missing value automatically with:
o-A global constant : e.g., “unknown” or ∞
o-The attribute mean or median
o-The attribute mean for all samples belonging to the same class
o-The most probable value

Question 10

Reasons for noisy data:

Answer 10

1-Faulty data collection instruments
2-Human errors at data entry
3-Data transmission problems
4-Technology limitation
5-Inconsistency in naming convention

Question 11

Data Smoothing techniques

Answer 11

1-Binning:
   o-Smooth by bin means
   o-Smooth by bin medians
   o-Smooth by bin boundaries
2-Regression:
    o-Smooth by fitting the data into regression functions
3-Clustering
    o-Detect and remove outliers

Question 12

Binning Methods steps:

Answer 12

1-Sort the data values
2-Partition data into
o-equal depth bins (equal-frequency) or
o-equal width bins, where interval range of values in
each bin is constant.

3-Smooth the data:
o-Smooth by bin means: replace each bin value by
the bin mean.
o-Smooth by bin medians: replace each bin value by
the bin median.
o-Smooth by bin boundaries: replace each bin value
with the closest boundary value (min, max).

Question 13

how to apply regression

Answer 13

1-Linear Regression: finding the best line to fit two variables, so that one variable can be used to predict the other.

2-Multiple Linear Regression: more than two attributes are involved and the data are fit into a multidimensional surface.