Data preparation

Question 1

How to clean noisy entries

Answer 1

• for Kernel smoother where you have a fixed size window and you perform the average accordingly
• kernel average leads to smoother result but loose alot of data

then there is binning where the data is sorted and partitioned into bins then smoothed according to mean or median
Smooth the data by regression function

Question 2

what are the main components of data preparation

Answer 2

● Feature extraction: derive meaningful features from data.
● Portability: cast data to unify the structure, and to fit the Algorithm.
● Data cleaning.
● Data Reduction.
● Feature Selection.
● Transformation

Question 3

explain how the image feature extraction is done

Answer 3

It is based on building a histogram according to red, blue and green distribution

Question 4

explain how the document feature extraction is done

Answer 4

● extracting from the document named entities
● stop word removal
● word count histogram

Question 5

explain type portability

Answer 5

casting one format to another in order to fit the learning model , thus include :
● Text to numeric
● Time series to discrete sequence
● Time series to numeric
● Discrete sequence to numeric
● Spatial to numeric

Question 6

why data cleaning is important

Answer 6

Since in real word data might be noisy, incomplete or inconsistent e.g., Age=“42” Birthday=“03/07/1997”

Question 7

what is central tendency

Answer 7

consist of summarizing the data using one number, this include mean, median and mode

Question 8

explain the relation between distribution skewing and mean median

Answer 8

mean= median = mode equally distributed
mode < median< median left skewed
mean < median< mode right skewed

Question 9

how to clean inconsistent and missing records

Answer 9

delete the entire reccord, not safe and leads to lose info
estimate the missing value, lead to bias
global constant , mean for instance

Question 10

why using the distance measure

Answer 10

distance reflects similarity

Question 11

what is scaling

Answer 11

scaling changes the range of the data, generally we use it when want to measure how apart is the data

Question 12

what is data normalization

Answer 12

it is a radical change of the data such that we want to obtain a data that is normally distributed