How to Mark and Remove Missing Data

Question 1

WHAT IS THE INDICATOR FOR MISSING VALUES WHEN USING “.DESCRIBE()”? WHAT COMMAND CAN WE USE TO FIX IT? P85,86

Answer 1

Missing values are frequently indicated by out-of-range entries, exp: Value of 0 for somethings that can’t have 0 as a value, indicates missing values.
We can use dataset[cols].replace(0,nan).

Question 2

CAN WE COUNT MISSING VALUES AS A CATEGORY IN DISCRETE (CATEGORICAL) FEATURES? P86

Answer 2

Yes

Question 3

WHAT IS STATISTICAL IMPUTATION? P92

Answer 3

Calculating a statistical value for each column (such as mean) and replacing all the missing values with that statistic.

Question 4

FOR IMPUTATION USING KNN, WHAT PARAMETERS SHOULD WE SELECT? P104

Answer 4

1-Distance measure
2-Number of contributing neighbors for each prediction (k parameter)

Question 5

WHAT IS “NA_VALUES” PARAMETER IN PD.READ_CSV? P105

Answer 5

We can choose the character which represents the missing values in the dataset and this will replace it with NaN.

Question 6

WHAT IS THE LIBRARY IN SCIKIT-LEARN THAT SUPPORTS KNN IMPUTATION? P106

Answer 6

KNNImputer

Question 7

HOW DOES ITERATIVE IMPUTATION WORK? P115

Answer 7

Iterative imputation is a process where each feature is modeled as a function of the other features, e.g. a regression problem where missing values are predicted. Each feature is imputed sequentially, one after the other, allowing prior imputed values to be used as part of a model in predicting subsequent features. It’s iterative because this process is repeated multiple times, allowing ever improved estimates of missing values to be calculated as missing values across all features are estimated.

Question 8

WHAT ARE THE OTHER NAMES FOR ITERATIVE IMPUTATION APPROACH? P115

Answer 8

Fully Conditional Specification (FCS)
Multivariate Imputation by Chained Equations (MICE)

Question 9

WHAT NUMBER OF ITERATION IS USUALLY ENOUGH FOR ITERATIVE IMPUTATION? P115

Answer 9

Low number, 10-20

Question 10

WHAT KIND OF ALGORITHMS ARE OFTEN USED FOR ITERATIVE IMPUTATION? P115

Answer 10

Linear models, for their simplicity.

Question 11

IN WHICH ORDERS CAN WE PROCESS THE FEATURES WITH MISSING VALUES? P115

Answer 11

The order that features are processed sequentially can be considered, such as from the feature with the least missing values to the feature with the most missing values.

Question 12

WHAT SHOULD WE IMPORT BEFORE USING ITERATIVEIMPUTER? P118

Answer 12

from sklearn.experimental import enable_iterative_imputer
from sklearn.impute import IterativeImputer

Question 13

WHAT ARE DIFFERENT IMPUTATION STRATEGIES IN ITERATIVE IMPUTATION? P121

Answer 13

Ascending, Descending, right-to-left(Arabic), left-to-right(Roman), random

Question 14

WHICH PARAMETER OF ITERATIVEIMPUTER IS FOR SETTING THE ITERATION NUMBER? P122

Answer 14

Max_iter