Week 2

Question 1

What is “Churn Rate”

Answer 1

refers to the percentage of customers who stop doing business with a company over a specific period. It is a key metric, especially for subscription-based businesses, as it indicates how well the company retains its customers.

Question 2

How to caculate ‘Customer Churn’?

Answer 2

ChurnRate=
TotalCustomersattheBeginningofthePeriod/
NumberofCustomersLostinaPeriod
​
×100

Question 3

What is “Customer Tenure”?

Answer 3

refers to the length of time (in months) a customer has been with the company. It is a measure of customer loyalty and can indicate how long a customer has remained subscribed to the company’s services.

Tenure is often used alongside metrics like “Churn” to assess customer retention and loyalty trends.

Question 4

What are the common techniques used in handling missing data?

Answer 4

Identifying Missing Data
Removing Missing Data
Imputing Missing Data

Question 5

What is ‘Identifying Missing Data’

Answer 5

detecting where data is absent or incomplete in a dataset.

Question 6

What is ‘Removing Missing Data’

Answer 6

involves deleting rows or columns that contain missing values.

Question 7

What is ‘Imputing Missing Data’

Answer 7

involves filling in the missing values with substituted values without deleting rows or columns.

Question 8

What are the 2 Common Approaches to impute missing data?

Answer 8

Mean/Median/Mode Imputation and Forward/Backward Fill

Question 9

What is ‘Mean/Median/Mode’ imputation?

Answer 9

replaces missing values with a central tendency value to ensure that the dataset can still be used for analysis or modeling. While easy to implement, these methods can distort the original distribution of the dataset and may introduce bias.

Question 10

What is ‘Forward/Backward Fill’

Answer 10

involves filling missing values with the previous or next available values, respectively, based on the order of the data.

Question 11

What is ‘Visual Inspection’ in Identifying Missing Data?

Answer 11

uses graphical representations to identify data gaps, patterns, and abnormalities.

Question 12

What is ‘Data Summary Tables’ in Identifying Missing Data?

Answer 12

shows an overview of the missing values in the dataset.

Question 13

What is ‘Box Plots’ in Identifying Missing Data?

Answer 13

summarizes the distribution of a variable and shows outliers. Still, if the data is missing, there may be visible gaps or unusual behavior in the plot.

Question 14

What is ‘Heatmaps’ in Identifying Missing Data?

Answer 14

a missing data heatmap highlights where missing data exists in a dataset. It uses color to indicate missing vs. non-missing values across the entire dataset.

Question 15

What is ‘Descriptive Statistics’ in Identifying Missing Data?

Answer 15

summarizes and describes the basic characteristics of a dataset.

Question 16

What is ‘Null’ or ‘NaN’ Counts?

Answer 16

counts the number of missing values in each dataset’s column (or feature). In Python, missing data is either NaN (Not a Number) or null.

Question 17

What is ‘Percentage of Missing Data’

Answer 17

provides a proportion of missing data relative to the total number of entries in a column.

Question 18

What are the 3 ‘Patterns of Analysis of Missing Data’

Answer 18

Missing Completely at Random (MCAR), Missing At Random (MAR), and Missing Not At Random (MNAR)

Question 19

What are the 3 types of Missing Data?

Answer 19

Missing Completely at Random (MCAR), Missing At Random (MAR), and Missing Not At Random (MNAR)

Question 20

What is ‘Missing Completely at Random (MCAR)’

Answer 20

where the absence of data is unrelated to any other variables in the dataset. Removing this type of missing data generally won’t introduce bias.

This occurs when the reason for the absence of a value is entirely random and unrelated to any other variables in the datasets. For example, a survey respondent accidentally skips a question, resulting in a missing value in the dataset.

Question 21

What is ‘Missing At Random (MAR)’

Answer 21

when the missingness is related to observed data, meaning that it is not random but can be explained by other variables.

the absence of data isn’t random and can be explained by other observed variables in the dataset. For example, in a health survey, individuals working night shifts may be less likely to respond to a survey conducted during daytime hours. The missingness of their responses is related to their work schedules, an observed variable, but not directly to their health status, which is the variable of interest.

Question 22

What is ‘Missing Not At Random (MNAR)’

Answer 22

when the missing data is related to the unobserved value, making removal potentially biased.

This occurs when the absence of data is directly related to the value itself, even after accounting for other variables. For example, In mental health research, individuals with more severe symptoms are less likely to complete assessments due to the nature of their condition. In this case, the missing data is directly related to the severity of the unobserved symptoms and not the assessment.

Question 23

What is ‘Low Percentage of Missing Data’ factor?

Answer 23

If only a small percentage of the dataset has missing values, dropping rows or columns may be an acceptable solution.

Example:
If less than 5% of the rows contain missing values, the loss of information may be minimal

Question 24

What is ‘Irrelevant Features or Low-Impact Variables’

Answer 24

if the feature or variable with missing data is not crucial for the analysis, it can be removed without significantly impacting the model’s performance.

Question 25

What is ‘Sparsely Populated Columns’

Answer 25

if a column has a significant number of missing values (e.g., >50%), it might be better to remove the column entirely, as imputing values would introduce too much noise or bias.

Question 26

What is ‘Removing Rows with Missing Data’

Answer 26

you can remove rows that contain missing data using dropna() with its default axis=0. This removes any row that contains at least one missing value.

Question 27

What is ‘Removing Columns with Missing Data’

Answer 27

to remove columns with missing data, you can use dropna() with the axis=1 argument. This will remove any column that contains at least one missing value.

Question 28

What is ‘Single Imputation Method’

Answer 28

In single imputation, each missing value in a dataset is replaced with a single estimated value. These methods are generally easier to implement than multiple imputation methods. However, they treat the imputed values as if they were true values, ignoring the uncertainty associated with the imputation process.

Question 29

What is ‘Regression Imputation’

Answer 29

utilizes a regression model built on observed data to predict the missing values based on relationships with other variables.

Question 30

What is ‘Hot Deck Imputation’

Answer 30

estimates missing values by randomly selecting similar values from “donor” records within the dataset. This method retains the original pattern of associations in the dataset but may introduce randomness due to the selection process.

Question 31

What ‘Multiple Imputation Method’

Answer 31

create several imputed datasets and analyze them together. These techniques consider the imputation process’s uncertainty and provide more accurate results than single imputation. However, these methods are generally computationally expensive and require larger sample sizes to provide accurate predictions.

Question 32

What is ‘Outlier Detection and Removal’

Answer 32

outliers are data points that differ significantly from the rest of the dataset. Outlier detection and removal ensures data quality.

Question 33

What are the 3 different types of Outliers?

Answer 33

Global or point outliers
Collective outliers
Contextual or conditional outliers

Question 34

Whar is ‘Global Outlier’

Answer 34

Global outliers are also called point outliers. Global outliers are taken as the simplest form of outliers. When data points deviate from all the rest of the data points in a given data set, it is known as the global outlier. In most cases, all the outlier detection procedures are targeted to determine the global outliers. The green data point is the global outlier.

Question 35

What is ‘Collective Outlier’

Answer 35

In a given set of data, when a group of data points deviates from the rest of the data set is called collective outliers. Here, the particular set of data objects may not be outliers, but when you consider the data objects as a whole, they may behave as outliers. To identify the types of different outliers, you need to go through background information about the relationship between the behavior of outliers shown by different data objects. For example, in an Intrusion Detection System, the DOS package from one system to another is taken as normal behavior. Therefore, if this happens with the various computer simultaneously, it is considered abnormal behavior, and as a whole, they are called collective outliers. The green data points as a whole represent the collective outlier.

Question 36

What is ‘Contextual Outlier’

Answer 36

As the name suggests, “Contextual” means this outlier introduced within a context. For example, in the speech recognition technique, the single background noise. Contextual outliers are also known as Conditional outliers. These types of outliers happen if a data object deviates from the other data points because of any specific condition in a given data set. As we know, there are two types of attributes of objects of data: contextual attributes and behavioral attributes. Contextual outlier analysis enables the users to examine outliers in different contexts and conditions, which can be useful in various applications. For example, A temperature reading of 45 degrees Celsius may behave as an outlier in a rainy season. Still, it will behave like a normal data point in the context of a summer season. In the given diagram, a green dot representing the low-temperature value in June is a contextual outlier since the same value in December is not an outlier.

Question 37

What is ‘Outlier Detection’

Answer 37

identifies data points that significantly differ from the rest of the dataset.

Question 38

What are the techniques used in Outlier Detection?

Answer 38

Statistical Outlier Detection and Visual Outlier Detection

Question 39

What is ‘Statistical Outlier Detection’

Answer 39

identifies outliers using statistical methods based on the distribution and characteristics of the dataset.

Question 40

What is ‘Visual Outlier Detection’

Answer 40

uses graphical methods to identify outliers in data.

Question 41

What is ‘Outlier Handling’

Answer 41

is the process and techniques used to identify, analyze, and manage outliers in datasets. When removing outliers, it is important to be cautious, as it can lead to loss of valuable info. If you need to remove outliers, you must investigate before taking action.

Question 42

What is ‘Noise’ in Data?

Answer 42

refers to random errors or variations in measured data that do not reflect the actual values of the underlying phenomena.

Question 43

What is ‘Data Smoothing’

Answer 43

eliminates or reduces the impact of random noise, outliers, and irregularities. Done by using an algorithm to removenoisefrom a data set. This allows important patterns to more clearly stand out.

Question 44

What is ‘Moving Average’

Answer 44

a method used in data preprocessing to smooth out fluctuations in data. It calculates the average of a specified number of data points (called the “window size”) and slides this window across the dataset. This helps reduce noise and reveals underlying trends.

Question 45

What is ‘Generating New Features’ in Feature Engineering means?

Answer 45

involves identifying aspects of the data that are not directly observable and making them explicit.

Question 46

What is ‘Aggregated Features’ in Feature Engineering

Answer 46

summarize data across multiple rows or over a group, often through operations like mean, sum, max, or count.

Question 47

What is ‘Interaction Features’ in Feature Engineering

Answer 47

Interaction features are new features created by combining two or more existing features in a dataset. These combinations capture the relationship or interaction between the original features, which may improve the performance of machine learning models.

Think of interaction features as a way to understand how two factors work together, rather than individually. For example, the effect of one feature on the target variable might depend on the value of another feature. By explicitly creating features that represent these relationships, we give models a chance to learn more complex patterns.

Question 48

What is ‘Temporal Feature’ in Feature Engineering?

Answer 48

derived from date or time-related data. They capture patterns based on the passage of time (e.g., age, seasonality).

Example:
You can calculate the age of the house from the Year Built column

Question 49

What is ‘Text-Based Features’ in Feature Engineering?

Answer 49

are numerical or categorical representations derived from textual data. Since machine learning models cannot directly understand raw text, we transform text into features that models can process.

Question 50

What is ‘Bag of Words (BoW)’ in Feature Engineering?

Answer 50

one of the simplest and most widely used methods for creating text-based features. It represents a piece of text (like a sentence or document) as a collection of words, disregarding grammar, word order, and context but keeping the frequency of words.

Question 51

What is ‘Domain Specific Features’ in Feature Engineering

Answer 51

are tailored to the problem domain, incorporating expert knowledge or industry-specific metrics.

Question 52

What is ‘One-Hot Encoding’ in Feature Engineering?

Answer 52

converts categorical values into binary columns.

Question 53

What is ‘Label Encoding’ in Feature Engineering?

Answer 53

assigns a unique integer to each category.