Supervised - regression

Question 1

What is Regression analysis

Answer 1

is a type of predictive modeling technique
which is used to find the relationship between a dependent variable (usually known as the “Y” variable) and either one
independent variable (the “X” variable) or a series of
independent variables

Question 2

objective/ what is it for: Regression model

Answer 2

estimates the nature of the relationship
between the independent and dependent variables
❖ Strength of the relationship and its significance!

Question 3

Explanatory variables?

Answer 3

independent variables

Question 4

variables to be explained?

Answer 4

dependent variables

Question 5

Regression analysis can be used for

Answer 5

1. Predicting specific outcomes from changes, like estimating production needs for a product.
2. Projecting future trends or values, such as forecasting stock prices.
3. Assessing the influence of different factors on an outcome, like measuring the impact of advertising on sales during an event.

Question 6

The simplest model is

Answer 6

linear regression

Question 7

linear regression objective

Answer 7

Fit data with the best hyper-plane which “goes through” the points

Question 8

in the following regression mode: y = b0 + b1x +e:
1. The relationship between x and y is
2. what are the Two parameters to estimate
3. what is e

Answer 8

• a linear or straight-line relationship
• The slope of the line β1
  and the y-intercept
  β0( least squares)
• is the unexplained, random, or error component

Question 9

The estimates are determined by

Answer 9

Drawing a sample from the population
of interest
❖ Calculating sample statistics
❖ Producing a straight line that cuts into
the data

Question 10

The best line is

Answer 10

the one that minimizes the Sum of Squared
Differences (SSD) between the points and the line

Question 11

The regression line’s representation of the data is evaluated through several methods

Answer 11

Coefficient of Determination (R-squared): This metric quantifies how much of the variance in the dependent variable is explained by the independent variable. Higher R-squared values, closer to 1, indicate better explanatory power.

Residual Analysis: By examining the differences between actual and predicted values (residuals), we assess how well the model captures the data’s variability. Smaller residuals suggest a better fit.

Visual Inspection: Plotting the regression line on a scatter plot allows for a visual assessment of its fit. A good fit is indicated by the line passing through the center of data points, capturing the overall trend.

Significance of Regression Coefficients: Evaluating the significance of coefficients, particularly the slope, determines if the relationship between variables is statistically meaningful. If coefficients are significant, the model provides valid insights into the relationship between X and Y.

Question 12

what is Coefficient of Determination

Answer 12

measure of how well the
regression line represents the data
❖ The percentage of variability in Y that can be explained by
variability in X
❖ The further the line is away from the points, the less it can explain the
variability
❖ The Coefficient of Determination lies between 0 and 1

Question 13

Outliers, Investigate possibilities, Identify outliers from, residual, standard residual

Answer 13

Outliers: Unusually small or large observations
Investigate possibilities: recording error, sample membership, validity
Identify outliers from scatter diagram
Suspect outlier if |standard residual| > 2
Residual: Difference between actual value and estimated value
Standard residual: Residual divided by standard deviation of residuals

Question 14

One of the most challenging aspects of machine learning is

Answer 14

finding the
right set of features, or variables, that can accurately capture the
relationship between inputs and outputs

Question 15

define feature selection

Answer 15

is the process of selecting a subset of relevant features
from the original set of features to improve model performance
❖ In essence, it is about identifying the most informative features that can
help the model make accurate predictions

Question 16

popular techniques for feature selection

Answer 16

stepwise
regression

Question 17

what is stepwise regression

Answer 17

a method that iteratively adds or removes
features from a model based on their statistical significance

Question 18

when is stepwise stopped

Answer 18

repeated until a set of features that maximizes the model performance is
identified

Question 19

advantages of stepwise

Answer 19

when dealing with a large
number of features, as it can help to reduce the number of features in the
model without sacrificing accuracy

Question 20

limitation of stepwise

Answer 20

assumes that the relationship between
the features and the target variable is linear, which may not always be the
case in real-world scenarios

Question 21

Types of Stepwise Regression

Answer 21

• Forward selection: Starts with empty feature set, adds most statistically significant feature iteratively until model performance can’t be improved further.
• Backward elimination: Begins with full feature set, removes least statistically significant feature iteratively until model performance can’t be improved further.
• Bidirectional elimination: Combines forward and backward selection, alternates between adding and removing features until no further improvements in model performance can be made.

Question 22

Forward Selection steps

Answer 22

• Start with empty or intercept-only model.
• Conduct separate regressions for each predictor.
• Select predictor with strongest relationship to target.
• Forward Selection:
  
  • Add selected predictor to model.
  • Repeat with remaining predictors.
  • Stop based on predefined rule or validation metrics.