Multiple Linear Regression Flashcards
What is simple linear regression, and explain dependent variable, independent variable, and linear relationship.
simple linear regression is satisfied when we have only one dependent variable and one independent variable and there’s is a linear relationship between independent and dependent variable
Y = A0 + (A1 * X1)
Y = dependent variable
X1 = independent variable and has linear relationship with Y
What does linear relationship mean?
describes relationship between dependent variable and independent variable, when one variable changes the other variable changes proportionally and consistently following constant rate of change
What is the difference between simple linear regression and multiple linear regression?
Simple linear regression only has 1 independent variable and multiple linear regression has more than one independent variable
What are three requirements of multiple linear regression?
Y = a0 + a1x1 + a2x2 + a3*x3
Y = dependent variable
X1, X2, X3 = independent variable
A1, A2, A3 = partial regression coefficients have linear relationship with each independent variable
a0 = intercept where when plotting on a graph A0 crosses y
What is variation of y (aka sum of squares total) and formula?
difference between the observation/variable in a multiple linear regression and the mean of data and then squared.
SST = (Yi - Y mean)^2
Yi = value of dependent variable in multiple linear regression
Y mean = mean of data
What are the 5 assumptions about multiple linear regression that must hold to make them valid? LHINI
- Linearity
- Homoskedasticity
- Independent of observations
- Normality
- Independence of independent variables
What is linearity assumption and how is linearity measured/confirmed?
Linearity: the dependent variable must have a linear relationship with each independent variable
Eg. y = a0 + a1x1 + a2x2 + a3*x3 (y is dependent on x1, x2, and x3)
all independent variables must be raised to the power of 1 (x1 = x1^1, x2 = x2^1, x3 = x3^1) or else it isn’t a linear relationship
- pairwise scatterplot, plotting each independent variable against predicted values of dependent variable
What is the homoskedasticity assumption? What is the difference between homoskedasticity and Heteroskedasticity?
Homoskedasticity: data points or variance of data has a constant and consistent variance from mean of data
Heteroskedasticity: data points or variance of data has a inconsistent or unequal variance from mean of data
Eg. Graph residuals on vertical axis and plot dependent variables predicted values on horizontal axis.
- homoskedasticity would have flat trend line
- Heteroskedasticity would have sloped trend line and clustering of data points.
What is independence of observation or also known as independence of errors assumption?
Independence of observations: we don’t want our data points to be dependent of the previous data point we want them random.
Eg. We don’t want to plot our observations or residuals and have the observations/residuals be negative, positive, negative, positive, negative, positive; we want them random
What is normality assumption for multiple linear regressions?
Normality: distribution of the residuals (the difference between actual data point and the predicted values) should follow a normal distribution, meaning the errors should be centered around zero and have a bell-shaped curve when visualized on a histogram or normal probability plot. bell shaped since extreme errors should be less common both negatively and positively and most errors will be centered around 0
What is a residual in multiple linear regression?
Residual: difference/how far between the observation on a graph and the line of best fit are.
What is independence of independent variables assumption and how do you asses for independence of independent variables?
Independence of independent variables: the values of one independent variable do not influence or affect the values of another independent variable.
- use pairwise scatterplots comparing 2 independent variables with each other on a graph. if truly independent there would be no clear patterns and points should be randomly scattered.
How do you detect violations in linearity?
- using pairwise scatterplot
plot independent variables on horizontal axis and dependent variable on vertical axis. (Points should be clustered around the line of best fit)
What is a logistic regression?
- model used if dependent variable only has discrete outcomes (bankrupt, not bankrupt)
Can multiple linear regressions contain both discrete and and continuous independent variables?
yes a multiple linear regressions independent variables can be both discrete and continuous.
