SDA: Statistical Modelling Flashcards
What is a Bi-variate Regression Model?
Used to quantify the linear relationship between two variables using a regression line/equation
“Numerical representation of the fixed relationship between an observes response variable and a number of explanatory variables together with a measure of uncertainty associated with the relationship”
What is the Regression Line?
A line showing what relationship should have been expected i.e. ‘line of best fit’
What is the Ordinary Least Squares approach to regression modelling?
Most frequent approach
Fits a linear trend line through a cloud of points
Minimises the deviations between observed values and predicted i.e. minimises number of residuals
What are residuals?
The disturbances, unsystematic variations response i.e. the distance from an individual point to the regression line
Above the line: Positive residuals
Below the line: Negative residuals
If residuals are too big to have occurred by chance then the chosen model is inadaquate
What does the coefficient of determination (r-squared) value show?
Squared PPMCC
Summarises residuals around the regression i.e. shows the % of the data that is closest to the line of best fit
Shows how well the regression line fits the data
Example:
PPMCC = 0.922
Therefore, r-squared = 0.850
Therefore, 85% of the variation in y can be explained by the linear relationship between x and y, 15% of the variation remains unexplained
What is the F-test?
Tests r-squared for significance directly
What are the 6 assumptions of OLS regression modelling?
- Residuals have a mean value of zero
- Error terms patternless and uncorrelated
- Error terms will have a constant and equal variance i.e. HOMOSCEDASTICITY
- No correlation between error terms and x-variables
- Linearity - must be assumed that the relationship will be linear
- Residuals are normally distributed
What is homoscedasticity?
Error terms will have an equal and constant variance
What is a Confirmatory approach to regression?
Significance tests (t and f) inferring from sample to population
Guarding against sampling error
Presumes well developed model
Necessitates demanding assumptions
What is an Exploratory approach to regression?
Graphs of residuals
Exemplifies problems with data and models
Trying to develop initial models into an improved/better model
What are Catch-All plots?
Formed by plotting standardised residuals against fitted values (thus it is a separate graph in itself)
Informal way of magnify defects and model ills
Can be used in multiple regression
What is Multiple-Regression modelling?
Models that include more than one independent (x) variable
As well as the assumptions for a bi-variate regression model, what is the extra assumption for a multi-variate regression model?
No MULTICOLLINEARITY i.e. there should be NO correlation between any independent (x) variables
When are t-test results usually insignificant?
When there is heteroscedasticity and multicollinearity
What is Heteroscedasticity?
Where the variability of a variable is unequal across the range of values across a second variable that predicts it
e.g. annual income is heteroscedastic when predicted by age because most teens have very similar income levels, whilst older generations have much more variations. Therefore, across age ranges the variance in annual income changes.
