Stats 2 - Stats & Linear Models Flashcards
What are the componenets of a Linear Model?
What are the types of variables you can have in a Linear Model?
How many Co-efficients do Continious terms and categorical factors have in a Linear Model?
Continuous terms always have one coefficient (β2)
Categorical Factors have N − 1 coefficients, where N is the number of levels in the category
Why N-1? Why are we missing a level?
The missing variable is incorporate into the baseline/refrence value known as the Intercept (β1) –> the level chosen as the refrence is done alphabetically
What makes a Linear model Linear?
Linear models are just a sum of terms that are linear in the coefficients –> Coefficients of each term are simple
Examples of types of Linear models?
The response variable is always Continuous
- Simple Regression –> Continuous Explanatory vairiables
- Multiple Linear Regression –> Continuous/Categorical explanatory variables
- ANOVA –> Categorical Explanatory Variable
- ANCOVA –> Categorical/Continuous explanatory variables
Note –> MLR and ANCOVA are very similar just place emphasis on one type of variable or the other
How can we decide what the best fit for our Linear Model is?
Solution Least Squares Fitting Solution
Given the data collected, we need to figure out a Linear model that best represents the data. This is done by…
- minimizing the sum of the distance (vertical distance between the data points and the line.
This is called the ‘Least squared solution’ which is where we minimize sum squared of ‘R’ –> we are squaring because we do not want negative values.
Plots below shows you the changes in resvar (Sum of squared R) when changing the coefficient –> The minimum corresponds to the solution.
How can we denote the Lienar model with least sqaures fitting solution?
Y with a hat
What are the assumptions of a Linear Model?
Outline the role of each of the diagnostic plots
Code
par(mfrow = c(2, 2), mar = c(5, 5, 1.5, 1.5))
plot(Model Name)
What are the two main tests performed in order to check whether your model realy explains the data? How do you know if we can rely on it?
F-Test –> Tests how much variation is explained
T-Test –> Tests the significane of the estimated Co-efficients
Outline the mean of the terms TSS, ESS and RSS?
TSS - Differences between the observed dependent variable (y) and the mean of y –> tells you the spread around the mean
ESS – differences between predicted y values and the mean of Y –> Looks at the spread around the mean when using the linear model
We expect and hope that it is smaller than TSS –> otherwise our model is shit
RSS –> sum of the squared differences between observed y and predicted y (model) –> basically the sum of the R2
RSS outlines how much variation our model cannot explain/take into account
What is the relationship between ESS, RSS and TSS?
TSS = ESS + RSS
How is the F-Statistic calculated?
Now that you have calculated the F-Statistic, how do you know whether it is significant?
How is the T-Statistic/Value Calculated?
Figure out the combinations of Effect size and precision for each plot.
Effect size –> is there a correlation between the two variables
Precision –> How close do the data points lie along the slope
Dotted lines represent confidence bounds visual representation of reliability –> bounds at the extremes are curve as extreme values can bias the slope more/more leverage
Hence, it is useful to sample at the extremes
Now that you have calculated the t-value for your co-efficients, how can you test whether they are statistically significant?
Generally speaking in statistics, what is a T-Test?
Why does using T-Tests not useful comparing more than 2 levels?
Formula to calculate the T-value?
How is the standard error calculated in a One-sample T-test?
One sampled
Breakdown the following One-Sample T-Test output.
One sample T-Test
Used to test whether the mean of a sample is different from a specific value
What is a two-sample T-Test? How does it compare to a one-sample?
Two-sample T Test is used to compare two means –> We are asking if they are statistically different from eachother
Main Difference
- One-sample T-Test we assume that one mean value is already known (Specific value) –> hence we only have one source of error in our single sample
- In the two-sample T-Test we are using two means estimated from two samples both of which will have error
How is the standard error calculated for a Two-sample T-Test?
Since we are dealing with two sources of error for our mean values we need to change our standard error calculation to reflect this.
Breakdown the following Two-sample T-Test Output
Generally speaking what is the F-Test?
What does var.test() on R do?
The var.test –> function can be used to do compare two variances –> E.g. Comparing the genome size variation for two species suborders
Is the ratio of variances = 1 –> meaning that they are they same or not?
If we want to perform a T-Test but our data is NOT normally distributed, what non-parametric test can we perform?
Wilcoxon Test
Difference between cor() and cor.test() on R?
- cor() –> calculates correlations between pairs of variables (multiple) –> For example you have a table and you want to find the correlation between each variable
- cor.test() –> Only looks at the correlation between a single pair of variables but also performs a t-test –> to indicate the significance of the correlation –> T-test used to check whether the correlation is statistically different from 0.
NOTE –> use = “pairwise” –> tells R to omit observations with missing data and use complete pairs of observations –> e.g. dismiss NA
Why is it a good idea to transform (log) your data when performing a correlation test, if it is showing a curved relationship?
A problem that may arise when trying to calculate the correlation coefficient –> it assumes as straight line relationship –> Hence, we run in to problems when there is a curved relationship.
Outline the code needed to plot two data sets on the same scatter plot
What code can you use to plot the model diagnostic plots on the same page?
Outline how the different Df values are calculated for TSS, ESS and RSS?
What is the quickest way to plot bar plots showing the mean and the confiedence intervals?
Instal –> Instal.package(gplots)
Load –> Library(gplots)
What is the Tukey Test used for?
Breakdown the following Tukey output
How can you test whether explanatory variables are independent? Can this test be used for all types of data?
Error in Image
Null hypothesis –> Factors are independent
Chi-sqaured is significant –> factors are not independent
Chi-sqaured is not significant –> Factors are independent
Breakdown the following Linear regression summary output
Note - ~ is used to define the response variable (LEFT) and explanatory variable (RIGHT)
1. Call –> Highlights the Response and Explanatory Variable in question –> We are asking whether our Linear model can explain the relationship between the two
2. Residuals –> Outlines the distribution of the residual values using the min, max, quartiles and Median
3. Coefficients Table –> Estimates co-efficients calculates the T-Value for the Coefficients and subsequently performs a T-Test to test whether they are statistically significant
- NOTE –> we have two co-efficients –> One that describes the Intercept and another that describes the Slope of the line (Log(covid$Totalcases)
- From the Coefficients we can generate a Linear Equation
- T-value is calculated by dividing Estimate/Std. Error
- T-Test to check whether our T-Values for each Co-efficient is equal to 0 (H0)
4. Residual Standard Error
5. Multiple R-Squared –> the proportion of the variance explained by your model –> ESS/TSS (ESS+RSS)
- Different to F-Value which is the ratio
6. Adjusted R-squared –> same as Multiple R-squared–> includes a penalty for degrees of freedom –> Hence, Value is Lower
Note - R-squared can be multiplied by 100 to obtain a percentage
7. F-Statistic –> Briefly tells you your F-Value for your Linear Model and the level of statistical significance –> Note running a var.test(..)/analysis of variance will give you a more comprehensive F-Test output
Breakdown the following ANOVA output
Logtotaldeaths ~ Logtotalcases
Remember, in theory we want our model to maximize the F-value ratio –> proportionately more variation is explained relative to variation that is not explained.
1. Left most column –> Each contributing term in the model
2. Degrees of freedom (Df)
a) Term Df –> Calculated by n-1 –> where n is the number of parameters –> simple regression we have two estimates parameters (slope and intercept)
b) Residual Df –> The residual degrees of freedom –> the number of data points that are allowed to vary from the Linear model
3. Sum sq –> Sum of squares
a) Term Sum sq –> thought of as ESS
b) Residual Sum sq –> thought of as RSS
4. Mean Sq –> Each each Sum of sq by their respective degrees of freedom –> yielding EMsq or RMsq
- Note - Mean sq = Variance
5. F-Value –> calculated by dividing each EMsq by RMsq
6. F-Test significance –> Testing our value against the Null hypothesis –> Null hypothesis (H0) = ratio of the variances = 1 –> Are EMsq and RMsq the same?
How would you approach this question?
Remember an F-test is fundamentally used to compare two variances –> Are they statistically the same or not?
Context –> two-sample T-test –> Comparing the Two means of hospital beds per 1000 populations between North America and South America
BUT! For a two-sample T-Test to be reliable there has to be homogeneity of variance –> So this can be followed up with a F-Test!
Output Breakdown
- F-Value –> Ratio of variance between NA and SA
- Df –> calculted by n-1 –> where n is the sample size –> work back to figure out the number of data points for each sample
- P-value –> indicates significance –> THis case it is above 0.05 –> No statistical significance –> we fail toe reject the null hypothesis, meaning the variances are the SAME
Means that the –> we can intrepret the results as they are!
If there was significance (different variance) we could perform a Wilcoxon test instead or attempt to transform the data (log)
