Lecture 6 - Statistical Tests II: Linear Regression

Question 1

in what instance would we select a linear regression?

Answer 1

interested in association → interested in trend [where x is continuous] → “experiment” → y-continuous → y-normal → linear regression

Question 2

linear regression is a statistical model which shows:

Answer 2

the relationship between 2 continuous variables

Question 3

what questions should we be asking when we are choosing a statistical test?

Answer 3

(1) what type of response variable? [continuous, discrete/count, proportion, binary]

(2) what type of explanatory [continuous, discrete/count, proportion, binary, categorical]

(3) interested in differences or trends/relationships?

(4) paired or independent sample

(5) normal/normal distribution

Question 4

what type are variables are present when we select a chi-squared statistical test?

Answer 4

when we are dealing with two categorical variables (y-counts, x categorical)

Question 5

what variables must be present in order for us to carry out a linear regression statistical analysis?

Answer 5

for a linear regression we must both have a continuous X & Y variable

Question 6

gradient =

Answer 6

change in y / change in x

Question 7

what are the three stages when trying to calculate a linear regression?

Answer 7

(1) choose your model: linear / non-linear

(2) estimate the parameters of the model

(3) model fit: how well does the model describe our data

Question 8

what is a Y bar that is found horizontally across the span of a graph?

Answer 8

a Y-bar, indicated by a dotted line labeled with a Y with a line on top of it shows the mean value line in your data

Question 9

how do you calculate the total sum of squares?

Answer 9

total sum of squares is the sum of all the squared distances between your data points and the Y-Line

Question 10

what is the equation of a line and its units?

Answer 10

y [w/ a hat] = a +bx

where:
a = intercept
b = slope

Question 11

what is the error sum of squares (residuals)?

Answer 11

error sum of squares or residuals = the sum of all the distances between each individual data point and the line of best fit (y[w/ a hat] = a + bx)

Question 12

what must all lines of best fit pass through and what allows us to choose what line of best fit is the most appropriate one?

Answer 12

all lines of best fit need to go through the mean-line of Y & X

we select the best line when the unexplained variation in our response is the smallest - when our residuals are the smallest

Question 13

with regression, if the slope is positive or negative, what does this show about the relationship between the two variables?

Answer 13

if the slope is positive: the relationship between the variables is positive

if the slope is negative: the relationship between the variables is negative

Question 14

what happens to the total sum of squares, SST, if we add additional data points?

Answer 14

the value gets larger

Question 15

how to calculate mean sum of squares:

Answer 15

calculate mean variability = mean sum of squares (MS) = divide our total sums of squares by our sample size

mean sum of squares = sum of square deviations from the mean / degrees of freedom

Question 16

how do you construct and fill out an ANOVA table?

Answer 16

source | SS |D.O.F| MS
regression | SSR| 1 | SSR
error | SSE| n-2 |S^2=SSE/N-2
total | SST| n-1 |

[for regression: you need an additional column called ‘F’ which is the statistic in which F = SSR / S^2

Question 17

degrees of freedom regarding SST & SSE:

Answer 17

• SST requires estimation of 1 parameter (mean of Y) => n-1 degrees of freedom
• SSE requires estimation of 2 parameters (mean of y, slope) => n-2 degrees of freedom

Question 18

SSR + SSE =

Answer 18

SST

Question 19

F-distribution percentile (5%) command in R:

Answer 19

qf(0.95,1,n-2)

Question 20

what is F is larger than the critical value?

Answer 20

this means that we must accept the alternative hypothesis and reject the null hypothesis - we infer that the probability that relationship is due to chance is <0.05

Question 21

we are only allowed to add a trend line if:

Answer 21

we are only allowed to finally add out trend line if we reject the null hypothesis that the slope = 0, if the slope is not significantly different from 0, we must not add a trend line

Question 22

we are only allowed to carry out a linear regression if certain assumptions are fufilled:

Answer 22

• residuals must be normally distributed
• the variance associated with the distribution of the the residuals ions constant (ie. variation in y does not increase with increasing x)
• individual measurements are independent
• data comes from a random sample

Question 23

how can we test if our assumptions are met or violated when questioning wether we can write a linear regression?

Answer 23

we can see if our assumptions are violated through using diagnostic plots in R

residuals vs fitted: in the first one we ask whether the variance is consistent or constant [we want it to look scattered like the sky at night]

normal Q-Q: int he second plot we check whether residuals are normally distributed and we want them to fall onto the regression line (just about)

Question 24

full and complete command needed to have a linear regression in R:

Answer 24

(1) data<-read.csv(“excel_sheet1.csv”, header = T, stringsAsFactor = T)

(2) attach(data)

(3) names(data)

(4) m1<-lm(y variable~x variable)
#”m1” is simply your model name

(5) summary.lm(m1)

(6) summary.aov(m1)

(7) plot(m1)

once you see the sky at night and straight line you can assume normal distribution and therefore plot your linear regression

Question 25

how can you interpret the results of your plot(m1) command in R?

Answer 25

you will be given two micro-graphs: Residuals vs Fitted & Normal Q-Q

for Residuals vs Fitted you want to see “sky at night”

for Normal Q-Q you want to see a plot where data point follow the line of best fit

Question 26

how can you create a linear regression model and make a graph with a trend-line in R?

Answer 26

create a linear regression model using the command:

m1<- lm(y-variable~x-variable)

plot(y-variable~x-variable, pch=19, las = 1)

abline(lm(y-variable~x-variable))

Question 27

what can you not add when in a linear regression when your p value is greater than >0.05

Answer 27

you can not add a regression line!

Question 28

what sort of sample does the data come from in linear regressions?

Answer 28

random samples are used in linear regressions

Question 29

type of variance in a linear regression:

Answer 29

variance is constant in linear regressions

Question 30

total sum of squares [SST] =

Answer 30

regression sum of squares [SSR] + error sum of squares [SSE]

Question 31

null and alternative hypothesis in regards to the slopes in linear regressions:

Answer 31

null hypothesis = the slope is not significantly different from zero

alternative hypothesis = the slope is significantly different from zero x

Question 32

regressions are used when:

Answer 32

• when we are interested in how x and y are related
• to analyse experimental data (x manipulated)
• when x and y cannot be swapped: assume y depends on x

Question 33

linear regression null hypothesis:

Answer 33

slope = 0

Question 34

when is the slope not significantly different from 0?

Answer 34

SSE = SST