Research Methods and Analytical Procedures Flashcards
Hierarchy of Evidence (Weakest to strongest)?
Expert opinion
Case Reports
Cohort studies / Cross-sectional Studies
Randomised controlled trials
Systematic Reviews
Whats a narrative review?
While narrative reviews are often written by experts it is likely that his/her expertise, experience and beliefs will influence which studies are included in the review
Narrative reviews do NOT influence policy makers as they are NOT ‘good science’, even if the studies they include are individually very good science.
So falls under Expert opinion, it is cherry picking
What are systematic review?
Systematic reviews minimise the likelihood of such bias as the methods used to identify, select and critically appraise relevant primary research, is done in an open, explicit and systematic way.
Aim of a systematic review?
Summarise the findings of ALL studies that address a very specific research question and meet clear inclusion criteria (e.g. must be RCT, >1 mth). And, if possible, quantify the average effect or finding.
Identify gaps or under-researched areas in the literature (e.g. there maybe a shortage of trials in a particular age group, ethnicity, or sport).
Search term used when doing a systematic review? (example for assessing the impact of exercise referral schemes on physical activity and health outcomes
PICOS
Participants - (Has/risk of disease, old, obese)
Intervention type - (exercise referral)
Control/Compared to - (usual advice, diet)
Outcome - (BMI, BP, physical activity)
Study design - (RCT, cohort, cross-sectional)
PECOS for observational studies - P = population, and E = exposure
Types of systematic reviews?
A qualitative synthesis of the studies: these are quite descriptive. They report the number or proportion of studies that report a statistically significant difference – p<0.05 (systematic review)
A quantitative synthesis of the studies. An ‘average’ intervention effect is calculated from all of the intervention effects from the included studies (systematic review and/with meta-analysis)
Description of a Meta Analysis?
The general aim of a meta-analysis is to more powerfully estimate the ‘true’ intervention effect size as opposed to a less precise effect size derived in a single study.
To produce an ‘average’ intervention effect the units need to be same across all studies, or standardised:
[e.g. Diff of Means divided by Pooled SD].
If there is high heterogeneity (wide range) of effect sizes (e.g. some zero effect, some very positive) then producing a single summary estimate is not appropriate (low I2 value = low heterogeneity = effects were similar across studies). This is a score on the forest plot as well (0% is very low heterogeneity,
Notes about a forest plot for a meta analysis?
Is in chronological order In terms of publication date - done in this order as pilot studies normally show more effects than in real world
Square indicates relative risk of each study. Relative risk shows the percentage increase or decrease the study had
The wings represent the 95%CI, if wings cross the line of ‘no effect’ (relative risk ratio=1) then that study did not find a statistically significant effect.
Relative risk found by (the percentage of those who met the criteria who had the intervention) / the percentage of those who met the criteria who didn’t have the intervention)
Bigger studies will have a greater weighting when determining the ‘average intervention effect’. Instead of each having equal weighting represented by a larger rectangle
The kite/diamond shape at the bottom represents the ‘weighted average’ intervention effect and 95%CI. (the amount 1 represents how much % it increases it by) - and if it crosses the line then it is not significantly different (the size of the diamond is its confidence intervals) - can read of the p value (if lower than 0.05) to check significance as well
What differences could come up if you did a quantitative synthesis/ meta-analysis, compared to a qualitative synthesis?
If looked at studies qualitatively, could end up saying all the studies aren’t significant, but when you look at them qualitatively as a collective they actually make the result significantly
Standard criteria for assessing quality/ risk of bias in forest plots?
e.g. Newcastle-Ottawa risk of bias scale:
0-3 = very high risk
4-6 = high risk
7-9 = low risk
Or can do a Meta regression for high heterogeneity
If you add and minus 1 standard deviation to a normal distribution what percent of the data will you cover?
1 standard deviation = 68%
2 standard deviation = 95%
3 standard deviation = 99%
How to test if you data is normally distributed?
Look to see if curve is bell curved - not finite
Skewness (how the curve leans) should be between -1 and 1
Kurtois = (the peakedness or flatness of a distribution) , Kurtois should be between -1 and 2
Can still go ahead and analyse if values are just outside of boundaries
Is under descriptive statistics in SPSS, you look at the statistic values in the skewness and kurtois
Whats negative and positive skew?
Positive is when most of the results are bellow the middle
Negative skew is when most of the results are past the middle
What’s mesokurtic curve?
Kurtois is normal
What’s a leptokurtic curve?
Very peaked - positive kurtois
Whats a platykurtic curve?
Very flat - negative kurtois
T test definitions?
Test whether there is a difference between 2 means
t-tests are used to test if the difference between the 2 means is likely to be a real difference or one that occurred by chance (sampling variation)
p-value = probability that it occurred by chance
two types of t tests
Independent samples t tests?
Independent groups: Different people in each sample mean
Looks at differences between 2 different groups
Dependent / paired samples t test?
One group the sample people in each sample mean (except twin studies)
Look at differences within the same group in different conditions/ time-points (before/after)
Hypotheses types?
Null hypothesis = states there will be no difference between the groups
The alternative hypothesis = states there will be a difference between groups
Are most t tests 2 directional?
Yes, eg. there will be a significant difference between the lowering of blood pressure from drug a to drug b
Occasionally 1 tailed = Drug A will be better than Drug B, (but if Drug B is actually better than Drug A you cannot report it as that is not what you’ve tested, and this would have important implications on people’s health). If Drug B was a newer, much cheaper drug than Drug A, then the implications would be the same whether Drug B was better or same as Drug A ….. under both circumstances we’d recommend people used Drug B. (non-inferiority study)
What does the P value / significance level have to be below for it to be significant and in a t-test mean that the 2 means are significantly different?
p < 0.05 (expect when applying the Bonferroni correction)
This is when we believe the observed difference hasn’t happened due to random chance (95%)
13
Type 1 error:
You reject the null hypothesis - when the null hypothesis is true (you say there is a difference when there isn’t)
Due to random sampling
Type 2 error:
You fail to reject the null hypothesis when the null hypothesis is false (you say there isn’t a difference when there is)
Due to random sampling
Easy way to remember is boy who cried wolf did a type 1 error then a type 2
Assumptions about the data that need to be satisfied for the results of a t-test to be reliable?
Data is continuous and is normally distributed (eg.height weight, strength, glucose)
Both groups are a random sample of a population (if testing whether males are taller than females, don’t take female basketball players and male jockeys)
Homogenity of variance - both groups have similar variances or standard deviations (can visualise this if the distribution curve is far more spread out than the other) - if the assumption is violated in SPSS it offers a an alternative corrected result
Whats a parametric test compared to a non parametric
A parametric statistical test is one that makes assumptions about the parameters (defining properties) of the population distribution(s) from which one’s sample data are drawn (normally distributed continuous data)
- are preferable as provide more reliable and interpretable results, however if sample is small and clearly not normal should use non-parametric tests
Pearsons correlation, independent samples t-test, paired samples t-test, one way ANOVA, Repeated measures ANOVA
A non parametric test is one that makes no such assumptions - used when data is not applicable to a parametric test ie. your sample is small and your data is clearly not normal (ordinal /rank data)
Spearman’s rank order correlation
Mann-Whitney U-test
Wilcoxon signed rank test
Kruskal-Wallis ANOVA
Friedman’s ANOVA
Strength of Evidence with p values?
p > 0.1 no evidence against null
p = 0.05 to 0.1 not quite enough evidence against the null (only relevant if this is the most significant value you got in the whole study
p < 0.05 moderate evidence against null
p < 0.01 strong evidence against the nulll
p < 0.001 very strong evidence against the null
What do ANOVAS do?
Compare 3 or more means
Type of ANOVA depends on:
The number of independent variables, 1 IV = one-way, 2 IVs = two-way and so on
The type of independent variables:
Is it independent samples = groups
Or repeated measures = conditions or time-points
Types of ANOVA?
One way independent sample ANOVA (one independent variable)
Two way independent sample ANOVA (two independent variables)
One way Repeated Measures ANOVA (one independent variable)
Two way Repeated Measures ANOVA (two independent variables)
Two way Mixed Design ANOVA (when you have independent samples and repeated measures) (Two independent variables)
Why not compare several means with lots of t-tests?
Cannot look at several variables simultaneously:
Performing multiple t- tests inflates the type 1 error rate (as for every test have to 0.95^x, with x being the number of tests)
Theory of independent sample ANOVA and how to work out F ratio?
The F-ratio = Between group variance / Within group variance
Between group variance = distance between group means (of the outcome variable) and the overall sample mean. (This is due to differences in the independent variable or experimental manipulation)
Within group variance = Distance between each participant and the mean of the group they belong to
How to find critical values of F for 0.05?
Degrees of Freedom: - how many values do you have to know for you to be able to work out the final value if know the sum of squares
A = For between groups = (Number of groups - 1)
B = Within groups = (Number of people in the study - the number of groups)
So Write out represents (A,B)
Look up in the table Where A and B meet, and this is your critical value, and if F is larger than the value you find in the table, the result is significant
How is the ANOVA an omnibus test?
The F-ratio can only tell us if group means are different
It does not tell us which group means differ from which
We need follow-up / ‘post-hoc’ tests to find out where the group differences ar
Hypothesis in One-way IS-ANOVA?
Null = There are no differences between the group means (mean1 = mean2 = mean3)
Alternative = At least one group mean is different from the other group means (not all means are equal)
Assumptions of IS-ANOVA
Random selection and representative sample
Normal distribution each group
Equal variance in each group (homogeneity) - done by Levene test - if significant homogeneity has not been met
(even if data is not normally distributed and variance is unequal, if sample sizes are large and close to equal, the F in ANOVA is reasonably robust
What is the Sum of Squares?
Distances from the means
Between groups = sum ((mean of group - overall mean)^2) x number of person in each group)
Within groups = Sum((Individual values - mean of group)^2)
Total sum of squares = Sum((individual values - overall mean)^2)
Working out degrees of freedom between groups, within groups and total
A = For between groups = (Number of groups - 1)
B = Within groups = (Number of people in the study - the number of groups)
C = Total = Number of people in study - 1
Example of reporting an IS-Anova?
There is very strong evidence to reject the null hypothesis in favour of the alternative hypothesis. There is a significant difference in the cognitive anxiety of sports men and women depending on the competitive level at which they play (F(3,60) = 14, p<0.001)
When looking cat SPSS output post hoc-tests how do you what is significantly different?
Look through and see what comparisons are significant
Example writing it:
Post Hoc independent t-tests with a Tukey’s correction to the alpha were performed in order to identify where differences in competitive level lay
Results revealed that club level athletes demonstrated significantly higher levels of cognitive anxiety than any of the other three competitive level performers (Club = 16.6, county = 12.2, national = 10.3, international = 9.1 , p <0.01). However Anxiety was not different between County, national and international level performers (all p > 0.05)
Levels in a two-way IS-ANOVA?
2 x 2 = eg. gender (M & F) x Common cold (Y/N) on minutes spent ‘not concentrating in statistics lectures’
2 x 4 eg. breakfast status (Y/N) x hours of sleep (2, 4, 8 , 8) on average power output performance in 30sec Wingate test
3 x 3 = eg. weight status (normal/overweight / obese) x no of cars in house hold (0/1/2+) on how much you exercise
Whats the issue with 2-way IS ANOVA?
Like any ANOVA F-ratio only can tell us not all the group means are not the same so have to do post hoc tests for follow up
However 2 way IS ANOVA is very clunky on follow up tests for sig interaction and very underpowered
What is an interaction?
This is when the effect that one factor has on the outcome variables differs depending on the levels of another factor
Examples:
If one line changes as the other factor does, but the other factor doesn’t shows an interaction
Both main effects significant but not the interaction (lines parallel and sloped with a gap so gender and comp level both effect)
(or parallel and straight = just gender effect)
(or parallel and sloped put lines on top of each other means only comp level is effecting)
How to inexpert main effects when the interaction is significant?
On SPSS see it on the gender*level significance for example (seen on graph from diverging towards or away from each other)
Usually if an interaction is significant we would not interpret the main effects even if they are themselves significant, as will not be true at certain levels
Sometimes can interpret main effects when the interaction is significant
One-way RM-ANOVA?
The levels within the ONE factor represent conditions/time-points measured on the ONE group of people
Each participant provides outcome data for all conditions/time-points of the one factor
Example of a One-way RM-ANOVA?
Each person had glucose and insulin monitored on three dats, each day under a different condition (the orders in which people did these conditions was random)
- Sitting all day (9hrs)
- Sitting all day apart from 2 mins of standing every 30 mins
- Sitting all day apart from 2 mins of walking every 30 mins
What does variance within individuals composed of?
‘Condition variance’ - variance due to the experimental condition (SSc)
‘Error variance’ Variance not explained by the condition (SSe)
Strengths of a RM-ANOVA?
High sensitivity:
Less unsystematic variance (less noise) compared to if you had 3 separate groups for example (Over 1 group doing all 3 conditions)
Greater sensitivity to sometimes subtle experimental effects, as each participant acts as his/her own control
Economical
You need less participants to achieve the same power
But they need to visit the lab more often
Assumptions for a One-way RM-ANOVA?
Assumption of Sphericity - checked using Mauchly’s test
Variances of the differences between each pair of the repeated measures factor should be the same
p > 0.05 = variances of the differences are not the same, sphericity is assumed
p < 0.05 = variances of the differences are not the same, assumption of sphericity is violated
eg. variance (sit - stand) = variance (sit-walk) = variance (stand - walk)
Assumption of normality:
Outcome data should be ‘normal’ at each level of the condition or time-point
Don’t be scared of slight violations to normality, RM-ANOVAS are reasonably robust to them
Correction factors for non-Sphericity?
If p < 0.05
Greenhouse-Geisser (conservative finds fewer differences)
These lower the DFs to correct for the effect of sphericity
So when looking at the critical value table your degrees of freedom will be lower, so the critical value will be higher so less likely to be larger than the F value so less likely to find a significant result
How to report Greenhouse-Geisser corrected F-test?
F(corrected df value, next corrected df value) = F value, p < or > sig value
Whats a two-way fully RM-ANOVA?
The levels within each of the two factors represent different conditions/time-points measured on the one group of people
Each participant provides outcome data for all conditions/ timepoints within each of the two factors
Example of a two-way fully RM ANOVA?
10 cyclists undergo a stability test on the bike measuring the amount of times they go off the white stripe at timepoints 3, 6, 9,12,15 minutes were resistance increases each time
They then undergo a 6-week core stability training programme designed to improved their cycling ability
They then repeated the test, providing 3 research questions
Does the cyclists stability change with increased fatigue? (TIME)
Does the cyclist stability score change as a result of the training intervention (between the two ‘TEST’ days 6-weeks apart)
Is there a ‘TEST day x fatigue Time’ interaction does the change in stability pre and post training program differ depending on the fatigue time (3,6,9,12,15?)
Post Hoc tests for significant interactions?
Only if:
The F ratio indicates a significant interaction effect
Need to be done manually in SPSS:
SPSS does not allow you to run post hoc analyses on significant interactions, so you have to reduce how best to do them
Be efficient:
If you make too many comparisons your new ‘Bonferroni corrected’ alpha level will be very small (or… your new ‘Bonferroni corrected p-values will be very high) and consequently it will be difficult to reveal sig.difference
Methods and when to use for post hoc tests for significant interactions?
LSD (least sig diff) (t-test):
Use when number of comparisons of interest is small or when there are at most 3 conditions to be compared
Tukey:
Use when there are 4 or more groups in between subjects (independent samples) designs AND approximately equal numbers of subjects per condition
Bonferroni:
Use when there are 4 or more conditions in within subjects (repeated measures) designs. Or when mixed model (between and within subject variables)
How to manually do the Bonferroni correction?
0.05/ number of tests = new alpha/sig threshold (compare to original p-values)
Or
p-vlaue x No. of tests = new p-value (compare these to 0.05)
So say there were 5 tests, could do 5 paired t-tests, then have to compare the p-values to 0.01 significance as we did 5 tests
This method works when only some points are significantly different and some others, if all the points are significantly can’t identify where the interaction is, so have to use a different method in comparing change
If only do a test comparing where you think the change will occur, will have done less tests, so more likely to find a significant result
So could look at the change between 2 points
Reasons why during the cycling protocol why the training intervention might not be the only reason why they improved fatigued next time?
Improved from familiarity
Those who did poor practiced and got fitter over 6 weeks
This is why we need a control group
Difference between independent samples and repeated measures?
Independent samples: The levels in one factor represent different groups, where each person only provides data for one of the groups
Repeated measures: The levels in one factor represent different conditions/time-points, where each person provides data at every level of this factor
How to interpret your interaction?
Sometimes the between groups effects are of more interest (eg. when groups are intervention or control)
Sometimes the between groups effects are secondary (eg. when grouped by gender. The ‘experimental conditions’ are the repeated measures ie.e everyone does all experimental conditions
Description of a two-way mixed design ANOVA?
Two groups of people were either told to practice putting explicitly (told instructions) or implicitly (learn by themselves) they then underwent 5 tests, the first 4 were casual and the last one was under stressed conditions
3 hypotheses tested:
Is there an interaction of learning style and trial number on putting performance?
Is there effect of learning style on putting performance? Is there a sig difference in the overall putting score (mean of all 5 trials combined) between the ‘explicit instructions’ and ‘implicit learning’ groups
Is there an effect of trial number on putting performance? (is there a sig difference in the overall putting score (mean of both ‘learning styles’ combined) between any 2 of the 5 trials?
To find an interaction can do t-tests before and after where you think it, one difference will be sig and the other won’t (most likely) can then confirm this with a t-test looking at the change between the 2 points
Why is Sample Size an ethical issue?
Too big = overpowered, you could have shown a statistical difference with fewer people, you are wasting resources and time, and put unnecessary discomfort on participants
Too small = underpowered, could fail to indetify a true meaning, unethical as wastes resources, time and could lead to misleading conclusions
Sample Size calculation?
Desired Power (1-B): Should not be less than 80%. Funders may request higher
Desired significance level (a): 5% (p=0.05)
Effect size (standardised - d (t-test), n^2(analysis of variance), R^2(regression)
What is Power?
(1-B) is the probability that your study will find a statistically sig effect (if the anticipated effect size is the true effect size)
E.g. Testing a drug that is known to reduce SBP by 10mmHg (Cohen’s d=1.2)
A ‘Effective Drug’ trial is conducted 100 times on 100 different samples of 8 people. Having
80% power means we would get a statistically sig. effect of the drug in 80 trials and fail to get a statistically sig. effect in 20 trials, even though the drug is ‘known’ to be effective (↓SBP=10mmHg; d=1.2).
Way to visualise this is imagine having 2 bell curves, 1 for the placebo and one for the intervention that is known to be effective (8 participants), the end of placebo curve (2.5% (which represents p < 0.05 as 2.5% at each end of curve) represents the cut off point for the intervention (80%), so even if get a result in the right direction for the intervention if does not land in that 80% is not significant
Bigger the amount of participants the tighter the normal curves become as less chance for randomness, so the overlap between intervention and placebo curves becomes smaller. So say we now have 16 people in, that overlap wouldn’t represent 80% anymore, but would represent 99%
Drug that reduces BP by 10mmHg (d=-1.2), is trialled 100 times (each with n=16), 99 trials will identify a significant effect, only 1 will not
Compared to Drug that reduces BP by 10mmHg (d=-0.8), is trialled 100 times (each with n=16), 80 trials will identify a significant effect, 20 will not. (this is because the -0.8 is less of an effect than the -1.2)
Conclusion of a larger sample size?
A larger sample size can………..
give you more power to find the same effect size
and/or
give you the same power to find a much smaller effect size
Determing the ‘size’ of effect options?
1.The smallest effect size that would be deemed important to health or performance (e.g. 5mmHg reduction in BP)
2.The likely effect size based on the results of a very similar trial, or data from a number of different studies (mechanistic, pilot, feasibility).
3.Pre-defined values corresponding to small moderate and large effects (if you cannot do points 1. or 2.)
The smallest effect size that would be deemed important to health or performance details?
Smaller effect sizes require larger samples sizes. If you say 5sec longer in Time-To-Exhaustion endurance test (of ~10min) is important, then you may need 1000+ people to be confident that this is a real difference and not one due to random variation.
Larger effect sizes require smaller samples sizes. However, you shouldn’t choose a large effect size as a way of justifying a small sample as you might wrongly conclude your intervention was ineffective.
E.g. If you power a beetroot intervention to detect a 20% reduction in BP, and yours leads to a 10% reduction, you will conclude beetroot is ineffective at reducing BP (yet 10% is still beneficial to health)
The likely effect size based on the results of a very similar trial, or data from a number of different studies (mechanistic, pilot, feasibility).
Easy if trial is very similar! (be guided by the ‘effect size’ not the ‘sample size’ as it may have been non-sig!). Better to use 2or3 studies rather than just 1.
If no trials are ‘very similar’, you can still make an informed estimate from different aspects of several ‘slightly similar’ or ‘relevant’ studies.
IT IS NOT AN ‘EXACT’ SCIENCE
The estimate of the likely effect size is based on mean effects, SDs and correlations that are themselves obtained from small studies.
However, this is still much better than the ‘always n=10’ approach.
Pre-defined values corresponding to small moderate and large effects?
t-tests
Effect: d
small = 0.2
medium = 0.5
large = 0.8
ANOVA
effect: Eta^2 (n^2)
small = 0.01
medium = 0.06
large = 0.14
Regression
Effect: R^2
Small = 0.01
Medium = 0.09
Large = 0.25
How to work out effect size dz?
(Change of Mean differences)/(SD of diffs)
SD of differences isn’t worked out just be subtracting one groups means from the other like the change of mean differences is done
Have to do (upper quartile - lower quartile from 95% confidence interval of the difference) / 4, then multiply this by the square root of the amount of subjects in the study
What do you do to your sample size if you are told a drop out rate?
Calculate how many people this is, then add it on top of your calculated sample size
Questions about G power sample size in the exam will only have one about the output, the rest will be more broad questions
ok
If you converted a study looking at creatine supplementation on cycling effect from young people to old people what might you have to change for sample size calculation?
A lower correlation might happen as older people are less consistent on a Wingate, or a wider variation in the effectiveness of creatine = effect size goes down, need more participants
Pre and Post SDs might increase to 0.75 reflecting a wider range of ability in old people = effect size goes down, need more participants
What is the correlation Coefficient (r)?
Summarises the strength and the direction of the linear relationship (sign of the number shows the direction, the size of the number shows the strength)
Can take any 2dp number between -1 and +1
Different strengths of relationships?
1 = perfect
0.7 = strong
0.5 = moderate to strong
0.3 = moderate
0.1 = weak
0 = none
Same applies for negative
What does SPSS report r and p as?
r = Pearsons
ion
p = Sig. (2-tailed)
When you report a correlation what are you comparing to?
You say there is a significant difference in r compared to r being 0
Assumptions for Pearsons correlation?
Continuous scale
Normal distribution - not an issue if just falls outside the -1 to +1 value if none of the other factors do, if more do fall out, should use the spearman rho or Kendall’s tau as non-parametric alternatives in same window as Pearson’s
Linear relationship - if the points tend to cluster around a line, but this is not a straight line, the relationship is described as non-linear or curvilinear
No obvious outliers - has to be evaluated (measurement error or not from population of interest)
Homogeneity of variance - homoscedasticity is good (2 parallel lines can encompass data), heteroscedascity is bad (fan or funnel shape goes around data) - have to eye ball this no formal test
Why correlation is not causation?
Just because 2 variables are correlated does not mean that one causes the other, the causal factor could be a 3rd variable that you haven’t analysed or even measured
E.g. for the correlation of Fat% v CVhealth a 3rd variable ‘stress’ could be causing high levels of fat (eating more due to stress) and independently of of fat, stress could be causing poorer CVhealth because of increased cortisol heroine that causes stress
What can regression analysis do?
We can see how much of the variance in one variable is ‘explained’ by variance in the other
At a basic level we square the correlation coefficient eg. r = 0.615 would become R^2 = 0.38
So in BMI vs Fat% this would mean BMI explains 38% of the variance in Fat% or visa versa as r = 0.615 and then R^2 = 0.38
So BMI explains 38% of the variance in Fat%
Variance explained in regression?
100% of the variance is found by the sum of the squares (distance from line squared from each point then all added up) compared to the mean line
If you then do the same sum of squares compared to the line of best fit and work that out as a percentage of what it is compared to the mean line, then that difference will be the R^2 value
How to get an equation to predict from a r value?
Y = b0 + b1X
Y = the dependent variable (outcome we are trying to predict)
b0 = the intercept of the y axis (when x = 0)
b1 = the slope or the regression coefficient
X is the value of the independent variable (the measure your using to predict the other)
How does simple linear regression lead to multiple linear regression?
When you have made the equation, you will see there will still be differences between that an the data points, these are residuals, so we need to know more information to make a better prediction
How is simple (bivariate) linear regression different to multiple linear regression?
Linear has one independent variable
Multiple has multiple independent variables (each variable might explain a certain amount of variance in the dependent variable, but when you put them all together will not be the sum of the variances)
(imagine it like a van diagram with circles overlapping each other)
Multiple Linear Regression Equation?
Y = b0 + b1X1 + b2X2 + b3X3…
Thumb rule: more than 10 people per variable in final model
Types of multiple regression module building processes?
Stepwise multiple regression (forward) SPSS selects which variables are entered into the model and in what order based on their strength of association with the outcome variable
Hierachial multiple regression
Experimenter decides the order in which variables are entered
Forced entry multiple regression
All predictors are entered into one model at the same time
Describe Stepwise regression?
SPSS identifies the IV which explains the most variance in the DV and puts that in the model
The IV which explains the most of the remaining unexplained variance (the residuals of the first model are now the DV) is then included in the model provided it explains a significant amount of the remaining variance
This process is repeated until there an no IVs left that explain further variance p < 0.05
IVs in the model can get taken out if they become ‘clearly no longer significant’ p>0.10 due to the entry of a new variable
When used?
Data driven, not theory driven. Researchers chose a list of variables they think are likely to be predictive but not sure as not enough research has been done to inform them. Also useful if you have lots of variables which you couldn’t possibly know which are most predictive
Describe the hierarchal multiple regression?
Researcher decides what order the IV are entered
Order of entry should be based on previous research or theory
Usually possible/known confounders such as gender, age, ethnicity, social class, smoker, alcohol, birthweight
Can examine R^2 - can see the amount of additional variation that is explained every time you add another variable to the model
When used? If you want to know whether a certain variable of interest is significant when you add it to a model that already contains a ‘known’ predictor or confounder
Describe forced entry multiple regressions?
All predictors are forced into one model simultaneously (irrespective of whether sig or non sig)
Should be a theoretical rationale for including all variables
When use? Does not determine the unique variance that each IV adds to the model. However you can re-run the analysis manually with the most non-sig variables removed (manual backward elimination method)
When looking SPSS multiple regression what model do you look at to get values?
Look at the final model
We also aren’t concerned about looking at ANOVA box during these analyses
Assumptions for multiple linear regression?
No Multi-collinearty between IVs in model - the square root of the ‘variance inflation factor (VIF)’ indicates how much larger the standard error is due to the co-correlation
No r values above 0.9
Largest VIF should be smaller than 10, if 2 are bigger than 5 remove one of them
Average VIF should should not be considerably bigger than 1
Homoscedasticity of residuals - look at the graph of regression standardised predicted values
Should be spread equally throughout the parallel lines, fuelling or fanning represents heteroscedascity - this represents high accuracy at parts where not very fanned and low accuracy where very fanned
Linearity of residuals - look at same graph of regression standardised predicted values, and imagine putting a line of best fit though it, if not violated this will be horizontal
This means the relationship is linear or that any non linear pattern in the outcome measure has been accounted for by our predictor values
Normality of residuals - looking at a normal bell shaped curve on histogram
Extra:
Casewise diagnostics
Used to identify outliers and potentially influential cases
In a normal sample 95% cases roughly + or - w, therefore about 5% will have residuals outside the + or - 2
So calculate roughly how many we expect to see fall out of the range and see if there’s actually a lot more shown in the case wise diagnostics and if this then violates the assumption
Assumptions for simple linear regression?
Homoscedasticity of residuals
Linearity of residuals
Normality of residuals
What is a moderator variable?
A moderator variable is one that affects the direction and/or strength of the relationship between an IV and Dv
Does the relationship between the predictor (IV) and the outcome (DV) carrying depending on the level/value of the moderator variable?
Moderator variable can be continuous or categorical (so example for social support could be amount, or we split them into people who have 3 or more up to 5, or 2 or less)
They are often categorical eg. gender 1 or 2
So its basically an interaction
Moderator equation for the example of golf performance being effected by stress and social support?
So the moderation bit is Stress X Social Support
Performance = Intercept + b1(stress) + b2(social support) + b3(stress x social support)
Why do we use hierarchal input for multiple regression looking at moderator?
Stepwise not suitable as if interaction is sig, then the non-sig main effects need to be in the model
Is multi collinearity an issue when looking at moderators in multiple regression?
Multi-collinearity can make choosing the correct predictors to include in the model very difficult. This is because it interferes in determing the precise effect of each predictor because they are so correlated/similar
However, multi collinearity doesn’t affect the overall fit of the model or produce bad predictions!!! So it is okay to know what variables are going in to the model.
Hinted to be in Exam, how do you work out golf score if given stress level and social support level?
Plug into the equation
Equation might be separated out categorically, ie a high support equation and a low support level, so need to plug numbers into relevant equation
These lines are also useful to show the interaction effect.
Chi-square test compared to Logisitc regression?
Chi-square - Testing the difference in the ‘proportions of yes answers’ between 2 or more categories/groups
Logisitic regression - converting those ‘proportions differences’ into Odds Ratios (OR) for multiple ‘risk factors’ simultaneously in a model-
(Outcome/ the dependent variable must be dichotomous (Yes or No), and risk factors/independent variable can be continuous, categorical or dichotomous
Assumptions of logistic regression?
Does not make any assumptions of normality, linearity and homogeneity of variance.
However you should check for very big’ outliers in the continuous risk factor/independent variables
The minimum expected count in any’ outcome x risk factor’ cell must be greater than 5
Sample size requirements:
For a large effect size, the preferred people to variables ratios are
20-1 for simultaneous or hierarchal logistic regression
50-1 for stepwise logistic regression
Goodness of fit in logistic regression?
The goodness of fit of the model is measured by’-2 log likelihood’. However, this is not easily interpretable so a pseudo R^2 value is provided
The pseudo R^2 is not comparable with linear regression R^2 as predicting risk of yes/no. not a continuous value
How to work out odds ratio
?
The amount of people who are yes in that category/ the amount of no in that category
Then do that for the other category
Then divide the first value by the next value if wanting an answer for the first category
Small odds ratio?
Logistic Reg (odds ratio):
Larger than 1.5, or smaller than 0.67
Correlation (r and R^2):
r is smaller than 0.01 or 1%
Moderate odds ratio?
Logistic Reg (odds ratio):
Larger than 3, or smaller than 0.33
Correlation (r and R^2):
r is smaller than 0.30 or 25%
Large odds ratio?
Logistic Reg (odds ratio):
Larger than 5, or smaller than 0.2
Correlation (r and R^2):
r is smaller than 0.5 or 25%
What is Exp(B)?
odds ratio
If Exp(B) is bigger than 1then the odds are greater in that category compared to the reference category
If Exp(B) is smaller than 1, the odds are smaller in that category compared to the reference category
To work out the percent more or less than category will do something do (odds ratio - 1) x 100 (the other gender won’t be the exact opposite of what you got from the other odds ratio)
(be careful of what value you are looking at SPSS as it depends on the reference and last category entered)
How to work out odds ratio from chi square SPSS table
Say wanted to find out females
Would do odds of females in the count (yes/no)
Would do odds of males in the count (yes/no)
Divide the females value by the males value
What does Cox and Snell ‘pseudo R^2 value’ mean?
Quantifies the goodness of fit’ of the model
What does Cox and Snell ‘pseudo R^2 value’ mean?
Quantifies the goodness of fit’ of the model
Whats in intra-class correlation (ICC)
ICC - data from the 2 variables are centred and scaled using a pooled mean and SD (so comparing the 2 measures)
Then compared to Pearson’s correlation (r) - when each variable is centred and scaled by its own mean and SD (looking at one of the measures)
How to read/interpret an ICC in SPSS?
For example The ICC = 0.70 (p<0.001) indicates a moderate-to-good level of agreement between the ACC and the LOGBOOK for measuring sitting mins/day
Look for Single measures value, then the signifificance
What are Bland-Altman plots used for?
Bland-Altman plots are used to evaluate the agreement between 2 measurements methods or techniques often between a new measurement instrument and a gold standard/criterion method (criterion validity). This is how ‘new’ tools/devices/questionnaires are developed (multiple interactions vs criterion measure
Bland Altman plots allow identification of any systematic difference between the measurements
How to interpret a Bland-altman plot example?
On average the accelerometer recorded 15min/day less sitting than the logbook (shown from mean line)
However, the 95% LoA (mean diff +/- (1.96 x SDdiff)) show that for 95% of individuals these differences range from 147min/day less, up to 118min/day more (both shown by confidence interval lines) sitting time compared to logbook
Modify either tool to reduce 95% LoA, could apply a 15min adjustment factor
Variables on each axis on bland Altman plot?
Mean of the 2 methods (Method 1 + method 2)/2
Diff between the 2 methods (Method 1 - method 2)
When copying and pasting his answers make sure to tweak them so TurnItin doesn’t flag it
ok
How to report a t-test result example?
is on week 4 worksheet
Can you use overlapping confidence intervals as a sign of significance in paired or independent t tests visually?
Can only do it in independent, can’t do it in paired
In independent t-tests can see if crosses 0 to represent hasn’t significantly changed
How to work out f ratio from mean squares?
F ratio = Mean square between groups / mean square within groups
When getting degrees of freedom for an ANOVA where do you get them in the SPSS output?
You get the first one from the effect your looking at, and the other from the error row
