Exam 2018 Flashcards
what level in the NHMRC evidence hierarchy are cross sectional studies?
IV
What are some other names for a cross sectional study?
Also known as a cross-sectional analysis, transversal study, or prevalence study
What kind of study is a cross sectional study and what does it do?
- Is observational
• Is descriptive
• Collects data from a population at one specific point in time
How are the groups determined in a cross sectional study?
• Groups determined by existing differences, not random allocation
What are the advantages of a cross sectional study?
- ‘Snapshot’ of a population at one point in time
- Can draw inferences from
existing relationships or differences - Can use large numbers of subjects
- Relatively inexpensive
- Can generate odds ratio, absolute risk, relative risk, and prevalence
6 Could combine finding with other research to develop a hypothesis about why the prevalence of certain disease increases with “factor”
What are the disadvantages of a cross sectional study?
- Results are static (time bound). No indication of a sequence of events or historical or temporal contexts
- Does not randomly sample
- Cannot establish cause and effect relationships
What are the ethical considerations of cross sectional research?
Must promote
• aims of research (knowledge, truth, and avoidance of error)
• values that are essential to collaborative work (trust, accountability, mutual respect, and fairness)
• public support for research
• moral and social values (social responsibility, human rights, animal welfare, compliance with the law, and public health and safety)
What are the 14 CASP questions for a cross sectional study?
- Did the study address a clearly focused issue?
- Did the authors use an appropriate study design to answer their
question? - Were the subjects recruited in an acceptable way?
- Were the measures accurately measured to reduce bias?
- Were the data collected in a way that addressed the research aims?
- Did the study have enough participants to minimize the play of chance?
- Have the correct statistical methods been selected? Are they clearly described with rationales?
- Was the data analysis sufficiently rigorous?
- Have the authors taken account of the confounding factors in the
design and /or analysis phase? - How are the results presented and what was the main result?
- How precise was the result?
- Is there a clear statement of findings?
- Can the results be applied to the local population?
- Howvaluableistheresearch?
What are the five P’s?
Population Problem Prevalence Pos/Neg Clinical implications Proposal
What does Pearson’s correlation coefficient [Rho-ρ] measure?
linear relationship between two variables with ρ=0 suggesting ‘no
linear’ relationship [may have non-linear relationships?]
What do Pearson’s product–moment correlation analyses measure?
Whether the continuous outcome variables were associated with the set of independent variables
What is the problem with Pearson’s correlation coefficient and Pearson’s product-moment correlation?
They offer crude linear associations and unable to adjust for other variables, so need multiple linear regression
What is the purpose of regression modelling?
• To investigate whether an association exists between the variables of interest
• To measure the strength (as well as direction) of an association between the variables
• To study the form of relationships
For a continuous outcome, relationships can be examined by linear or non-linear regression models
For a categorical outcome, logistic regression is usually used to examine possible relationships
Consider a linear model with a positive slope: Y=3+2X
What would it mean If X=0, Y= 3 + 2 (0) = 3
►1 unit increase in X results in 2 units increase in Y
►A positive slope (+2) implies upward slopping line and a positive association
Consider another linear model with a negative slope: Y=3-2X
What would it mean If X=0, Y= 3 - 2 (0) = 3
►1 unit increase in X results in 2 units decrease in Y
►A negative slope (-2) implies downward slopping line and a negative association
What are the considerations for Linear regression?
- The response or outcome variable [DV] must be continuous (e.g. weight, balance measure)
- The independent variables can be categorical or continuous, or a combination of both
• In linear regression, we test the null hypothesis of no relationship between the DV and the IV. If β represents regression coefficient of the DV and the IV:
H0: β = 0
Ha: β ≠ 0 [two-sided]
• Alternative hypothesis can be one-sided (Ha: β>0 or Ha: β<0), depending on the research question.
What are the assumptions for Linear Regression?
- The relationship between DV and IVs is linear
- The observations are independent and randomly selected
- Homogeneity of variances – constant variance
- The residuals (differences between observed and predicted observations) are independent and normally distributed
- The effects are additive
- Absence of outliers and multi-collinearity
What are some steps to take before examining the data for relationships?
- Descriptive statistics of all variables
- Distribution of outcome variables using histogram, quantile-quantile (QQ) plot, normality tests
- Scatter plot to examine linearity
- Collinearity diagnostics
- Appropriate transformation for normality if an outcome variable is not normally distributed
- Use median, range, inter-quartile range to summarize non-normal data
If a variable had skewness=0 & kurtosis=0, what would it’s distribution be?
Normal
• The further the value is from zero, the more likely it is that the variable is not normally distributed.
How will the data be skewed if If the mean > median -
positively skewed
How will the data be skewed if If the mean < median -
negatively skewed
What would it mean if mean, median and mode were all equal?
Normally distributed variable
What does a positive value of skewness indicate?
A pile-up of scores on the left side of the distribution (positively skewed).
What does a positive value for kurtosis indicate?
A pointy and heavy tailed distribution.
What is leptokurtic?
- pointy, heavy tailed
What is Mesokurtic?
normal
What plots can you use to check normality?
Histogram, Box-Whisker, and QQ plots
What are two tests of normality?
Kolmogorov-Smirnov test and Shapiro-Wilk test
What do Kolmogorov-Smirnov test and Shapiro-Wilk test do?
compare the shape of the sample distribution to the shape of a normally distributed curve:
Which test of normality is better for a large sample size?
Kolmogorov-Smirnov test
Which test of normality is better for a small sample size?
Shapiro-Wilk test
What does a non-significant (p>0.05) test suggest in a test of normality?
Suggests that the distribution of the sample is not significantly different from a normal distribution- NORMAL
What does a significant (p≤0.05) test suggest in a test for normality?
That the distribution in question is significantly different from a normal distribution- NOT normal
When is a test of normality particularly important?
Normality test important especially in small samples
What is multicollinearity?
- In regression analysis, “multicollinearity” refers to IVs that are correlated with other IVs.
- In presence of multi-collinearity, regression models may not give valid estimates of the individual predictors.
What is Variance inflation factor (VIF)?
Variance inflation factor (VIF) is a measure of how much the variance of the estimated regression coefficient is “inflated” by the existence of correlation among the IVs in the model.
What are the VIF cut off values?
VIF = 1 No correlation among the predictors
• VIF > 4 warrants further investigation
• VIF > 10 are signs of serious multicollinearity
When is multicollinearity more of a problem?
Large correlation - more significant problem with multi-collinearity
Small samples are vulnerable to multi-collinearity
How would evidence of homoscedasticity look on a scatter plot?
If a plot of the fitted values against the residuals scattered randomly around zero, this is an evidence of homoscedasticity
How else can you test homoscedasticity?
In addition, we can use statistical tests to examine constant variance (homoscedasticity) assumption
What does an insignificant p-value [p>0.05] of the homoscedasticity test tell us?
The constant variance assumption is supported
When would you transform your data?
If the data do not satisfy the assumption of normality, they can be transformed to make them resemble normal data
Does transformation of the data work better for large or small samples?
Transformation works better for large samples
What do you do if the data cannot be transformed?
If the outcome data are not suitable for any transformation to make them normal and/or sample is small, alternative non-parametric options to explore relationships include:
- Spearman rank-correlation coefficients
- Quantile regression
What is the B value in Linear regression?
for each unit increase in the independent variable score, how much unit increase would there be in the dependent variable
How do you check the assumption of linearity for multiple linear regression?
Lack of patterns in the scatterplots of standardised residuals support the linearity assumption
How do you check the assumption of constant variance for multiple linear regression?
- Plot of the residuals against the predicted values does not show any particular pattern = supports constant variance assumption
- Insignificant p-values of Breusch-Pagan test (p=0.25) and Koenker test (p=0.29) support the constant variance or homoscedasticity assumption
What is the R squared value?
R2 = 0.36 suggests that 36% of variability in the dependent variable is explained by the fitted model
What are two methods you can use to transform data (to make normal)?
A power transformation
A cubic transformation
What level study is a cohort study?
III-2
What are five examples of observational studies?
- Cohort
- Case-control
- Cross sectional
- Prevalence
- Case report
What are three components of observational studies?
- Subjects are observed in their natural state.
- The groups of subjects that are compared are self-selected e.g., manual workers versus non-manual workers or subjects with and without disease
- Subjects may be measured and tested (e.g., disease status ascertained) but there is no intervention or treatment (e.g. patients not allocated to different exercise programs, or to new drug or placebo).
What is a cohort longitudinal study?
• A population of subjects is identified by a common link (e.g., living in the same geographical area, working in the same environment, attending the same clinic, diagnosed with the same condition/disease e.g., brain injury).Cohort studies consider factors not under the control of the researcher
• Cohort studies often follow a cohort (people with a shared
characteristic) over time and can provide information about long term outcomes. They can also provide predictive information.
What are three ways that you can conduct cohort longitudinal studies?
- The researcher can follow them across time to see what happens to them (e.g., following people with moderate brain injury over time to see what the range of functional outcomes are). This is useful for establishing the natural history of a condition.
- The cohort can be divided at the outset into subgroups of people whose experience is to be compared (e.g., people who have had a coma with brain injury vs those not having a coma ). They are followed over time and the incidence of outcomes of interest (e.g., return to work) is compared between groups. This is helpful for considering possible causative factors or for establishing predictive factors.
- Or the cohort may be followed over a set period of time or until the event of interest occurs. The characteristic of those who have the event of interest with those who do not are then compared. This helps identify those most likely to develop the outcome
What are two examples of samples in a cohort study?
- Select group (e.g. occupational or professional group)
- Exposure group (Person having exposure to some physical, chemical or biological agent.)When starting out, groups should be free of disease, groups should be equally susceptible to disease and groups should be otherwise
comparable
What are some ways of obtaining data on exposure?
- Personal interviews / mailed questionnaire • Reviews of records
- Dose of drug, radiation, type of surgery etc • Medical examination or special test
- Blood pressure, serum cholesterol • Environmental survey
What are the different types of comparison groups in an exposure cohort study?
• Internal comparison
(Only one cohort involved in study
Sub classified and internal comparison done)
• External comparison
(More than one cohort in the study for the purpose of
comparison e.g. Cohort of radiologist compared with ophthalmologists)
• Comparison with general population rates
( If no comparison group is available we can compare the
rates of study cohort with general population. E.g., Cancer rate of uranium miners with cancer in general population)
How can you obtain follow up data in a cohort study?
- Mailed questionnaire, telephone calls, personal interviews
- Periodic medical examination
- Reviewing records
- Surveillance of death records
- Follow up is the most critical part of the study
- Some loss to follow up is inevitable due to death change of address, migration, change of occupation etc.
What is a major drawback of cohort studies?
Loss to follow-up is one of the draw-backs of cohort studies.
What is the aim of data analysis in cohort studies?
- Calculation of incidence rates among exposed and non exposed groups
- Estimation of risk
What are the strengths of cohort studies?
- We can find out incidence rate and risk
- More than one disease related to single exposure
- can establish cause - effect
- good when exposure is rare
- minimizes selection and information bias
What are the weaknesses of cohort studies?
• losses to follow-up • often requires large sample • ineffective for rare diseases • long time to complete • expensive • Ethical issues
What is internal validity?
Internal validity :
- Ability to reduce confounding factors
- so no other variables, except the one you are studying, caused the results
What are the three main focus points of a study question?
- The population studied
- The outcomes considered
- The prognostic factors/predictors of interest
What is an inception cohort?
Inception cohort = designated group of people assembled at a common
point in time early in the development of the disorder
How do you assess for bias in the recruitment of subjects to a cohort study?
Was there an inception cohort?
Was the cohort representative of a defined population? Was everybody included who should have been included?
What are some ways to assess for bias in terms of measurement in a cohort study?
• Did they use subjective or objective measurements?
• Do the measurements truly reflect what you want them to (have they been validated)?
• Were all the subjects classified into exposure/condition groups using the same procedure?
• Were the measurement methods similar in the different groups?
• Were the subjects and/or the outcome assessor blinded
to exposure/condition (does this matter)?
How would you assess power of a cohort study?
- Did the authors present power analysis/sample size calculation with adequate information?
- Was the sample size calculation based on pilot data or assumptions?
NOTE: cohort studies generally need large sample sizes
How would you assess for confounding factors?
Look for restriction in design, matching, and techniques e.g. modelling, stratified analysis, or sensitivity analysis to correct, control or adjust for confounding factors.
What is continuous vs. dichotomous data vs polychotomous data?
continuous = data that can take any value in a range. Dichotomous = two options e.g. yes/no polychotomous = >2 options
What to consider for continuous data in cohort study review?
What is the expected value (e.g. mean) at the time point of interest?
• Have data for prognostic factors/possible predictors been provided?
• Are the results statistically significant? How strong is the association
(e.g., effect size)?
What to consider for dichotomous data in cohort study review?
• What is the expected rate/proportion of the outcome between those
exposed/unexposed, (or the ratio/ absolute rate difference) at the time
point of interest?
• Have data for prognostic factors/possible predictors been provided?
• Are the results statistically significant? How strong is the association
(e.g., effect size)?
How to measure precision in data for cohort study?
Look for the range of the confidence intervals, if given. Were the confidence intervals for the effect large or small?
How do you assess whether you believe the results of a cohort study?
a. Could the result be due to bias, chance or confounding? b. Are the design and methods of this study sufficiently
flawed to make the results unreliable?
c. Are the author’s conclusions supported by the study’s
findings?
d. If the authors try to establish causality, did the they
consider temporality, dose-response, biological plausibility, consistency?
Which test can be used when you want to see if there is a significant relationship between two categorical variables?
Chi-square
What is the expected frequency per cell assumption for chi squared?
assumption: expected frequency in each cell ≥ 5
What test would you use if you wanted to conduct a Chi-square test, but one or more of your cells has an expected frequency less than 5.
Fisher’s exact test
Which test would you use when you want to compare the means of a normally distributed continuous variable for two groups?
An independent samples t-test
Which test would you use when you want to compare the means of a normally distributed continuous variable for more than two groups?
ANOVA
What is an odds ratio?
An odds ratio (OR) is a measure of strength of association between an exposure and an outcome.
The OR represents the odds that an outcome will occur given a particular exposure, compared to the odds of the outcome occurring in the absence of that exposure.
What are the odds ratio measures?
- OR=1 - exposure does not affect odds of outcome
- OR>1 - exposure is associated with higher odds of outcome
- OR<1 - exposure is associated with lower odds of outcome
e.g. OR=1.3 for association between gallstone disease and women suggests that the odds of women having a gallstone disease was about 30% more than that in men.
What are the rules for the Confidence interval of odds ratio?
- If the confidence interval of an OR contains 1 (the null value of OR), the relationship is likely to be insignificant
- If the confidence interval of an OR does not contain 1 (the null value of OR), the relationship is likely to be significant
What is the limitation of test that examine association between two variables using statistical tests and measures (e.g., Chi-square tests, t-tests, odds ratios)?
These tests can not adjust the associations for the potential effects of other covariates.
How do you overcome the limitation of tests that can not adjust the associations for the potential effects of other covariates.
Logistic Regression
What is Logistic Regression?
- A regression with an outcome variable that is categorical (e.g. success/failure) and independent variables that can be a mix of continuous and/or categorical variables
- Addresses the same research questions that multiple regression does but with no distributional assumptions
- Predicts which of the possible events are going to happen given certain other information on independent variables
- Identifies factors that determine whether an individual is likely to benefit from a certain type of rehab program
What are the assumptions of logistic regression?
• Ratio of cases to variables – using categorical variables requires that there are enough responses in every given category
- Linearity in tIf there are too many cells with no responses, parameter estimates and standard errors may be extremely large
- This can also make maximum likelihood estimation (MLE) of parameters of the model impossible
• Linearity in th logit – the regression equation should have a linear relationship with the logit form of the outcome
Logit(p)=log[p/(1-p)] where p = sample proportion of event of interest
• Absence of multicollinearity and outliers
• Independence of residuals
What are three types of logistic regression models?
- Binary logistic regression (dichotomous outcome)
- Multinomial logistic regression (Polychotomous outcome)
- Ordinal logistic regression (ordered outcome)
In Logistic regression, a researcher may want to predict an outcome which has more than two response options; e.g. discharge destination: (a) home, (b) nursing home,(c) hospital
How would you manage this?
Multinomial Logistic Regression
The researcher needs to choose one category as “reference” to compare with the others
• Considering “home” as a reference, the following comparisons can be made:
• Nursing home vs. Home
• Hospital vs. Home
What is Multinomial Logistic Regression?
• In multinomial LR, you are essentially building two binary LR models with additional multinomial constraints:
- Model 1 will predict the chance of going to ‘Nursing home’ compared to going ‘Home’
- Model 2 will predict the chance of going to ‘Hospital’ compared to going ‘Home’
• The interpretation would be as if you are interpreting two binary logistic regression models
What is Ordinal Logistic Regression?
- Ordinal logistic regression is used to model for ordinal outcome variables (e.g. physical activity level of stroke patients: very low, low, medium, high)
- As a regression analysis, ordinal regression explains the relationship between one ordinal dependent variable and a set of independent variables, which could be combination of categorical and/or continuous variables
What is the formula for OR?
OR =a x d/ b x c
What does it mean if the OR confidence interval includes 1?
Insignificant
What does it mean if the B value CI includes zero?
Insignificant
What is the Omnibus test in logistic regression?
The Omnibus Tests of Model Coefficients is used to check that the new model (with explanatory variables included) is an improvement over the baseline model.
How does the Omnibus test measure whether the new model is an improvement over the baseline?
It uses chi-square tests to see if there is a significant difference between the Log-likelihoods (specifically the -2LLs) of the baseline model and the new model. If the new model has a significantly reduced -2LL compared to the baseline then it suggests that the new model is explaining more of the variance in the outcome and is an improvement! If Chi squared is significant, the new model is an improvement over baseline!
What is the “model” part of omnibus test?
Model = Compares new model to baseline
What are Block and Step components of omnibus tests?
Step and Block = only useful if we are adding the
explanatory variables to the model in a stepwise or hierarchical manner
What is Pseudo-R2 ?
how much variation in outcome
is explained by model . Must be used with caution!
What tests are preferred to Pseudo- R2?
Nagelkerke R2 value, which is a modification of Cox & Snell R2.
What does Exp (B) mean?
Odds Ratio
What is the case wise list?
Cases with studentized residuals greater than 2.580 (which represent the extreme 1% of distribution)
What is the value cut off for the extreme 5% of distribution?
2
What do the list of extreme cases/casewise list tell us?
Presence of influential cases, which require further examination.
Which test can be used for Goodness of fit in logistic regression?
Hosmer-Lemeshow goodness of fit test can be used to examine whether the estimated LR model fits the sample data
What does a p-value ≤ 0.05 in Hosmer-Lemeshow goodness of fit test suggest?
A poor fit
What does a p-vaue of > 0.05 suggest in Hosmer-Lemeshow goodness of fit test?
A good fit
What does an OR of 4.17 mean?
e.g. the odds of having a diagnosis of dysphagia would be increased by
a factor of 4.17 for patients with head and neck burn, compared to those without head and neck burn, controlling for other variables in the model (p=0.003)
What is a classification table?
Looks at predictive accuracy of the fitted model.
How do you calculate predictive sensitivity?
True yes cases (observed and predicted) divided by Total ROW
How do you calculate predictive specificity?
True no cases (observed and predicted) divided by total ROW
How do you calculate false positives in predictive classification?
Predicted yes’s divided by column TOTAL
How do you calculate false negatives?
Predicted no’s divided by total column.
What does a ROC curve do?
Can be used to measure predictive accuracy of the fitted model
What does the “area” value represent in area under the curve?
% of predictive accuracy
What does area of 0.5 mean?
No predictive power
What does area of 1 mean?
Perfect predictive power
What is logistic regression
Logistic regression is the appropriate regression analysis to conduct when the dependent variable is dichotomous (binary). Like all regression analyses, the logistic regression is a predictive analysis. The independent variable can be numerous or categorical
Logistic regression can be
1. Binary logistic regression (dichotomous outcome)
2. Multinomial logistic regression (Polychotomous outcome)
3. Ordinal logistic regression (ordered outcome)
What is the difference between simple linear regression and multiple linear regression?
Simple linear regression : a single independent variable is used to predict the value of a dependent variable. Multiple linear regression : two or more independent variables are used to predict the value of a dependent variable.
What would you consider when asking if the study addressed a clearly focussed issue?
- The population studied
- The outcomes considered
- The prognostic factors/predictors of interest
When appraising a cohort study, what would you consider when asking Was the cohort recruited in an acceptable way?
- Was there an inception cohort?
- Was the cohort representative of a defined population?
- Was everybody included who should have been included?
When appraising a cohort study, what would you consider when asking Was the exposure/condition accurately measured to minimise bias?
- HINT: Look for measurement or classification bias:
- Did they use subjective or objective measurements?
- Do the measurements truly reflect what you want them to (have they been validated)?
- Were all the subjects classified into exposure/condition groups using the same procedure?
When appraising a cohort study, what would you consider when asking Were the outcomes accurately measured to minimise bias?
a. Are the methods of measurement suitable for determination of
the target variable?
• Did they use subjective or objective measurements?
• Were the measurement methods similar in the different
groups?
• Were the subjects and/or the outcome assessor blinded to exposure/condition (does this matter)
When appraising a cohort study, what would you consider when asking Was the follow up of participants complete enough??
• How many were lost to follow-up?
• If the persons that were lost to follow-up may have different
outcomes than those available for assessment? N/A
•Was there anything special/different about the outcome of
the people leaving the cohort?
When appraising a cohort study, what would you consider when asking Was the study adequately powered and was the sample size
included in the analysis adequate???
Did the authors present power analysis/sample size calculation with adequate information?
•Was the sample size calculation based on pilot data or assumptions?
When appraising a cohort study, what would you consider when asking about Missing data?
a. Was the percentage of missing values given?
b. How the missing values were treated in the analysis?
c. Was the number of missing values too large to permit
meaningful analysis?
When appraising a cohort study, what would you consider when asking about confounding factors?
Look for restriction in design, matching, and techniques e.g. modelling, stratified analysis, or sensitivity analysis to correct, control or adjust for confounding factors.
When appraising a cohort study, what would you consider when asking How precise are the results?
Look for the range of the confidence intervals, if given.
Were the confidence intervals for the effect large or small? Sensitivity and specificity + confidence intervals were provided throughout each stage of the analysis.
When appraising a cohort study, what would you consider when asking Do you believe the results?
a. Could the result be due to bias, chance or confounding?
b. Are the design and methods of this study sufficiently flawed to make the results unreliable?
c. Are the author’s conclusions supported by the study’s findings?
d. If the authors try to establish causality, did the they consider temporality, dose-response, biological plausibility, consistency?
When appraising a cohort study, what would you consider when asking can the results be applied to the local population?
• Are the subjects covered in this study sufficiently different
from your population to cause concern?
• How much does your local setting likely to differ much from that of the study?
• How generalizable are the study results?
When appraising a cross sectional study, what would you consider when asking Were the measures accurately measured to reduce bias?
- Did the authors use subjective or objective measurements?
* Do the measures truly reflect what they were meant to measure (have they been validated)?
When appraising a cross sectional study, what would you consider when asking Were the data collected in a way that addressed the research aims?
- if the setting for data collection was justified
- if it is clear how data were collected (e.g., interview, questionnaire, chart review)
- if the author has justified the methods chosen
- if the author has made the methods explicit (e.g. for interview method, is there an indication of how interviews were conducted?)
When appraising a cross sectional study, what would you consider when asking Did the study have enough participants to minimize the play of chance?
- if there is a power calculation to estimate how many subjects are needed to produce a reliable estimate of the measure(s) of interest.
- Did the authors present power analysis/sample size calculation with adequate information?
- Was the sample size calculation based on pilot data or assumptions?
When appraising a cross sectional study, what would you consider when asking Is there a clear statement of findings?
• if the findings are explicit • if there is adequate discussion of the evidence both for and against the authors’ arguments • if the author have discussed the credibility of their findings • if the findings are discussed in relation to the original research questions
What level of the NHMRC heirarchy are RCT?
II
What type of studies are randomised control trial?
- Individuals collected at random to receive one of a number of interventions
- All participants have equal chance of allocation to each intervention
- Experimental
- Often answers question as to whether an intervention is effective compared to something else
What are the three different designs of RCT?
- Historical controls
- Non randomised concurrent control
- Quasi randomised design
What is Historical controls RCT?
(compare results of new treatment on new patients - prospectively- with records of previous results of a historical group who received standard treatment)
What are the limitations of Historical controls RCT?
Change in patient population over time (?healthier or less healthy) * Change in diagnostic criteria over time
- Changes in international coding systems
- Overall standards of patient management improve over time * Differences in data quality between groups
What are non-randomised concurrent control RCTS?
There are two groups each one receiving a different treatment roughly at the same time. Less costly option to RCTS, relatively simple
What are the limitations of non-randomised concurrent control RCTS?
- we could end up with groups that are not comparable * ? Subconscious bias in allocation to either group
- no control of potential confounding factors
What are quasi randomised design RCTs?
Allocation to intervention is not truly random. For example alternative allocation, allocation by order of enrolment etc, day of enrolment
What are the limitations of quasi randomised design RCTs?
- subject to contamination by confounding variables * without proper randomisation, statistical tests
can be meaningless as unexpected factors might affect results
Why is random allocation important?
• Gives “gold standard” evidence
• Eliminates bias in treatment assignment
• Covariates are equally distributed across groups at baseline – unbiased distribution of confounders
• Experimental and control groups are treated exactly the same, except for the intervention received
• Facilitates blinding of treatments from investigators, participants, assessors
• Allows researchers to make causal inferences
- Randomisation ensures any differences between groups is
due to chance
What are the disadvantages of random allocation?
Expensive: time and money Ethical considerations Hawthorne effect
What are the three different types of RCT?
- Parallel Trials
- Crossover RCT
- Factorial RCT
What is parallel RCT?
Sample selected from population , baseline variables measured, participants randomised, intervention applied, outcomes measured.
What is crossover RCT?
Sample selected from population, baseline variables measured, participants randomised, interventions applied, outcomes measured, washout period, intervention applied to previous control group, outcomes measured (placebo and treatment group swap after washout period)
What is Factorial RCT?
Sample selected from population, baseline variables measured, 2 active interventions and 2 controls assigned randomly, intervention applied, outcomes measured.
What are the sources of bias for RCT?
Inadequate randomisation sequence, inadequate concealment of allocation, inadequate blinding, exclusion of participants, significant attrition, participants from wrong group analysed, selective reporting of outcomes
What type of bias is reduced via randomisation?
Selection bias and performance bias
What are some ethical considerations for RCT?
- Social and clinical value
- Scientific validity
- Fair subject selection
- Favourable risk-benefit ratio • Independent review
- Informed consent
- Respect for potential and enrolled subjects
What are the CASP questions for RCT?
- Did the study address a clearly focused issue?
- Was the method used appropriate to answer the clinical question?
- Was assignment of participants to experimental and control groups randomised
- Were groups similar at start of trial?
- Was the follow-up of participant complete enough?
- With the exception of the experimental intervention, were groups treated equally?
- Was the study adequately powered?
- Missing data
- Method of measurement
- Are important parameters considered and adjusted for in analysis and presented appropriately?
- Have correct statistical methods been selected and are clearly described with rationale?
- How large was the treatment effect?
- Are the results believable?
- Can results be applied in your context
- Were all clinically important outcomes considered?
- Are benefits worth the harms/costs?
- How might evidence inform practice or guide future research?
What is H0?
Null hypothesis
What is H1?
One-sided alternative hypothesis
This is important as it determines which p-value to look at when interpreting results and has implications for sample size computation
What is H2A?
alternalitve hypothesis
How are sample size and effect size associated?
Sample size is inversely associated with effect size. e.g. Effect size = 0.5 so n= 102
Effect size= 0.2 so n = 620
What is an intention to treat analysis (ITT)?
compares treatment groups as originally allocated (random), irrespective of whether patients received or adhered to treatment protocol
How does ITT promote external validity?
it is a pragmatic approach that aims to evaluate the effectiveness of an intervention in routine practice
What is per protocol analysis?
compares treatment groups as originally allocated but includes only those patients who completed the treatment protocol, which compromises the internal validity of the findings.
What is GEE?
generalized estimating equations
• Generalized Estimating Equations (GEE) is an extension of the general linear model (GLM) of statistical regression for modelling clustered or correlated data
• GEE offers robust estimates of standard errors to allow for clustering of observations
• GEE produces consistent estimates of regression coefficients and their standard errors
• GEE can deal with both normal and non-normal outcome data
• GEE is useful when the aim is to investigate differences in population averaged responses
What is RMANOVA?
- Repeated Measures Analysis of Variance (RMANOVA)
- Complete case analysis (so only those who completed the trial are included)
- Assume everyone is measured at the same time and at equally spaced time intervals
- Require restrictive assumptions about the correlation structure
- Does not provide parameter estimates (just p-values)
- Can not handle time-dependent covariates (predictors measured over time)
What is LDA?
Longitudinal Data analysis
What does LDA do?
LDA helps us to …
assess changes in a response variable over time
measure temporal patterns of response to treatment identify factors that influence changes
include time-varying predictors in the model investigate causality
better handling of missing data (all available data)
What are the two types of LDA?
Mixed effects models and Marginal models
What are mixed effects models?
e.g. linear mixed models
• to compare individual changes over time (trajectories)
• to study natural history
What are marginal models?
e.g. GEEs
• to compare populations over time
• to evaluate interventions or inform public policy
Why do you use Mixed effects models and marginal models?
Because In repeated measures, the assumption of independence of observations is likely to be violated
How can you measure intra rater reliability
Intraclass correlation coefficient
What does a correlation coefficient of >0.8 represent
GREAT
In parametric testing, when is it inappropriate to use mean and SD?
When data is not normal. Need to use Median and Interquartile range instead
What does a significant GEE result tell you?
Whatever is not reference is significantly better than reference
Randomised controlled trials
• Individuals are allocated at random to receive one of a
number of interventions (minimum 2)
• All participants have equal chance of allocation to each intervention
• NOT determined by researcher • NOT predictable
• Experimental
• Often comparative studies
to answer the question as to whether an intervention is
effective compared to something else
What is selection bias?
If participants or researchers can predict which treatment participant is receiving
What is performance bias?
if the attention that researchers provided to intervention and control group is not equal
What is detection bias?
if outcome assessors know what treatment the participant has received
What is Attrition bias?
if there are systematic differences between groups in withdrawals and incomplete data
What is Reporting bias?
if there are systematic differences between reported and unreported findings
What are ethical considerations in RCT?
- Social and clinical value
- Scientific validity
- Fair subject selection
- Favourable risk-benefit ratio • Independent review
- Informed consent
- Respect for potential and enrolled subjects
What are the assumptions of GEE?
In addition to meeting the assumptions of general linear model (e.g. multiple regression), GEEs require …
• the responses are from a known family of distribution with specified mean and variance, where variance is a function of the mean
• The mean is a linear function of the predictors
• A correlation structure for the responses must be specified, meaning that a working guess of the correlation structure is required.
• Any missing data are either missing completely at random or that the data are missing at random
What are the choices of structure in GEEs?
Choices of correlation structure include: Exchangeable Autoregressive Unstructured Independent
