Lecture 9: Randomized Controlled Trials and Statistical Power

Question 1

What are RCTs?

Answer 1

• experimental investigation that randomly assigns participants to an intervention or control group
• ensures a balanced distribution of confounders between study groups
• able to determine causality rather than only association

Question 2

What are the aspects of RCT that are difficult to achieve in nutrition research?

Answer 2

• blinding

- placebo-control

Question 3

RCTs vs observational studies in terms of randomization

Answer 3

RCTs is able to determine causality because there is an actual intervention and you are holding all of the other factors constant if the randomization is done appropriately. It is different from an observational study bc in an observational study you are not doing anything to your participants, you observe relationships, but you cannot comment on causality. They might be related but you cannot say one caused another. The issue of knowing directionality, what caused the outcome.

Question 4

CONSORT

Answer 4

Consolidated Standards of Reporting Trials

Question 5

What do you have report when you are registering your RCT?

Answer 5

Protocol
Outcome measures of interest
- Primary outcome
- Secondary outcome

Question 6

Why report trials before hand?

Answer 6

• transparency
• track changes
• prevent fishing expeditions where you look at lots of variables and measurements trying to find something interesting.
• keep scientific integrity more robust

Question 7

What are the commonly used randomization methods?

Answer 7

simple and stratified

Question 8

Explain simple randomization

Answer 8

each participant is randomly assigned to treatment or control group

Question 9

Explain stratified randomization

Answer 9

participants are first placed into strata, in order to control for a particular variable, and then participants within strata are randomly assigned to treatment or control group.

Question 10

What are the potential risks for bias in RCTs

Answer 10

• loss to follow-up (attrition): when participants withdraw from the study or are unreachable
• non compliance: when participants in intervention group do not comply with the intervention (or when participants in control group adopt the intervention)
• these can lead to missing data

Question 11

True/False

RCTs eliminate confounders and risk of biases.

Answer 11

False

RCTs eliminate the potential for confounders but they don’t eliminate the risk of bias. Just because you randomize your participants, it doesn’t eliminate this risk of error getting into your analysis, it eliminates confounding.

Question 12

What is bias?

Answer 12

Bias is a systematic or random error that will skew the validity of your results.

Question 13

Why is lost to follow up is a problem?

Answer 13

• missing data
• if you have disproportionate numbers of participants who finished your intervention than your control then there could be something about those participants who didn’t finish the intervention that would have been important to have their data for.
• journal and peer reviewers pay attention to these numbers.

Question 14

What are the ways of dealing with missing data in RCTs?

Answer 14

• per protocol analysis
• as treated analysis
• intention-to-treat analysis

Question 15

Explain the per protocol analysis

Answer 15

includes only participants who have completed the originally allocated treatment (i.e. removes drop-outs and non-compliant participants)
- sample size reduces.

Question 16

Explain the as treated analysis

Answer 16

groups participants according to the treatment they actually followed, rather than what they were intended to follow (i.e. attempts to mitigate non-compliance)

Question 17

Explain the intention-to-treat analysis

Answer 17

includes all participants in the groups to which they were randomly assigned, regardless of compliance or study completion (“once randomized, always analyzed”)

Question 18

How to manage missing data in ITT?

Answer 18

• last value carried forward
  (use the last available measured value in the final analysis)
• imputation
  (use statistical modelling to estimate what the missing value(s) would be if the participants completed the study.)

Question 19

What is type 1 error?

Answer 19

alpha
it is rejecting a true null hypothesis (false positive)
you say you have found statistically significant results when you actually have not.

Question 20

What is type 2 error?

Answer 20

Beta
is when you accept a false null hypothesis means that you have incorrectly generated results that are not statistically significant.

Question 21

What is statistical power?

Answer 21

statistical power is the likelihood that a study will detect an effect when there is an effect there to be detected. If statistical power is high, the probability of making a Type II error, or concluding there is no effect when, in fact, there is one, goes down.

Question 22

Why to calculate sample size?

Answer 22

• to know what is the minimum sample size needed to detect a statistically significant result?
• done a priori

Question 23

Why to calculate statistical power?

Answer 23

• given a specified sample size and effect size, what is the statistical power to detect a statistically significant result?
• done after the study

Question 24

Why is power calculation done after the study?

Answer 24

for examples NHANES, CCHS are collecting data constantly and if you wanted to do a study on that then your sample size is set for you so you cannot control that. So, you run a power calculation to see if you will have enough power to detect a significant result. You need to know your effect size.

Question 25

What information do we need to know to calculate sample size?

Answer 25

• the statistical test (study design, variable types)
• p-value that will be used as significance cut-off
• power level of analysis (generally set at 0.8)
• measure of effect size (how large the outcome of interest is?)

Question 26

How to obtain effect size?

Answer 26

Can be estimated by pilot data or previous research. If these are not available, must make an educated guess based on knowledge of the discipline.

Question 27

What are the categories of effect sizes?

Answer 27

small - 0.2
medium - 0.5
large effect - 0.8

Question 28

Let’s say you have a really significant p value , but your correlation r value is really small, what does that mean to have really small correlation that is statistically significant?

Answer 28

if you have large sample size, you are going to have high statistical power and you might pick up statistically significant results.

Just because something is statistically significant doesn’t mean that it has practical relevance.

Question 29

What is the difference between statistical significance and clinical significance?

Answer 29

You might find results that are statistically significant but might not have any relevance practically.

Question 30

How to guess effect size based on previous data?

Answer 30

look at meta analyses or individual studies
ex in class: the meta analysis had small to medium effect and individual study had a large effect. it is better to be conservative so this topic has small to medium effect size.

Question 31

What were the inputs of the sample size calculation that we did in class?

Answer 31

• independent two-tailed t-test
• effect size d
• alpha level
• power level
• allocation ratio

Question 32

What were the inputs of the power calculation that we did in class?

Answer 32

• ANOVA, repeated measures, between factors
• post hoc power analysis
• effect size cohen’s f
• alpha level
• total sample size
• number of groups
• number of measurement
• correlation among repeated measures 0.5 (default value)

Question 33

What are the type of variables for ANCOVA?

Answer 33

main predictor is categorical, outcome is continuous

Question 34

What are the type of variables for binary logistic regression?

Answer 34

outcome variable is categorical and dichotomous

Question 35

What are the type of variables for linear regression?

Answer 35

can use both continuous and categorical variables but the outcome variable is always continuous

Question 36

How to use categorical variables in regression analyses?

Answer 36

in order to utilize categorical variables in regression analyses, the variables must be dichotomous (dummy variable: typically coded 1 and 0)

Question 37

What are dummy variables?

Answer 37

method of dichotomizing variables

Question 38

How many dummy variables are needed for a group 5?

Answer 38

4

it is always number of groups -1
the base case is always 0, 0, …, 0
rest is 1 and 0

Question 39

When do you not have to dummy code in SPSS?

Answer 39

the dummy coding of binary logistic regression is done automatically for categorical variables that have >2 categories as long as the variable is specified as being categorical in the set-up of the model

Question 40

What is odds ratio?

Answer 40

• Compares the presence of an exposure to the absence of an exposure, given that we already know about the outcome.
• Appropriate for case-control studies, cross-sectional studies, prospective cohort studies that measure prevalence

Question 41

What is the equation for odds ratio?

Answer 41

OR = odds that an exposed has outcome/odds that an unexposed has outcome

(a/b)/(c/d) = (ad)/(bc)

Question 42

When is it appropriate to report odds ratio?

Answer 42

appropriate for case-control studies, cross-sectional studies, prospective cohort studies that measure prevalence

Question 43

When do you get an odds ratio?

Answer 43

When you do logistic regression, you get an odds ratio rather than a beta value. Odds ratio of 1.25 which is interpreted as 25%

1 = null value meaning there is no difference between groups.

Question 44

What are the two approaches for regression modelling for covariate assessment?

Answer 44

1. some only include significant independent (predictor) variables in the final model. Argue that only these covariates are relevant to the relationship between the main predictor variable and outcome.
2. others decide on covariates a priori (based on scientific literature) and include pre-specified covariates in the model regardless of whether or not they are significant predictors. Argue that this limits the number of hypotheses to be tested, provides findings that are more likely to be reproducible, and keeps the interpretation of results consistent across a set of multiple dependent variables (when this is the situation)