Research Design Flashcards
Why is research design important?
To provide a framework of methods and techniques
Enables research to be conducted in a structured way
Research design tells us “how” a study is conducted.
Descriptive Research: “What is going on?”
Experimental (or mechanistic) Research: “Why is it going on?”
Observational vs. Experimental
Observational studies:
there is no intervention
Provides information between an ‘exposure’
and an ‘event’ or ‘characteristics’ of
the population
Experimental studies:
involves an intervention:
Allows for the determination of
cause and effect if the study is
designed appropriately
Cross sectional
Cross-Sectional Longitudinal
Assesses a phenomenon at one point in
time
Measures different samples/populations
(or only one sample/population ONCE)
Provides a snapshot of a given point in
time
Examines trends and changes at a
societal/national/international level
E.G. health survey for England
Longitudinal
assess a phenomenon at several points in time
measures the same sample/ population at several points in time
provides information on change at the individual level
examines changes in the same people over time
E.G. lothian birth cohort study
Cross-sectional design: single group
One sample of participants recruited from target population
Information is collected from these people once
Type of research question:
– What are the associations between energy/nutrient
intakes and frailty in older
migrant women in the UK?
Cross-sectional design: multiple groups
Different groups of defined participants are recruited
Information is collected from these people once
Type of research question:
– Are athletes different from non-athletes? Is one patient group different from
another?
Retrospective design
Backward looking: examine data that already exist
Tries to identify factors that predict whether something will happen
(e.g., disease, sports performance)
Type of research question:
– Does meeting the UK physical activity recommendations over a
lifetime have a protective effect against falling in older people?
Prospective design
Forward looking: collects new data, then sometimes, watch/wait
Waits for outcomes (e.g., development of disease, or sporting
performance) and relates this to suspected influencing factors
Type of research question:
What is the relationship between premature mortality and red
meat consumption?
Case control study
Usually retrospective (but not exclusively)
“Cases” have the outcome (e.g., heart disease)
“Controls” do not have the outcome
Type of research question:
What is the association between allotment gardening and
mental well-being?
Randomised controlled trial
Forward looking (prospective)
Participants are randomized into a “control” or “intervention” group
Follows groups over time to determine a difference in outcomes
Type of research question:
– What is the effect of exercise on cognitive impairment in older adults
with mild to moderate dementia?
Time
Past- Retrospective cohort study
Case control study
present-
cross-sectional study
Retrospective cohort study
Case control study
Quasi-experimental design
An intervention study that DOES NOT:
– Randomise participants AND/OR
– Have a control group
Example: Testing the impact of a physical activity intervention on risk for
type 2 diabetes in South Asian adults
– Only one group (e.g., an exercise intervention)
– Without a control group we cannot say with confidence that the
intervention is/is not effective (it could be due to time/season
Measurement phase 1-
time-
(intervention group (exercise intervention or drug)
Measurement phase 2
Feasibility and pilot studies
Feasibility studies:
– Research done before a main study to determine whether the methods are
feasible and acceptable
– “Can this study be done? Can we recruit patients?”
Pilot studies:
– A small-scale study conducted prior to a large-scale experiment to test and
refine procedures
– “Does initial data suggest our method/intervention could work?”
Both are used to inform the design and implementation of large, definitive
randomized controlled trials
Is there an ideal study design?
Each design has a purpose, strengths and weaknesses
Some designs may have more “bias” than others
Other designs are less generalizable to all populations
Important to avoid drawing conclusions that are not consistent with the study
design used
– For example, you can’t make claims of cause and effect from a study
assessing associations
Scientific considerations and generalisability of research
title
Conflict of interest:
A situation in which a person is (or persons are) in a position to
derive personal benefit and unfair advantage from actions or
decisions made in their official capacity
researcher bias
Any factor—such as investment in the product being studied or
gifts from the product manufacturer—that might influence the
researcher to favour certain results
Why is conflict of interest a problem?
industry may influence the design, analyses, and results of studies for financial gain
(e.g., due to increased sales of the product due to favourable findings)
It can compromise the ability of researchers to conduct research and analyse and
report the results in an accurate and impartial manner
Does this mean researchers and industry can never work together?
No: lots of high-quality research is funded by industry in partnership with
academics and conducted in an impartial manner.
Researchers are required to be transparent about funding sources and any
potential conflicts of interest.
Researchers reveal potential conflicts of interest that could affect the integrity of
their research when they submit their results for publication in scientific journals.
Types of bias in research
heading
researcher bias
definition-Any factor (e.g., gifts from the product
manufacturer) that might influence the
researcher to favour certain results
impact on report outcome- Study may be designed so that increases the chance
of finding favorable results. Outcomes are not
accurate or representative of the data.
selection bias
definition- individuals or groups recruited, or the data used in analyses, are selected in a way
that is not random.
impact on report outcome-study participants do not represent the population limiting generalizability. Selecting/excluding data
could change the direction/magnitude of the results
self report bias
definition- error introduced with self-report data
impact on report outcome- Prevents accurate assessment of relationships between variables
recall bias
definition-differences in the ability of participants to accurately recall the variable being
measured (e.g., physical activity level).
impact on report outcome- Erroneous (incorrect) conclusions made; data from one group could be more accurate than another; very relevant to case-control studies
reporting bias-
definition-differences between reported and unreported journals
impact on report outcome- Selective reporting of positive, favorable outcomes. Results do not represent the full set of data collected
publication bias-
definition- tendency for journals to publish studies with positive results
impact on reported outcome- lack of transparency. Not reporting negative findings
leads to ineffective or harmful interventions or drugs
confounding
definition-Error or inaccuracy in the effect of an
exposure on an outcome due to the
influence of another factor
impact on report outcome-incorrectly concluding that a given exposure causes
an outcome
residual confounding
definition- The error or inaccuracy that remains after controlling for confounding in the design
and/or analysis of a study.
impact on report outcome- incorrectly concluding that a given exposure causes an outcome.
Research has shown that research can be biased!
Background: Studies examining the effects of artificially sweetened beverages on weight have
discrepancies in their results and conclusions.
Objectives: To determine whether risk of bias, results, and conclusions of reviews of effects of
artificially sweetened beverage consumption on weight outcomes differ depending on review
sponsorship and authors’ financial conflicts of interest.
Methods: Systematic review of reviews of the effects of artificially sweetened beverages on weight.
Mandrioli et al. 2016. PLoS ONE 11(9): e0162198. doi:10.1371/journal.pone.0162198
Results:
-Artificial sweetener industry sponsored reviews were more likely to have favorable results (3/4)
than non-industry sponsored reviews (1/23), RR: 17.25 (95% CI: 2.34 to 127.29)
-All reviews funded by competitor industries reported unfavorable conclusions (4/4).
-In 42% of the reviews (13/31), authors’ financial conflicts of interest were not disclosed.
-Reviews performed by authors that had a financial conflict of interest with the food industry
were more likely to have favorable conclusions (18/22) than reviews performed by authors
without conflicts of interest (4/9), RR: 7.36 (95% CI: 1.15 to 47.22).
Ways to minimize bias in research
Journals enforce disclosure requirements and penalize anyone who does not disclose.
* Prevent industry influencing research design, data analysis, and interpretation of findings.
* Develop more accurate measurements to reduce self-report and recall bias.
* More funding for well-designed, large-scale, long-term RCTs that examine cause and effect
* Improve transparency by:
* Researchers providing access to datasets from studies for re-analysis by other researchers.
* All RCTs must be registered (methods described) and results/data made accessible to all.
* Require journals to publish studies with negative findings.
* Avoid using language implying causation when the results are indicative of associations.
* Support researchers and journalists to work together to help the general public gain a more
complete understanding of what studies are really telling us.
An explanation of confounding
You conduct a large observational study and find a positive association
between bacon consumption and increased risk for premature mortality
Residual confounding:
You conduct another large observational study and measure/control for past-year physical
activity levels.
You once again find an association between bacon consumption and increased risk for
premature mortality.
What other confounding factors should you have considered, measured and controlled for?
Can you be sure that bacon consumption is associated with premature mortality?
Health inequalities and generalisability
title
Health inequalities
the preventable, unfair and unjust differences in health status between groups,
populations or individuals that arise from the unequal distribution of social,
environmental and economic conditions within societies, which determine the risk
of people getting ill, their ability to prevent sickness, or opportunities to take
action and access treatment when ill health occurs.”
The 2010 Marmot Review
The 2010 Marmot Review
‘Fair Society, Healthy Lives’ – an evidence-based strategy to address social determinants
of health
Includes conditions in which people are born, grow, live, work and age and which can
lead to health inequalities
The lower a person’s social and economic status, the poorer their health and the lower
their life expectancy
Health inequalities arise from complex interactions between:
* Housing
* Income
* Education
* Employment status
* Social isolation
* Disability
* Gender
* Ethnicity
Deprivation
“The damaging lack of material benefits considered to be basic necessities in a
society.”
The Index of Multiple Deprivation (IMD)
It is used as an overall relative measure of deprivation in England
Deprivation, life expectancy and health inequalities
Has a negative impact on
healthy life expectancy
Increases the risks for
chronic diseases:
-Obesity
-Type 2 diabetes
-Heart disease
-Some cancers
Makes it more difficult to
access healthy options and
to adopt and sustain
healthy lifestyle behaviours
Nearly two-thirds (64%) of adults in England are overweight or obese
Health Survey for England 2019
More than 6 out of 10 men are overweight or obese (68%)
6 out of 10 women are overweight or obese (60%)
Increased obesity rates in those with lowest household income
Why is this important?
Obesity and overweight are associated with increased risks for various chronic diseases
Type 2 Diabetes – over 4.9 million people in the UK
Cardiovascular Disease - ~7.6 million people in the UK
Various cancers (13 different types): breast (post-menopausal), bowel, womb, oesophageal,
pancreatic, kidney, liver, upper stomach, gallbladder, ovarian, thyroid, myeloma meningioma
Cardiovascular disease
most common among-
Male, older adults
Someone with severe mental illness
South Asian or African Caribbean heritage
The most deprived communities
Bias and Biology in the UK:
Sex Inequalities in Heart Attack
Coronary heart disease kills twice as many
women in the UK as breast cancer
Heart disease considered a “man’s disease”
Women delay seeking medical help as they
don’t recognise the symptoms
A woman is 50% more likely to receive the
wrong initial diagnosis for a heart attack
Women are less likely than men to receive
some treatments
Physical activity
People who are more active are less likely to develop chronic diseases including:
– Type 2 diabetes, cardiovascular disease and some cancers
Age (years)
People can be encouraged to become more physically active to
reduce the chance of developing disease
But health inequalities, deprivation, age/sex influence how active
people are…
Different “baseline” and different abilities to become more
active.
Why are study samples not always reflective of population diversity?
Cost, practicalities, convenience (language, willingness, availability, recruitment strategy)
Lack of diversity in areas where research is conducted
Research questions are purposively targeted to a limited demographic
Desire to protect “vulnerable” groups (e.g., older adults, women of child-bearing age, children)
To optimism positive findings (e.g., testing medicines on young, healthy adults vs. older adults)
What are the implications of conducting research with samples that do not reflect population diversity?
Research findings may not be applicable to those who are under-represented
Those who are not represented may be deprived of the benefits resulting from the research
Messages generated from the research can lead to lack of awareness of disease risks
Findings do not reflect the complex “lived experiences” of diverse populations
Policies created from research could be harmful to those not represented
Is it always necessary to recruit diverse study samples?
No – this depends on the research question and whether representation matters
It isn’t necessary if there is no plausible expectation of treatment differences
If treatment effects or outcomes are expected to be different for various groups:
– A sub-group analysis should be done (quantitative studies)
Other scientific considerations
Circadian rhythms
How do circadian rhythms happen and why is this relevant?
helps regulate body temperature, eating and digestion, hormonal activity, and sleep.
Behaviour: important pre-measurement controls
Exercise in the 1 hour before a metabolic challenge
influences results
Rynders et al. J Clin Endocrinol Metab 2014. 99: 220-228
Exercise in the 5 days before a metabolic challenge
influences results
Ten insufficiently active adults completed
three trials in a randomised order. Each trial
comprised five consecutive days of 30 min of
exercise either accumulated in three separate
10-min bouts (ACC) after main meals; a
single 30-min bout after dinner (CONT) or a no-exercise control (NOEX)
