Research methods Flashcards
Blinding vs concealment
Single blind- patient blinded
Double blind- patient and clinician blinded
Allocation concealment- third party/ hidden method used to allocate groups, don’t know what each groups treatment options are. Reduced selection bias, acts similar to randomisation. Especially when blinding not possible due to nature of the intervention.
ITT vs Per protocol
ITT- all participants included in analysis regardless of if they complete treatment.
+ Preserves randomisation
+ greater generalisability, reflects clinical situation
+ maintains sample size
PP- only analysis those who kept to study protocol
- harder to generalise
- may introduce bias
Strengths and weakness of vital statistics
Government collected population level data
+ cheap and easily available
+ mostly complete
+ Contemporary
+ Used for monitoring trends
- Not 100% complete
- Potential for bias (underreporting, post mortum status inflation)
- Can become put of date (census)
Ways to improve routine data quality
Computerise data collection and analysis
Feedback of data to providers
Presentation of data in a variety of ways
Training
Sources of routine stats in England
Census
Mortality stats (ONS)
Morbidity- GP codes, Clinical practice research database, HES, lab results,
National registries (cancer, Congenital abnormalities, Prostheses, transplants), Confidential inquiries
Notifiable diseases, general lifestyles survey
ONS Psychiatric Morbidity Survey
The Association of Public Health Observatories in the UK
Dimensions of descriptive epidemiology
Time. E.g Secular trends (decades/centuries), seasonal, Epidemics, point events)
Place: Where the incidence is high/low
Person: Who is affected? Demographics, occupation, behaviours.
Right censoring
Subjects leaving the at risk population in a cohort study. E.g lost to follow up, die from other diseases.
Left censoring
Subjects joining after the event has occurred. Uncommon, and subjects mostly excluded.
Incidence rate
New cases/ person time at risk
Cumulative incidence
No of new cases/ population at risk
In any given time period.
Assumes a closes population.
e.g attack rate during a pandemic
Direct standardisation
Age specific mortality rates of study population are KNOWN.
Mapped on to reference population to make the rate comparable for differently structured populations.
Age standardisted rate
Indirect standardisation
Age specific rates are NOT KNOWN. Often true in smaller populations e.g ethnicities
Apply age standardised rates from reference population on to study population to calc expected deaths. compare actual and expected.
Standardised mortality rate
Caution using this i n occupational exposures, as whole population contains the sub group. Often require comparison with two groups.
YLL and HALE
YLL- Years of Life Lost
Sum of years lost up to 75
Weights to death at younger age, underestimates burden from chronic disease
HALE- Health adjusted life expectancy
Sum of number of life years lived x health state score.
Attributable risk
Risk that can be attributed to the exposure.
Absolute risk in exposed group- absolute risk in unexposed
e.g Incidence of CHD in smokers vs non smokers
0.1-0.01= Attributable risk of CHD in smokers 0.09
Attributable fraction
What proportion of the disease in the exposed group can actually be blamed on the exposure.
E.g
AR in CHD smokers = 0.09
0.09/0.10= 0.9= 90%
90% of CHD in smokers can be attributed to smoking, as 10% would have occurred anyway.
Population attributable risk
Excess rate of disease in the whole population that is attributable to exposure
Rate in whole pop- rate in unexposed
Population attributable fraction
Effect of exposure on the whole population as a proportion
Rate in whole pop- rate in unexposed (PAR) / Rate in whole population
Risk ratio
Risk of disease in exposed/ risk of disease in unexposed
Calc using 2x2 contingency table
Rate ratio
Incidence in exposed/ incidence in unexposed
Odds ratio
Odds of exposure in diseased (case control) or odds of disease in exposed.
Calc via 2x2 contingency table
(a/c) / (b/d)
Reverse causation
Where the outcome causes a change in exposure
e.g
Breast feeding and poor growth in developing countries- actually due to poor weaning
Sleep and Qol
Drugs and psychological harm
Bradford Hill criteria
Criteria to assess causality
Strength of association- The greater the association the more likely it is due to causation (not true in reverse)
Biological plausibility
Consistency of findings
Temporal sequence
Dose response
Specificity - If the exposure causes on or more outcomes.
Coherence - No conflict with the natural history of the disease
Reversibility- remove risk, disease reduces
Analogy- similar to other established cause-effects
Types of selection bias
Volunteer, Control, Healthy worker effect, follow up bias
Types of measurement bias
Instrument, responder (recall, placebo), observer
Minimising bias
Randomisation
Blinding
Irrelevant factors- collect irrelevant factors to check bias between groups
Repeated measurement - inter observer agreement
Training
Written protocol
Choice of controls
Ease of follow up
High risk cohorts - increase event rate
Duplication/ triangulation
Confounding
A variable that can influence both the dependent variable and independent variable, causing a spurious association.
Residual confounding
Confounding effect when all known confounders have been felt with. this can be reduced with randomisation as these effects are equally distributed between groups.
Effect modifiers
Where the effect of the exposure on the outcome is modified by a third variable. e.g smoking and CHD- worse effect of smoking younger so age is an effect modifier
Analysis of results alongside different age bands should be completed, along side a Chi sq test of heterogeneity
Controlling for confounding: Design stage
Randomisation - In large samples this is effective at minimising confounding. But not always possible
Restriction - limit sample to one group e.g to reduce effect of age and ethnicity. Cheap and easy method, less generalisable results, may get residual confounding
Matching- useful in smaller studies, difficult and expensive, no control when factors can’t be matched.
Controlling for confounding: Analysis stage
Stratification- Mantel-Haenszel method. Divide confounders into strata and provide strata specific estimates (with CI), and weighted average of overall effect. Only controls for a few confounders
Standardisation
Multivariate analysis- multiple regression and logistic regression. Transparency lost, but overall preferred method.
Case study/series
Hypothesis formulation, descriptive, individual based.
+ Rapid, low cost
- No causation/analysis
- not generalisable
- No comparison group
- Not assessing disease burden
Ecological studies
Descriptive, hypothesis generating, population level
Compare large groups.
+low costs and quick
- unknown confounders
- Only considers average exposure
- Spatial auto correlation (assumes all areas are independent)
- leakage of exposure through migration
- Not individual, ecological fallacy
Cross sectional
Can be descriptive, analytical and/ or ecological. Hypothesis formulation.
Simultaneous prevalence of exposure and disease
Disease frequency (odds/prevalence)
Sample representative of population.
+ Multiple exposures and outcomes
+quick and cheap
+Useful for rarer diseases
+can detect disease burden
- Prevalence not incidence (Can’t differentiate determinants and survival)
- risk of reverse causation (no temporality)
- Recall bias
Case control studies
Identify disease and exposure of interest. Cases defined as with disease, controls matched other than without disease.
Can be retrospective or prospective. Odds ratio measures
+Rapid and cheap
+ ideal for rarer diseases/outcomes
+ Disease with longer latent periods
+ Can measure large number of potential exposures
- Selection bias
- Hard to assess temporality
- Recall bias
- Poor for rare exposures
- no incidence
- Misclassification of disease/outcome skews results
- Data fishing
Cohort studies
Exposure and outcome identified.
Select cohort of exposed patient, disease status unknown.
Measures incidence, rate, risk, mean, median. RR, AR and survival analysis
+ Temporal
+ Good for rare exposures
+ Multiple effects of single exposure
+ Minimal selection bias in prospective study
+ Good for long latency
- Expensive and time consuming
- Loss to FU
- not good for rare disease
- Retrospective - poor records
- Healthy worker effect
Intervention studies
Exposure is allocated.
Stopping rules: Indépendant group monitor interim results. Stop for extreme positive or negative results, unblind for serious single events, high significant is required to stop.
Non compliance can lean result to the null
Analysis of frequency, effect, placebo effect and ITT
+high quality
+ Valid
+ Bias minimised if done well
- Generalisability sacrificed
- high cost
-ethics - Bias from loss to FU, observation bias, placebo
Can be cross over, cluster and factorial
Limitations in health research context:
Resources, Timescales (prevention), Changes in policy, differences across the country, difficult to study organisational changes.
Small area analysis
+ Large analysis may hide variation at regional level e.g coastal areas
- There may be little variation of exposure at smaller scale
- data errors and chance have a greater effect on results
- Poor quality data available
Validity
Degree to which an instrument measures what its supposed to measure.
Criterion- Concurrent (compared to gold standard) and predictive.
Face- how well it corresponds to expert option
Content- is it representative of the issue
Construct- does it represent the construct.
Improving validity- measure against gold standard, triangulation, address measurement bias.
Reliability
Consistence of instruments performance
INTRA - observer- same observer, same subject
INTER observer- multiple observers same subject
Measured with a correlation coefficient/KAPPA. >0.7 is generally deemed reliable.
Equivalence - two instruments
Internal consistency- within the instrument e.g specific questions on a survey
Clustered data
Groups/linked data
- Need to adjust sample size to compensate for individuals within a cluster being more similar to each other (ICC- intra cluster coefficient)
- Calc summary stats for each cluster,
- Calc robust standard errors
Using ANOVA
- Random effect models - analyse similarities between individuals within a cluster
Fixed effects- assumptions about the independent variable
NNT
Reciprocal of absolute risk reduction
How many patients do I need to treat to benefit one patient
+ More initiative
- not generalisable to populations where the baseline risk of disease differs
- Can only compare NNT for different therapies of baseline disease risk is the same
Time trend analysis
Describing events over time periods, account for seasonality/ change over time, assessment of exposure/outcome over time, Evaluate impact of an intervention, Effect of an unplanned event, projections.
Analysis using moving averages, and segmented regression analysis
Examples: Time series designs ( two time points in series)
Repeated measures before/after, Crossover (baseline, intervention, baseline), At different locations.
- Secular changes - e.g demographics
- Concurrent interventions/events/ exposures
- Latency periods
- Diffuse exposure
- Seasonal changes
- Auto correlation - for some outcomes the value at one time point affects the value at another - needs adjusted in analysis
Probability sampling
Requires a sampling frame (complete list of the population from which the sample is to be drawn), sampling error can be calculated.
Random
Systematic
Stratified
Cluster
Non probability sampling
Convenience
Purposive
Quota
Snowball
Types of randomisation
Simple- unrestricted randomisation. potential to create unequal group sizes
Blocked- set group allocation and block size. Vary block size to vary sequence. Allows for equal groups.
Stratified- randomisation form within strata
Cluster- groups randomised
Matched pair
Stepped wedge- intervention randomly introduced to all groups over time. Good if intervention is thought to be beneficial.
Effective reproduction number
Average number of secondary cases per primacy case observed.
R= 1 endemic
R> 1 spreading, start of epidemic
R<1 decrease/control
Basic reproduction number
Average number of secondary infections when an infected individual is introduced to population where everyone is susceptible
R0
Secondary attack rate
Risk of secondary cases in exposed.
Hard to know who is exposed. Used mostly for households.
Cases/exposed
Serial interval
Time interval between the onset of signs/symptoms of two successive cases. e.g between primary and secondary case.
Combo of incubation period, latent period and infectiousness.
Critical population size
Th minimum number of people for an ID to remain endemic.
Varies based on population structure, urban/rural/ sanitation, vaccine coverage, prevention measures etc.
Epidemic threshold
The fraction of the population which must be susceptible for an epidemic to occur.
Herd immunity threshold
The proportion of the population to be immune for the incidence of an ID to decrease.
= R0-1/R0
Index case
First recognised case
Primary case
First case of the outbreak (may be realised in retrospect)
Secondary case
Acquired their case from the primary case
Uses of epi curves
Determine the current position of the pandemic,
Projecting future course
Estimating time of exposure
Identify outliers
Infer epidemic pattern
Causes of an early outlier on an epidemic curve
Background/unrelated case
Source case
Early exposure
Causes of a late outlier on an epidemic curve
Unrelated
Long incubation period
Late exposure
Secondary case
Advantages of systematic reviews
Increased power an precision
Greater generalisability
Efficiency and cost
Forest plots
Visual representation of the results of individual studies within a meta analysis
Fixed effect meta analysis
Fixed effect: Only if no/ minimal evidence of heterogeneity.
Assumes the effect of the exposure on the outcome is the same across studies. Summary statistic is an estimate of this common effect.
Therefore variation is due to chance, confounders.
Weighting determined by study size, allows for narrower CI and P value
Random effect meta analysis
Used when there is heterogeneity. More cautious.
Assumes the true effect size is different across studies, and the analysis is on a random sample of effects. Study statistic is an average of these effects.
Therefore wider confidence intervals compared to fixed effect to allow for this.
Bias in meta analysis
Poor selection protocol, Poor quality trails,
Publication bias (detected via a funnel plot)
Limitations to electronic medical data bases
No single database has access to all publications
Bias towards english language publications
Most often searched are health databases, and social sciences aren’t always considered
Time delay between completion and publication
Grey literature
Written material from a body where its primary activity is not publishing.
+ presents less orthodox views
+can provide a perspective to polished material
- not readily available through databases
- minimal quality control
- readers must assess credibility
Advantages and disadvantages of evidenced based medicine
+Explicit use of best evidence
+Opinion least valid form of evidence
-Publication bias
-Retrieval bias
- lack of evidence does not mean lack of benefit
- Less evidence for non drug treatments
- Evidence applies to populations not individuals
- Diminished the value of clinical knowledge
Family studies
Genetic epi, used to identify if there’s a genetic factor to a disease
Is there a higher risk to family members of diseased person than the rest of the population.
Familial relative risk: risk of disease in siblings/risk in general population.
Twin studies
Genetic epi
Explore relative contributions of genes vs environment
Pairwise concordance - % of concordant twins in a group where at least on twin is affected.
Probandwise- % of second twin affected during the study when the first is already affected.
Limitations
- identical twins may not have identical gene expression
- Intrauterine environment may not be the same
- twins differ from the general population reducing generalisability
Linkage studies
Genetic epi,
Find the region on the genome where the gene is located for diseases that run in families.
Linkage disequilibrium- non random association of alleles in close proximity on the genome
LOD score - odds of linkage
- Only indicate broad region of genes
- strong linkage patterns only occur with highly penetrant, recessive diseases.
Association studies
Genetic epi,
measure relative freq of a polymorphism together with the disease of interest.
Case control design
Often follow linkage studies.
- many mutations may lead to the disease so significance of one is often unlikely
Discuss- Qualitative research in policy formation
+ depth of data
+flexibility to explore the topic
+ Narratievs of human experience extracted
+ Reveal complexities
+ engage communities/ stakeholders
- Could be sidelined as not as generalisable
- Reliability difficult to establish
- Acceptability - may not fit with poly makers agenda
- Credibility - more difficult to demonstrate rigor
- Time intensive
Standard error
Standard deviation of the sampling distribution.
How precisely a population parameter (e.g mean) is estimated by the sample mean.
Confidence intervals : AR and RR
Absolute risk-If 95% CI includes 0 = No evidence there is a true difference
Relative risk: If 95% CI includes 1 = No tree difference
Measures of location
Way to summarise data in terms of average values/ central tendencies
Mean:
Arithmetic (sum) good for stats analysis but poor if there’s outliers or asymmetrical distribution
Geometric (product) good for positively skewed distributions, not for anything with negative values.
Median : unaffected by outliers and good for skewed data. Doesn’t offer info about other values
Mode: Not affected by outliers, offers other insights e.g bimodal distributions. not used in stat analysis, sometimes no mode exists, hard to interpret multiple modes.
Percentiles : Can compare measurements
Measures of dispersion
Describes the spread of data
Range: intuitive, simple, but affected by sample size
IQR: Better at larger sample sizes but worse at smaller
Variance and SD
Good for making population inferences
Coefficient of variation: Ratio of the SD to the mean. Gives an idea about the variation compared to the sample. Can compare populations with different means, smaller means are more sensitive to the SD.
Graph descriptors
Type of graph
Axes
Type of data
Units of analysis
Findings
Interpretation
Stem and leaf
+ quick to construct
+ retrain values
- hard to large data sets
Box plots
+ Lots of information displayed
+ good to compare data sets
- Loose exact values
Histograms
+ demonstrate central tendances (location measures )
+ Demonstrates distribution
- Loose exact values
_ difficult to compare data sets - Only continues data
Scatter plots
two variables plotted against east other, shows association.
Positive, negative or non existant
Linear or non linear
Strong or weak
+ Shows trend
+ Exact data values
+ Shows min/max and outliers
- Hard with large data sets
- flat line inconclusive
- Continuous data
Correlation coefficient = strength of linear association between two variables
Type 1 error
Boy who cried wolf story
Incorrectly accepting the alternative hypothesis
Study shows an effect that does not exist.
Cause by random chance- significance level is set such that the Ha is accepted but the random sample isn’t representative of the population
Multiple comparisons
Caused by poor research technique.
Type 2 error
Accepting null incorrectly.
Observing no difference in sample when there is within the population.
Due to small sample size
Bonferroni correction
Used when multiple variables are being tested.
Adjusts the significance level to account for testing multiple variables
Adjusted significance level = significance level / number of variables tested.
e.g testing 15 variables
0.05/15 = 0.0033
Should test to a significance of 0.0033 to reduce type 1 error .
Parametric tests
Two groups
Z test - large samples
T test - small samples
Multiple groups
ANOVA
Pearsons correlation coefficient
Non parametric tests
Mann Whitney U test
Wilcoxon rank sum test
Wilcoxon signed rank test
Kruskal- Wallis test
Spearman’s correlation coefficient
Uses when data is not normally distributed, when data are ordered categorically but have no scale, or to deal with outliers
- Low power
- CI more difficult
- simple bivariate analysis only
Power (stats)
probability that it will be able to detect statistical significance
Normally set at 80% +
Things that affect sample size calc
Size of difference (effect size) - smalls effect requires larger samples
Significance level- smaller p value req large sample
Power- higher power req larger sample
Exposure in baseline population- smaller prevalence requires a larger sample. For prevalence study
Reasons to modify sample size
Increase -
High loss to follow up
low response rate
cluster sampling
confounding
interaction
Decrease -
Matched case - controls
Life tables
Used to display patterns of survival in a cohort of people when we don’t know individuals survival but we know the survivors at each time point.
Cohort life tables- show actual survival. Main table used
Period life tables - current age specific death rates applied to a hypothetical population.
Cohort life tables
Collect at time intervals
- individuals alive
- Deaths
- Individuals censored during the time interval (lost to FU, deaths due to other causes etc)
Can calc- Persons at risk, Risk of dying, chance of surviving, survival function (cumulative chance of surviving up to that time period).
Heterogeneity
Differences between studies
Methodological - study design
Clinical - sample, intervention, outcome measures
Statistical- differences in reported effects
Can test via cochrans Q or I2 stat
Cochrans Q - Chi Sq test for heterogeneity. low P value suggests heterogeneity
Low power with few studies, p= 0.1 often used
I2- Describes % total variation between studies
<25% - low heterogeneity
> 75% high
Funnel Plots
Scatter plot used in meta analysis and sometime performance
Study size and effect size are plotted with CI. Large studies should have higher precision, creating the funnel shape.
If its asymmetrical there may be small study effect, either due to publication bias.
This means the meta analysis will over estimate the treatment effect.
Bayes Theorem
Incorporation of prior beliefs in probabilities
+ More flexible
+ makes use of all available knowledge
+ mathematics isn’t controversial
- Different priors = different conclusions
- hard to quantify prior beliefs
Why conduct a HNA
Consult the population
Establish partnerships
Ensure healthcare provision is evidenced based
Corperate HNA
Consider views of interested parties
+ Incremental process
+ quick
+ Small data collection
+ Responsive to interested parties
- Driven by power rather than need
- Can be influenced by media/events
- Incremental approach may not be appropriate
Comparative needs Ax
Used surveys or hospital data to c compare local situation to what is expected nationally/ reference population.
Evidence based needs Ax (mini needs Ax)
Literature review of guidance/ consensus statements to shape provision.
Conducting a HNA
- Purpose of the assessment: Why any assessment of the population is being undertaken?
o What are the issues?
o What are the service pressures?
o Who is commissioning the work? - Define the population: Pick an example
o could be a country, a local area, a general practice or a neighbourhood.
o The area selected will influence how the needs assessment will be carried out.
o Is it the whole population, a sub set by age (older people, young people), or a group with specific needs such as homeless people? Socioeconomic status? - Epidemiology: Morbidity/mortality sources inc
o ONS Psychiatric Morbidity Survey (or equivalent), and census data.
o The Association of Public Health Observatories in the UK - Comparative assessment:
o Local provision against national norms.
o Prescribing data and other data from general practice will provide valuable information. - Individual assessment of need.
o The Care Programme Approach in England contains individual level data which it may be possible to access or to sample.
o Linked individual level data in some form - Service User/ provider Views / Rapid Appraisal
Core data in a HNA
Demographics
Social and environmental context- employment/housing
Lifestyle
Burden of ill health
Service provision
Activity data can underestimate - don’t include unmet need, people who self care, private treatment
Epi data can over estimate - people don’t need treatment, don’t want treatment, had treatment.
Participatory HNA
Involved the local population through qualitative methods
Increased acceptance of results
Reach smaller populations
Donabedian Framework
Assess quality of healthcare
Structure (staff, budgets etc)
+ Easy to measure
- May not be comparable
Process ( Procedures, referrals, prescriptions)
+ Easy to measure
+ Can be directly related to outcomes
- Some don’t predict outcomes
Output (No of operations)
Outcome ( Health status)
+ Aim of service
+ Can use surrogate end points
- Affected by case mix
- Long term
- Costly
Measures and components of QoL
Components - Expectations, needs and normal living
Measures- Short form 36, Nottingham health profile, EQ5D, functional assessment of cancer therapy, hospital anxiety and depression scale, multi dimensional fatigue scale.
Maxwells dimensions of quality health / healthcare
Access: Tangible (geographic) or Intangible barriers (language)
Relevance to need
Equity
Efficiency
Effectiveness
Acceptability
Public Health Outcome indicators
Reflect the effect of healthcare and PH policy and activities (at population level)
E.g suicide in MH, 30 day mortality post op, life expectancy vs healthy life expectancy.
Uses
Prompt assessment of local outcomes (in relation to targets)
Monitor variation in healthcare
Monitor trends in healthcare
Monitor QoL as part of a HNA
Characteristics
- Relevant
- Valid
- Practical - Available at local level, costs to gather, clear methods, Suitability (to compare groups), easy to gather.
- Meaning- can it show variations effectively? Can it show were processes have failed?
- Value- will the indicator allow for change? Open for abuse (gaming, unintended consequences?)
Townsend score
Comparative deprivation, measured at census.
Household factors inc: more than one person per habitable room, No car, Not owner occupied, unemployed resident
Features of a good evaluation
Clear purpose/ objectives
Robust process
Sufficient resources
Flexibility
Stakeholder engagement
Inverse care law
Availability of good medical care seems to vary inversely to need.
Confidential enquiry
Utilise a case control design to investigate serious incidents
NCEPOD- National Confidential Enquiry into Patient Outcomes and Death
Delphi methods
Method to generate consensus without face to face meetings.
Experts surveyed
Views shared anonymously with the group highlighting areas of disagreement
Respondents review their responses
Repeat until consensus reached
+ Anonymity and no F2F discussion
+ Time efficient
+ Encourages open critique
- Written format may suit some people more
- Administrators can manipulate the process
- Sometimes consensus is not the best result, may need innovative thinking.
Prevention: Define Population approach vs High risk approach
Population approach: The whole population receives the prevention approach e.g legislation
High risk: Target to high risk groups e.g chlamydia screening targeted at 15-24 year olds.
Prevention paradox
Interventions that bring large benefits to the community offer minimal benefit to the individual
Health Impact Assessment: Steps
Screening- Is there health relevance to the policy?
Scoping- Decide the questions the HIA should assess. (individual, environmental, institutional…)
Appraisal- Assess health impacts. (positive and negative)
Reporting - recommendations to minimise impacts
Monitoring
Health Impact Assessment: Challenges
Evidence : is there available evidence?
Time and resources
Action: ensuring the recommendations are considered/implemented
