EBP Flashcards
Prevention
Action that prevent disease occurrence / slow disease progression / minimise impact of disease
Examples of clinical preventive interventions:
- Immunisation
- Behavioural counselling e.g. smoking cessation
- Screening
- Chemoprevention e.g. statins for CVS disease prevention
Levels of prevention
Primary:
- aim to ↓ incidence
- targets ***risk factors / causes / disease in populations / individuals
- e.g. immunisation
Secondary:
- aim to ↓ prevalence
- targets individuals with ***early disease
- e.g. screening
Tertiary:
- aim to ↓ complications
- targets individuals with ***established disease
- e.g. stroke rehabilitation
Primordial:
- targets underlying ***social, economic, environmental determinants of health
- accomplished through **public policies and **intersectoral action
- e.g. control of air pollution
Quaternary:
- identifies individuals and groups at risk of ***over-diagnosis / over-treatment
- accomplished through actions that prevent iatrogenesis
- e.g. evidence-based medicine
Individual vs Population approach
Individual (high-risk) approach (only target individuals at high risk):
- Intervention appropriate to individual —> high motivation for physician and subject
- Screening has inherent diffulcties and costs —> only palliative, not radical
- Limited potential for populations —> large number of people at small risk may outweigh small number of people at high risk
Population approach (lower risk for whole population):
- Large potential for populations
—> aims to shift whole distribution of exposure in a favourable direction (i.e. lower prevalence of disease of whole population: shifting the normal distribution curve)
—> behaviourally appropriate; changes norms - Limited by ***prevention paradox
—> individuals perceive little benefit from intervention even though measure may benefit population
—> lower physician, subject motivation
Screening
Presumptive identification of an unrecognised disease/defect by application of tests/examinations/other procedures which can be applied rapidly
- Sort out apparently well (asymptomatic) people who probably have a disease from those who probably do not
- NOT intended to be diagnostic
Goals of screening
Early detection —> Early intervention —> Reduction in morbidity / mortality
- Reduction in morbidity / mortality from the newly identified disease among screened individuals due to ***early treatment
- Reduce disease burden for the population
- Provide cost-effective screening to target population
Wilson and Jungner screening criteria
Knowledge of diseases:
- Condition should be ***important health problem
- There should be a ***recognisable latent / early symptomatic stage
- ***Natural history of condition, including development from latent to declared disease, should be adequately understood
Knowledge of test:
- ***Suitable tests / examinations
- Test ***acceptable to the population
- Case-finding should be ***continuous (NOT “once and for all”)
Treatment of disease:
- ***Accepted treatment for patients with recognised disease
- ***Facilities for diagnosis and treatment should be available
- ***Agreed policy on whom to treat as patients
Cost considerations:
1. Cost of case-finding (including diagnosis and treatment of patients diagnosed) should be ***economically balanced in relation to possible expenditure on medical care as a whole
Assessing a screening test: Reliability and Validity
Reliability: degree to which results can be replicated
- depends on
—> Variability between people
—> Measurement error
Validity: degree to which results obtained by a test are true
- Sensitivity
- Specificity
***Sensitivity and Specificity
Used to decide whether a test should be used
Sensitivity (probability of true positive):
With disease + Positive test / Number of people with disease
Specificity (probability of true negative):
Without disease + Negative test / Number of people without disease
There will be trade-off between Sensitivity and Specificity
—> ↑ Sensitivity —> ↓ Specificity
Receiver operator characteristic (ROC) curve
- used to help decide best cut-off point
- Sensitivity plot against Specificity
- maximising **True positive (Sensitivity), minimise **False positive (1-Specificity)
- cut-off point usually at “shoulder” of ROC curve
- ***Area under ROC curve: used to compare overall accuracy of tests
***Predictive values
Predictive values:
- used to determine how likely a patient has / not has disease
- ***Post-test probability
PPV = True positive / All positive —> ***↑ with prevalence of disease (given constant SN, SP) (∵ ↑ true positive) NPV = True negative / All negative —> ***↓ with prevalence of disease (given constant SN, SP) (∵ ↓ true negative)
Sensitivity = True positive / All disease
—> ability to pick up all diseased people
—> high sensitivity
—> high NPV (very few false negative) (無就一定無, 有病就一定可以pick up到)
—> low type II error
Specificity = True negative / All healthy
—> ability to exclude normal people (無就一定無)
—> high specificity
—> high PPV (very few false positive) (有就一定有, 所有無病一定exclude)
—> low type I error
Use of sensitive test: correctly identify people WITH disease, rule out dangerous conditions (negative result tends to rule out diagnosis)
Use of specific test: correctly identify people WITHOUT disease, false positive results are highly undesirable (positive result tends to rule in diagnosis)
***Bayes’ theorem
Prevalence of disease:
- reflects Pre-test probability (probability of disease before diagnostic test result is known)
Predictive values:
- reflects Post-test probability (probability of disease after diagnostic test)
—> ***influenced by Prevalence (Pre-test probability)
SN, SP:
- ***Inherent characteristics of test
- ***Independent of Pre-test portability / prevalence
Implications of Predictive values dependent on Prevalence:
1. Better estimate prevalence by using best available information sources
—> high quality surveillance data
—> epidemiological studies with minimal bias
- Increase Pre-test probability may increase PPV
—> Apply diagnostic tests to more susceptible patients (use sparingly in primary care)
—> Interpret results of studies with care (tertiary care have higher disease prevalence, cannot be applied to primary / secondary settings)
Risks of screening
- Discomfort and inconvenience
- Psychosocial consequences
- Anxiety
- Family discord on further procedures - Adverse health outcomes
- False-positives: possibly invasive diagnostic procedures
- False-negatives: undetected disease
Screening for colorectal cancer
Asymptomatic Hong Kong residents 50-75
- Faecal occult blood test every 1 to 2 years OR - Flexible sigmoidoscopy every 5 years OR - Colonoscopy every 10 years
PICO question
Patient/Population in question
Intervention being considered
Comparison group
Outcome
***Biases associated with screening
- Lead-time bias
- Lead-time: Time between detection by screening and usual diagnosis
- Apparently improved survival time with no change in time from disease onset to death
- To avoid: Use ***mortality rather than survival rates - Length-time bias
- Slow-growing lesions are more likely to be detected by screening than fast-growing lesions
- Screening tends to pick up slower growing lesions which are associated with better prognoses
- To avoid: Compare outcomes in ***RCT - Compliance bias
- Individuals who are adherent to interventions usually have better prognosis than non-compliant individuals
- tend to be more health conscious and higher socioeconomic position
- To avoid: Compare outcomes in ***RCT
Why studies showed that screening cannot save lives?
- Studies may be underpowered to detect small overall mortality benefit
- Any reductions in mortality may be offset by deaths due to ***downstream effects of screening
- e.g. PSA screening yields numerous false-positive —> numerous prostate biopsies —> unnecessary harms and deaths
Misuse and Overuse of unnecessary tests/interventions:
Misuse: Interventions more harmful than beneficial
Overuse: Interventions that yield little useful information
Appraising a study
Relevance:
- Is population similar to the patient?
- Intervention feasible?
- Comparison group reasonable?
- Outcome patient-oriented?
Validity (3 factors affecting):
- Random error
- Systematic error / bias
- Selection bias
- Information bias - Confounding
Variations in responses:
- Participant variability
- Result of errors
Validity:
- Study sample
- Selection bias (representative of target population?)
—> inclusion / exclusion criteria reasonable?
—> where were patients recruited?
—> enrolled consecutively?
—> response rates reasonable? - Study design
- Systematic reviews > individual studies
- RCT for effectiveness of intervention
- Prospective cohort studies for prognosis / etiology
For assessing prognosis:
- is there an inception cohort? (group of patients assembled at onset of disease)
- at the same point / stage in their illness?
- Study conduct
- Selection bias (allocation to groups concealed? follow-up reasonably complete? drop-out rate differ between groups?)
—> loss to follow-up —> reduce statistical power
—> need >=70-80% patients follow up
- Information bias (observer bias, recall bias)
—> measurement of outcomes differ by exposure status?
- Confounding (adjust for important potential confounders) - Study results
- Statistically significant (confidence intervals, P values, adequate power: 0.8 (sample size) for null associations)
—> bigger effect size —> smaller sample size needed
- Clinically significant (effects in clinical practice, relative risk reduction, absolute risk reduction, NNT)
- How to apply results to patient
- Adjusted for confounders - Other factors that could affect interpretation of results
- STEPS (Safety, Tolerability, Effectiveness, Price, Simplicity)
- Conflicts of interest
Bias
- Information bias
—> Observer bias (by investigators)
—> Recall bias (by subjects, particular problem if public is aware of exposure-disease association)
- reduce bias:
—> Blind assessment of outcome status (investigator blinded)
—> Prospective assesssment of outcome (no need to recall)
—> Use objective measures - Selection bias:
- systematic difference between subjects and target population
- subjects not representative of target population —> affects EXTERNAL validity
- subjects groups are not comparable —> affects INTERNAL validity
- reduce bias:
—> random sample
—> maximise response rate
—> minimise attrition rate
—> select case and control from same underlying population
***Relative risk (RR)
Risk of benefit / harm of intervention compared to another
- depend only on relative difference and does not account for risk of no treatment (absolute risk at baseline)
EER (experimental event rate) = treated with effect / total treated
CER (control event rate) = control with effect / total control
Relative risk reduction (RRR) = (1-RR) x 100% OR ***CER-EER / CER
Absolute risk reduction (ARR) = Incidence in control group - Incidence in intervention group (i.e. CER-EER)
Number needed to treat (NNT) = 1 / ARR
***Number needed to treat (NNT)
NNT = 1 / ARR
- Number of patients need to be treated for one patient to receive benefit/prevent disease
- Ideal NNT = 1
- take into account that not everyone benefit from an intervention
- may be more useful than RR
—> combines relative benefit with background risk of patients
Prognosis
Likely course and outcome of disease / condition over time
Clinical course = prognosis of a condition that has received intervention
Natural history = prognosis of a condition that has NOT received intervention (determines when to intervene, disease staging etc.)
Types:
- Average / Overall prognosis: “most likely course of a condition”
- Prognostic factors: factors associated with outcome
- Risk prediction models: risk groups who are likely to have worse outcomes
Finding out:
1. Clinical experience (prone to bias)
(2. Systematic review of cohort studies)
- Cohort studies (exposure睇outcome)
- **better evidence on prognosis
- recruit based on **exposure characteristic:
—> prognostic factor: exposed (e.g. smoking) vs not exposed (e.g. no smoking)
- prospective —> look for ***outcome (morbidity / mortality) - (Control arm of) RCT
- treatment arm —> clinical course
- control arm —> natural history
- problem: participants may not be representative of population of interest
- may give information on prognosis but ***participants may not be representative of population of interest - Case-control studies (outcome睇exposure)
- recruit based on **outcome:
—> Cases
—> Control
- retrospective —> look for prognostic factors / **exposure
- problem: does not provide information on outcome rates (∵ you select cases) - Cross-sectional studies
- cannot know temporal sequence
Causation
Necessary cause: outcome would not occur without exposure
Sufficient cause: exposure guarantees cause will occur (very few in medicine)
E.g.
- HIV is a necessary cause of AIDS (must have HIV to have AIDS)
- but not a sufficient cause for AIDS (not everyone has HIV develop AIDS)
Finding cause:
- Experimental studies e.g. RCT
- Observational studies only tells ***associations
Bradford Hill’s Criteria
- Consistency (on replication)
- Strength (of association)
- Specificity
- Dose response relationship
- Temporality
- Biological plausibility
- Coherence
OA risk factors:
Non-modifiable:
- age
- sex
- genetics
- bone deformities
Modifiable:
- obesity
- joint injuries
- occupational exposure
Dealing with confounding
- Multivariable analysis
- adjust / control for confounders
- Regression analysis
Regression analysis:
- mathematical model to describe relationship between Dependent variable and ***>=1 Independent variables
- allow estimation of a range of exposures, each one adjusted for potential confounding effect of others
- describe relationship between exposure and outcome adjusted for effects of potential confounders
—> “all else being equal, what is effect of X on Y?”
Types of regression analysis:
- Logistic regression —> Binary outcome
- Linear regression —> Quantitive outcome
- Cox regression —> Time to occurrence of an event (e.g. survival)
- ***effects of several variables on time to occurrence of an event
- Hazard: instantaneous rate of the event at time t
- Cox proportional hazards model assume that Hazard Ratios comparing different exposure groups remain constant over time
Hazard ratio
Ratio of hazards between 2 groups
- interpreted in the same ways as risk ratio
HR>1: high hazard than comparison group
HR=1: no difference in hazard from comparison group
HR<1: lower hazard than comparison group
Survival curve vs Survival rate
Survival curve:
- Probability of survival starts at 100%, decreases over time
- Shows proportion of study population that is still alive at successive time points
- Patients are censored when lost from study
Survival rate:
- alternative to survival curve
- survival rates at different time points (1 year, 5 years), median survival rate, disease-free survival rate etc.
Mortality
All-cause mortality
- includes all causes of mortality, even those seemingly unrelated to study exposure
- ***less prone to misclassification
Disease-specific mortality - heavily relies on accuracy and completeness of death certification —> immediate cause —> intervening cause —> underlying cause - ***more prone to misclassification
—> intervention may just reduce one of the above (All-cause / Disease-specific mortality)
Doctors when issuing death certificates, causes of death must be consistent with those in ICD-10 (classification system by WHO)
Standardised mortality ratio (SMR):
- Mortality of study group to Mortality of standard population
- SMR =1: same mortality rate as standard population
- SMR >1: higher mortality rate in study group
- SMR <1: lower mortality rate in study group
(Sometimes 100 is used rather than 1)
Diagnosis
Diagnostic tests:
- Laboratory tests
- Imaging
- Endoscopy
- History + Physical examination
- combination of above
Diagnosis is a ***Probability based on:
- Frequency of disease:
- Prevalence (more commonly available)
- Incidence - Index of suspicion:
- Risk factors / risk predictors: age, smoking status etc.
—> Effect measures: Odds ratios, Risk ratios
- Careful history and clinical judgement - Accuracy and validity of diagnostic tests
Screening vs Diagnosis
Screening:
- for Asymptomatic patients with early disease
—> secondary prevention
—> lower prevalence of disease
- Aims to identify people with high risk of disease where interventions may reduce morbidity / mortality
- NOT intended to diagnose
Diagnosis: - for Symptomatic patients —> higher prevalence of disease - Aim: make a definitive diagnosis / narrow down DDx - NOT intended to prevent disease
Issues with determining validity
- An appropriate “gold standard” (reference standard)
- best available test to determine presence / absence of disease
—> new test should not be compared to imperfect test - sometimes too invasive / costly
—> simpler tests should only be used as proxies when risk of misclassification is low - disease may lack objective standards for diagnosis e.g. mental disorders
- Lack of information on negative tests
- clinicians reluctant to further test when initial test is negative
***Likelihood Ratio (LR)
Describes performance of a test:
LR = probability (test results if disease is present) / probability (test result if disease is absent)
LR+ = P(+ve test in diseased) / P(+ve test in normal people)
—> Sensitivity / 1-Specificity
—> ***high LR+ —> better the positive test can rule in disease
LR- = P(-ve test in diseased) / P(-ve test in normal people)
—> 1-Sensitivity / Specificity
—> ***lower LR- —> better the negative test can rule out disease
LR = 1
—> no change in likelihood of disease —> not a useful test
Use:
1. Allow application to individuals with different **pre-test probabilities:
- ∵ Predictive values are not transferable with different prevalence, but LR still applicable even though prevalence is low
- **Pre-test odds x LR = Post-test odds
(need to convert probability to odds)
- Summarise test performance for a range of possible ***test values
- rather than SN and SP defined by a single cut-off point
- disease should be more likely for extreme than borderline test results
Multiple tests
Parallel testing (囊括越多人) —> ↓ False negative:
- ↑ SN and NPV
- but may ↑ false positive
- clinical prediction rules may be used to combine multiple test results and pre-test probabilities
Serial testing (逐漸縮窄範圍) —> ↓ False positive:
- ↑ SP and PPV
- but may ↑ false negative
- serial LR may be calculated
Potential harms of diagnostic tests
- Diagnosis does not necessarily lead to improvement in outcomes
- Hazards of inappropriate diagnosis
- false positive results
- false negative results
- overdiagnosis - Overuse / Misuse of diagnostic test is waste of resources
Therapy / treatment
Any intervention intended to improve course of established disease
May include:
- medications
- surgery
- changes in organisation / financing of health care
Observational vs Interventional studies
Observational studies:
- Examine distributions and determinants of outcomes (observe exposed vs not exposed)
- Without any attempt to influence them
- Easier to conduct, less expensive
- ***Residual confounding problem —> should equally allocate confounders on 2 side
1. Descriptive (cross-sectional, cohort)
2. Analytical (cross-sectional, cohort, case-control)
Interventional / Experimental studies (e.g. clinical trials):
- Modify an exposure within a population (observe intervention group vs control group)
- Examine its effect on outcome
- More time-consuming, costly
- ***Randomisation minimise confounding
- ***Selection bias, Information bias problem
RCT:
- gold standard study design to provide evidence for effectiveness due to randomisation
- participants randomly allocated to different intervention group to minimise confounding
- followed up prospectively for assessment of outcomes
Level of evidence
Highest —> Lowest:
- Evidence summaries
- Meta-analysis / Systematic review
- RCT
- Cohort studies
- Case-control / Cross-sectional / Ecological studies
- Expert opinion
Elements of clinical trials
- Ethics
- Intervention
- Control / Comparison
- Outcomes / End-points
- Ethical principles
- Equipoise:
- randomisation is ethical when either of the allocated interventions is better than the other - Respect for rights of individual participants:
- life, health, dignity, integrity, self-determination, privacy, confidentiality
- informed consent
- voluntary participation
- specific consideration for vulnerable groups - Minimising risks:
- measures to assess, monitor, minimise risk to human subjects - Research ethics committee:
- research protocol should be submitted for consideration and approval
- right to monitor ongoing studies, esp. for serious adverse events - Reporting and dissemination of results:
- registration in publicly available database
- full reporting + dissemination of results
- Intervention considerations
- Feasibility:
- some interventions may not be available in all settings - Complexity:
- treatment plan usually involve multiple interventions (will an intervention work with others?) - Effect size:
- relative effects may exaggerate absolute effects (does the intervention has a clinically significant effect)
- Control group
- No treatment
- Hawthorne effect: participants may change their behaviours even without interventions (modify an aspect of their behavior in response to their awareness of being observed) - Placebo
- intervention intended to be indistinguishable from active treatment
- intended to have no active effect
- Placebo effect: report improvement even with placebo - Usual care / another intervention
- ***meaningful if this is the most effective treatment available (i.e. “gold standard”)
- Outcomes / End-points
- Primary
- outcome investigators are most interested in
- basis for ***hypothesis testing + sample size calculation - Secondary
- additional outcomes of interest
- associations are only ***hypothesis-generating as may have occurred by chance
—> study may not have enough power to detect these associations
—> interpret with caution
- e.g. SE, differences by subgroups - Hard
- defined according to **objective criteria, involves **no subjective assessment
- usually preferred over soft outcomes
- e.g. All-cause mortality - Soft
- require subjective assessment by investigator / participants
- potential ***information bias —> blinding is important
- e.g. Cause-specific mortality - Patient-oriented
- focus on helping patients live longer and better
- ***preferred over disease-oriented outcomes
- e.g. mortality, morbidity, QOL - Disease-oriented
- focus on pathophysiological mechanisms / markers of disease
- not preferred but ***easier to measure
Types of clinical trials
Function:
- Prevention / Diagnosis / Treatment
Design:
- Cluster vs Individual
- Randomised vs Non-randomised
- Parallel-group vs Cross-over
- Superiority vs Non-inferiorly / Equivalence
Objective:
- Explanatory (efficacy) vs Pragmatic (effectiveness)
- Phases 1 to 4
Superiority vs Non-inferiority trials
Superiority trials
- aim to establish that a new treatment is BETTER than another
- e.g. a new drug more effective than usual care
Non-inferiority trials
- aim to establish that a new treatment is NOT WORSE (not clinically inferior) than current / control treatment by more than a non-inferiority margin
—> smallest difference in effect considered to be clinically important
—> control treatment must be more effective than placebo
- one-sided
- Equivalence trials: 2-sided (new treatment is not better not worse than current treatment)
- e.g. a new drug with similar effectiveness is safer, cheaper, easier to administer
- conducted when new treatment is ***not expected to be better than control in terms of primary health outcomes, but may offer other benefits
***Non-inferiority is only shown if 95% CI for different between test and reference treatments are > -Δ (過左-Δ —> 代表同之前一樣; 過曬-Δ —> 差過之前)
Non-inferiority: CI well below non-inferiority threshold
Superiority: CI well above non-inferiority threshold
Explanatory (efficacy) vs Pragmatic (effectiveness) trials
Explanatory (efficacy) trials:
- Can treatment help under **ideal circumstances (e.g. RCT)?
- include highly selective and homogeneous patients who are adherent
—> higher internal validity
—> may have lower external validity / generalisability
—> limitations: **selection bias
Pragmatic (effectiveness) trials: - Can treatment help under ***ordinary circumstances? - done after efficacy is established - done in routine health care settings - include more diverse patients —> participants not followed as closely to ensure adherence —> ***higher external validity —> may have lower internal validity
Clinical trial phases
Phase 1:
- safety + dosage
- 20-100 participants over several months
- “is it safe in health people?”
Phase 2:
- efficacy + SE (i.e. explanatory trials)
- several hundred participants over months - 2 years
- “does it work?”
Phase 3:
- efficacy / effectiveness + adverse reactions (i.e. pragmatic trials)
- hundred to thousands of participants over 1-4 years
- “is it any better than current treatment?”
Phase 4:
- post-marketing surveillance of long-term benefits, risks
- several thousand participants
- “will it have any long term results?”
***Limitations of clinical trials
- Confounding
- ***Randomisation —> randomly distribute confounders in intervention / control groups - Selection bias
- Healthy volunteer bias
—> participants may be healthier than average patients —> need to note how ***sampling was done
- Cherry-picking participants
—> assign healthier participants to desired treatment group —> need to ensure ***allocation concealment - Attrition
—> participants who drop out may not be representative of all enrolled participants —> need to note follow-up rates and whether ***intention to treat analysis was done
- Information bias
- ***Blinding - Practical limitations
- time-consuming
- cost
- not feasible / ethical interventions
Randomisation
- Randomly distribute confounders in intervention / control groups
- Bad examples:
—> assignment based on day of admission (not random)
—> assignment based on day of birth (not concealed) - Non-randomised studies: overestimate / underestimate effect of intervention
- ***Check if baseline characteristics of participants are similar across groups
—> if sample size is small —> baseline differences are even exaggerated
—> significance tests of baseline differences (giving P values) may not be very useful
Types of randomisation
- Unrestricted (simple) randomisation
- no constraints in generation of random allocation sequence (e.g. a table of random numbers) - Restricted randomisation
- ensure particular allocation ratios to treatment groups (e.g. 1:1)
Sampling
Meet strict criteria for participation
Inclusion criteria:
- diagnostic criteria to ensure patients have the condition under study
- patients with unusual, mild / equivocal manifestations of disease may be excluded
Exclusion criteria:
- usually involve patients who have
—> comorbidities that may affect outcomes
—> limited life expectancy that may affect assessment of outcomes
—> contraindications to treatment
—> do not participant
—> do not cooperate in early stages of trial
However, problem of highly selective:
- Limit ***external validity / generalisability
Example of sampling:
Sampled population —> Invited to participate —> Able to contact and agreed to participate —> Eventually randomised
Allocation concealment
Ensure investigators do not know / have no influence over which group participants are allocated to
—> minimise selection bias
Low risk of bias:
- opaque sealed envelopes containing group assignment
- central randomisation by a third party
High risk of bias:
- allocation based on day of admission
- allocation sequence posted on staff room wall
Inadequate allocation concealment yield ***larger effect estimates when differences in results did occur
Incomplete outcome data
Missing outcome data may:
- Bias the observed effects
- esp. when % missing differ across groups
- patients who drop out / missing data may have worse prognosis - Reduce statistical power
- esp. when % missing is high
Attrition / Drop out because:
- withdrawal (due to SE)
- do not attend follow up (busy)
- do not provide relevant data
- cannot be contacted (migration / death)
- later found to be ineligible
- investigators ceased to follow up (inadequate funding)
Non-adherence
Extent to which patients fail to follow treatment recommendations / take medications as prescribed
Patient factors: illness, perceived benefits and risks
Treatment factors: cost, complexity, duration, SE
Other factors: poor doctor-patient communication
Measuring adherence in clinical trials:
- cheaper but less accurate
—> self-report, diaries, pill counts, prescription refills, third party observations
- more accurate but more costly
—> measuring drug levels in plasma, directly observed therapy
Dealing with non-adherence:
- Exclude non-adherent patients during run-in period
- patients given placebo and observed for adherence —> non-adherence patient then excluded before randomisation - Intention-to-treat analysis
- analyse study outcomes in all participants according ***original assigned treatment group
Advantages:
- perserves randomisation (reduces confounding)
—> provide more conservative estimate of effect
—> selection bias may still exist if outcome data incomplete
- more accurately reflects what happens in reality
-
**different from per protocol analysis: include participants only if they received the intended intervention in accordance with the study protocol
—> may **overestimate treatment effect due to confounding (lose randomisation)
—> may be done alongside ITT analysis to investigate influence of missing data
Blinding
Keeping people unaware of which intervention is administered to which participants
—> prevent ***information bias
Done on various levels
- Participants
- Care providers
- Assessors of outcomes (most important, always feasible)
- non-blinded outcome assessors tend to ***overestimate effect of intervention
Types of blinding
1. Single-blind, double-blind, triple-blind
2. Open-label
—> usually assumes no attempt at blinding
3. Placebo-controlled
—> usually assumes that participants are blinded
Lack of blinding may have more influence on subjective outcomes than objective outcomes
Allocation concealment vs Blinding
Occur at different timeline
Allocation concealment: Prevent **selection bias
Blinding: Prevent **information bias
CONSORT statement
Consolidation of Standards of Reporting Trials
- specifies content to be covered when reporting RCT e.g. Full trial protocol, to be repeated
—> help us assess quality of reporting
- consist of:
1. Checklist
2. Flow diagram (showing passage of subjects through RCT)
- enrolment —> allocation —> follow-up —> analysis
Harm to patient safety in medical practices
- Only 1/3 medical practices = Beneficial / Likely to be beneficial
- ~20% expenditure wasted on unnecessary diagnostic tests / treatments
Overuse: interventions that yield little useful information
Misuse: interventions that may be more harmful than beneficial
Hospital:
- Unsafe use of medication (Leading cause)
- Healthcare-associated infections
- Surgical complications
Primary care:
- Administrative errors
- Diagnostic errors
Combatting Overuse / Misuse:
- Choose wisely
- Prudent healthcare
- NICE
Measures of patient safety
Hospital-acquired infection:
- Ventilator pneumonia
- Wound infection
- Decubitus ulcer (i.e. Bedsore)
Operative / Post-op complications:
- Complication of anaesthesia
- Post-op sepsis
- Post-op DVT
Sentinel events:
- Transfusion reaction
- Wrong surgical site
- Foreign body left
Obstetrics:
- Birth trauma
Other care-related adverse effects:
- Falls
- Fractures
Unsafe care
Structures - Resources and organisational arrangements in place that influence care —> Safety culture —> Inadequate Human Resources —> Stress and fatigue
Processes - Activities of physicians and other providers of healthcare —> Misdiagnosis —> Poor test follow-up —> Inadequate measures of patient safety
Outcomes
- Results / Consequcnes of clinical activities
—> Adverse events due to medications
—> Surgical complications
Example:
Adverse events to drug treatment
1. Health system level:
- lack necessary information for safe use of drug
- error-prone conditions: look-alike / sound-alike medication
- Provider level:
- lack of knowledge about drug
- lack of follow-up
- fail to recognise drug SE
- fail to prescribe drug
Types of medical errors
- Preventive
- fail to provide prophylactic treatment
- inadequate monitoring to follow-up - Diagnostic
- delay in diagnosis
- fail to use indicated tests
- use of inappropriate tests
- fail to act on test results - Treatment
- error in administering drug
- error in procedure / operation
- delay in treatment
- inappropriate treatment
Progress in patient safety
- Little evidence that patient outcomes improved despite research
- Investment in safety research is much lower than magnitude of problem
- Medical errors are caused by faculty system, processes / conditions that lead people to make mistakes rather than result from individual recklessness
Ethical issues in clinical research
- Respect for rights of individuals participants
- informed consent
- confidentiality
- specific consideration for vulnerable groups - Minimising risk
- measures in place to minimise risk - Research ethics committee
- approve research protocol
- monitor ongoing studies esp. adverse effects - Reporting and dissemination fo results
- registration in publicly available database
- full reporting and dissemination of results
Ethical issues concerning patient safety research:
- Subject may be system / healthcare provider rather than a patient
- Tensions may arise between a researcher’s duty of care and aims of research
- Liability of health care providers when health care failures are documented
- Patient confidentiality
- Dissemination of findings
Study designs
Observational (e.g. Cohort studies):
- Examine outcome without attempt to influence them —> too see Prognosis
Interventional (e.g. RCT):
- Modify an exposure and examine outcome —> measure effectiveness
Hierarchy of evidence
Evidence summaries —> Systematic reviews / Meta-analysis —> RCT —> Cohort studies —> Case-control studies —> Expert option / Consensus —> Pathophysiologic reasoning
Systematic reviews (i.e. Synthesis review)
- Review of all available evidence on a specific topic
- Apply **Systematic, **Explicit strategies that limit bias
—> Assembly, Critical appraisal and Synthesis of evidence of all relevant studies
—> provide more reliable evidence than individual studies
—> identify gaps in existing evidence
Process of systematic reviews
Literature search, Selection of studies, Data extraction
—> done by **>=2 **independent reviews to limit ***selection bias
—> should declare conflicts of interests (financial / intellectual conflicts of interest)
Steps:
1. Ask an answerable clinical question
2. Define inclusion + exclusion criteria
3. Develop search strategy + Identify relevant information sources
- Search strategy:
—> Determined by Eligibility criteria of studies for a specific research question —> PICO (Population, Intervention, Comparison, Outcome)
—> Information sources: Databases (PubMed), Grey literature (Reports by government agencies), Bibliography
—> Search terms: plus different combination
—> Search filters: time period, language, document format
- Select relevant studies
- Extract relevant data
- Explicitly and systematically assess quality (i.e. risk of bias) of individual studies:
- Synthesise results from individual studies
- Meta-analysis
Systematic reviews vs Narrative reviews
Systematic review:
- Systematic + Explicit methods to identify, appraise and synthesise evidence
- 2 independent reviewers —> less selection bias
- May involve Meta-analysis
Narrative review (Summary review):
- NO systematic / explicit methods to identify, appraise and synthesise evidence —> ***Uncertain validity
- **Content experts who pick and choose relevant studies —> **prone to selection bias
- NOT involve Meta-analysis
- Explicitly and systematically assess quality (i.e. risk of bias) of individual studies
1. Use Explicit criteria should include: - Confounding - Selection bias - Information bias
Checklists:
- Jadad score
—> evaluate **quality of reporting of RCT (評核三方面)
—> Randomisation
—> Blinding
—> Account of all withdrawals and dropouts
- **NOT include allocation concealment
- Newcastle-Ottawa Quality assessment scale:
—> for Case-control studies, Cohort studies
—> Selection, Comparability, Exposure
- Sensitivity analysis
- to assess potential influence of studies at high risk of bias —> repeating analysis by excluding them
- many arbitrary assumptions are involved in a meta-analyses:
—> Case definitions
—> Study designs
—> Study quality
—> Type of analysis etc.
- involve testing of different assumptions —> ***robustness of findings to different assumptions suggests that results from the meta-analysis are valid
EQUATOR network
Reporting guidelines for research studies according to study design
- recommend ***required reporting items for systematic reviews
- can be used as a framework to evaluate ***quality of reporting
- PRISMA: systematic reviews
- Identification
- Screening
- Eligibility
- Included - CONSORT: RCT
- STROBE: observational studies
- STARD: diagnostic / prognostic studies
- AGREE: clinical practice guidelines
Reporting biases
When dissemination of research findings is influenced by nature of direction of results
- Publication bias —> positive results more likely to be published
- Time lag bias —> positive results published sooner
- Language bias —> English studies more likely found
- Citation bias —> positive results more likely to be cited
- Selective outcome reporting bias —> outcomes with favourable associations more likely to be reported
解決方法: Funnel plot
Funnel plot
Scatter plot of **effect estimates (OR, RR) from individuals studies against **measure of study size
- Dotted vertical line: Combined effect
- Solid vertical line: Null effect
- Diagonal lines: Triangular region with which 95% of studies are expected to lie without bias / heterogeneity
Precision of effect estimate ***↑ with larger sample sizes
- smaller studies scatter more widely around bottom of funnel
- “small-study effect”: tendency for smaller studies to show larger effects
Interpretation:
Plot should be symmetrical in the absence of bias / heterogeneity between studies
—> 越高(越大sample size) —> 結果(effect estimate)應該越中庸之道 —> 三角形
—> scatter of studies only due to ***sampling variation
-
Asymmetry could be due to
1. **Reporting bias
2. **Heterogeneity
3. Chance —> statistical tests (e.g. Egger test) help to determine likelihood
Funnel plot ONLY suggest potential biases but do NOT address them
- Synthesise results from individual studies: Meta-analysis
Statistical method to **quantitatively **combine results from individual studies
- Usually a ***weighted average of estimates from individual studies
Increase overall ***statistical power of individual studies
—> increases chance of detecting small but potentially important effects
Should only be done when studies are reasonably ***homogeneous (i.e. results from individual studies are reasonably similar / consistent)
Forest plot
- Graphical summary of results from individual studies and meta-analysis
- Aid in ***identification of heterogeneity
- Solid vertical line: Null line
- Horizontal line: 95% confidence interval
- Square:
—> Centre: **Mean effect
—> Size: Proportional to **% weight of study - Diamond (Only一粒): **Combined result
—> Centre + Dotted vertical line: **Pooled estimate
—> Horizontal tips: ***95% confidence interval
How to identify heterogeneity:
—> Use statistical tests —> when there is poor overlap of confidence intervals
1. I^2 statistic
- estimated proportion of variability that is explained by heterogeneity rather than sampling variation
- ***I^2 >30-40% usually suggests significant heterogeneity
- Cochran’s Q (Chi^2) statistic
- weighted sum of squared differences between individual study effects and pooled effect
- has low power; ***P value <0.1 suggests significant heterogeneity - ***Identify potential sources of heterogeneity
Heterogeneity
- Clinical diversity
- variability in participants, interventions, outcomes studied - Methodological diversity
- variability in study design / risk of bias
Dealing with heterogeneity
- Meta-analysis may not be appropriate when studies too heterogeneous
—> combined estimate not a meaningful representation of studies - ***Random effects model in meta-analysis
—> assumes that there are different underlying effects between studies other than sampling variation
—> Fixed effect model: assume that one true effect underlie all studies and that all differences in observed effects are due to sampling error (does not account for heterogeneity between studies)
Strengths and Limitations of Systematic reviews
Strengths:
- Increase statistical power of individual studies to detect small but important effects
- Explicit and more objective summary of evidence compared to narrative reviews
Limitations:
- Limited by quality of individual studies
- Reporting biases
- Meta-analysis may not be appropriate when studies are too heterogenous
Summary
- Relevance:
- PICO - Validity:
- study design —> Observation / RCT
- study conduct —> Search strategy, Selection bias, Validity / Risk of bias (Confounding, Selection bias, Information bias), Reporting bias (Asymmetry), Meta-analysis (Heterogeneity)
- study results —> Statistically significant, Clinically significant, STEPS, Conflict of interest
STEPS:
- Safety
- Tolerability
- Effectiveness
- Price
- Simplicity
