Research Methods Midterm Flashcards
Systematic Review
- Structured review of primary studies integrating statistical analysis to the results, if appropriate
- Has scientific protocol with objectives and methods
- May include a meta-analysis
Meta-Analysis
Quantitative synthesis of 2 or more studies to produce a single estimate of the effect of an intervention or exposure
Narrative Review
Qualitative, narrative summaries of evidence on a given topic that involves informal and subjective methods to collect and interpret information, and is usually written by experts in the field
Systematic Reviews vs. Narrative Reviews
Narrative reviews are worse because…
-They are subjective (prone to bias and error)
+No description of the methods use by review
+Literature search may be incomplete or selective
+May not be able to distinguish research from opinion
-They are usually not quantitative
+No estimate of overall effects
+Cannot detect small effects
Systematic Review: Reasons to Perform
- Manage health information efficiently
- Distinguish sense from non-sense
- Provide evidence based health care
- Summarize and integrate results from individual studies
- Determine if new studies are needed
Systematic Review: Role
- Identifies original research
- Facilitates critical appraisal of original research
- Summarizes large body of knowledge
- Estimates treatment/exposure effects
- Promotes evidence-based medicine
- Identifies research gaps
Systematic Review: Steps
- Objectives and Protocol
- Literature Search
- Quality Assessment
- Assemble Data Set
- Data Synthesis
- Perform sensitivity analysis, subgroup analysis, publication bias analysis, if appropriate
- Conclusions
Systematic Review: Steps - Objectives and Protocol
- Objectives/study question (PICO/PECO)
- Inclusion/exclusion criteria for individual studies
- Literature search and data analysis plan
- Can register on PROSPERO website
- No REB required
Systematic Review: Steps - Literature Search
- Database, search strategies, screening/reading studies
- Identification of all relevant studies
- Structured search strategy
- Title/abstract screening
Systematic Review: Steps - Quality Assessment
- Tabulate study characteristics and assess quality
- Study Characteristics (design, participants, interventions, outcomes)
- Study Quality Assessment (RCT, use Cochrane Risk of Bias)
- Judging Methodological Quality (focus on internal validity)
Systematic Review: Steps - Assemble Data Set
-Extraction of data and data entry
-Full text review
-Data Extraction
+Discrepancies resolved by discussion/external referee
+Write to researchers to get additional info
+Estimate data
Systematic Review: Steps - Data Synthesis
-Statistics and heterogeneity
-Qualitative Data Synthesis: Summary information based on words, NOT stats (i.e.. participants, interventions, outcomes, study design)
-Quantitative Data Synthesis: Statistical analysis combining the results of independent clinical trials considered by the analyst to be combinable
+Statistical model can be Fixed or Random Effects
Systematic Review: Interpreting Treatment Effects - Statistical Measures of Effects for Dichotomous Outcomes
-Relative Risk: Relevant event rate
RR = [a/(a+b)]/[c/(c+d)]
-Risk Difference: Absolute difference in event rate
RD = a/(a+b) - c/(c+d)
-Odds Ratio: Relative number of individuals that fulfill the criteria for a given outcome divided by the number who do not
OR = (a/b)/(c/d)
+Approximates RR when baseline probabilities of the outcome are low (less than 0.1-0.2)
-NNT = 1/RD
Systematic Review: Interpreting Treatment Effects - Statistical Measures of Effects for Continuous Outcomes
-Mean Difference: Weighted difference in mean between intervention and control groups
MD = Mean (int) - Mean (cont)
-Standardized Mean Difference: Absolute difference in mean value between intervention and control divided by SD
Systematic Review: Interpreting Treatment Effects - Testing for Heterogeneity
- Inspect scatter in data points and overlap in CI
- Forest Plot - Check results of I2 and chi-squared tests
- I2 estimates percentage of variability in results across studies that is likely due to true differences in treatment effects (not chance)
- Chi-square estimates the variation between results of trials due to chance
Systematic Review: Steps - Perform sensitivity analysis, sub-group analysis, publication bias analysis
-Sensitivity Analysis: Involves repeating analysis in different subgroups of studies (between-study)
+According to methodological quality. publication status, language, etc.
+If overall results do not change with different approaches, then he results of the statistical synthesis is considered robust
-Sub-Group Analysis: Statistical analysis that focuses on a particular sub-group of participants across studies, not sub-groups (within-study)
+According to age, sex, etc.
+Planned in advance (a-priori) in protocol
+Supported by a biological hypothesis
-Publication Bias
+Funnel Plot should look like inverted cone
Systematic Review: Steps - Conclusions
- Evidence based conclusions, future research
- Summarize and publish review
- Make clinical and scientific recommendations
Systematic Review: Tools for Assessing Quality
- AMSTAR
- GRADE
- Others
AMSTAR
Assessment of Multiple Systematic Reviews
-Quality rating that speaks to the credibility of the methods used for systematic review
AMSTAR Steps
- Was an ‘a priori’ design provided?
- Was there duplicate study selection and data extraction?
- Was a comprehensive literature search performed?
- Was the status of the publication used as an inclusion criteria?
- Was a list of studies (included and excluded) provided?
- Were the characteristics of the induced studies provided?
- Was the scientific quality of the included studies assessed and documented?
- Was the scientific quality of the included studies used appropriately in formulating conclusions?
- Were the methods used to combine the findings of studies appropriate?
- Was the likelihood of publication bias assessed?
- Was the conflict of interest stated?
Systematic Review: Key Points for Critical Appraisal
- Are the results valid?
- What are the results?
- Can I apply the results?
Systematic Review: Tools for Reporting
- PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
- Has 27 items covering 7 topics: title, abstract, intro, methods, results, discussion, funding - MOOSE: Meta-Analyses of Observational Studies
Systematic Review: Weaknesses
- Authors decide inputs, processes, and results
- Judgments are prone to random and systematic errors
- Having more than 2 reviewers can help reduce errors and bias
Possible Sources of Error
- Clinical question
- Study selection and inclusion
- Data extraction
- Assessment of study quality
- Assessment of ability to combine studies
- Statistical methods for data synthesis
Clinical Practice Guidelines
-Systematically developed statements which assist clinicians and patients in making decisions about appropriate treatment for specific conditions/circumstances
Clinical Practice Guidelines: Purposes
- Improve quality of care and patient/health care outcomes
- Distil a large body of knowledge into a convenient readily useable format
- Assist patients and clinicians in making decisions and advocating for quality care
- Provide guidance by which to hold the health care system accountable
- Reduce practice variation
- Optimize resource utilization
- Expose gaps in knowledge
Clinical Practice Guidelines: Guideline Development Steps
- Prioritize problems, establish guideline panel
- Identify clinical questions
- Conduct systematic reviews
- Assess the relative importance of outcomes
- Prepare evidence profile: quality of evidence for each outcome and summary of findings
- Assess overall quality of evidence
- Decide direction and strength of recommendation
- Draft guideline
- Consult with stakeholders and/or external peer reviewer
- Disseminate guideline
- Implement guideline and evaluate
Clinical Practice Guidelines: Methodology Decisions
- Convene a multi-disciplinary team
- Select guideline process (AGREE-II)
- Select clinical questions (GRADE)
- Select evidence synthesis and recommendation process (GRADE + Cochrane)
AGREE-II
- Scope and Purpose
- Stakeholder Involvement
- Rigour of Development
- Clarity and Presentation
- Applicability
- Editorial Independence
AGREE-II: Scope and Purpose
-Overall aim of the guideline, the specific clinical questions and the large patient population
AGREE-II: Stakeholders
- Extent to which guideline represent views of intended users
- TEAM: Together Everyone Achieves More
AGREE-II: Rigour of Development
-Process used to gather and synthesize the evidence, methods used to formulate the recommendations and to update them
AGREE-II: Rigour of Development - Process for Developing Recommendations
-C
