Term 3 EBP 2 week 2 module 2 Flashcards
What is the preferred reporting items for systematic reviews and meta-analysis
PRISMA
An evidence based minimum set of items designed to help researchers transparently report systematic reviews and meta analyses which are used to assess the benefits and harms of a health care intervention
A 27 item checklist and a four phase flow diagram
The four phase flow diagram illustrates the flow of information through the different phases of a systematic review, mapping out the number of records identified, included, excluded, and the reasons for exclusions
elements of a SR and Meta - analysis
Anatomy of a systematic review article
A. Introduction
B. Methods
* eligibility criteria, search strategy/study selection, review of individual
studies (Quality Assessment), and data extraction/analysis (evidence
synthesis)
C. Results
* Flowchart on included studies; characteristics of included studies (aka
subjects= not human participants)
* presented in tables & graphs (meta-analysis)
D. Discussion and conclusions
E. Other information (registration and protocol)
F. References
Introduction of a systematic review article
Research question
✓Background of the study
➢What’s known and unknown in the literature
✓Rationale for the systematic review study
✓Objective of the study
Methods
PRISMA # 7 deals with terms, databases with dates of coverage, T or F ?
T
PRISMA # 7 search strategy refers to ______ acquire the evidence in EBP steps ( 5 A’s)
ACQUIRE
The 5 steps in EBP are
ASSESS - a questions arises during care of a patient
ASK - completely articulate all parts of the question
ACQUIRE - conduct a thorough focused search and select the highest quality evidence
APPRAISE - evaluate the evidence for validity and clinical applicability
APPLY - use clinical expertise to integrate applicable evidence with attention to patient values
search engines for general medical databases
pubmed
CINAHL complete
Cochran library
search engines for PT specific databases
physiotherapy evidence database (Pedro)
PTNow
retrieved too many citations. How can
I limit my search?
Replace general search terms with
more specific ones (e.g., “low back
pain” instead of “back pain”).
* Include additional terms in your
query.
* Use the sidebar filters to restrict
results by publication date, full text
availability, article type, and more.
I retrieved too few citations. How can I
expand my search?
On the abstract page for a citation,
see the Similar Articles section for
a pre-calculated set of additional
PubMed citations closely related to
that article.
* Remove extraneous or specific
terms from the search box.
* Try using alternative terms (or
synonyms) to describe the
concepts you are searching.
Summary of Finding the Evidence
- Determine which database will be most useful or efficient for
your search (e.g., PubMed, PEDro) - Identify search terms (hip, aquatic) and their synonyms to use
in the search (e.g., ‘femur’ for ‘hip,’ ‘hydrotherapy’ for ‘aquatic’) - Use the search engine features to expand or limit your search
(e.g., ‘OR,’ ‘AND’ etc.) - Be prepared to reformulate your question (if too many/few
hits) or work with evidence that is indirectly related to your
question (e.g., clinic- vs. home-based) - Aim for the highest quality articles you can find (e.g., evidence
hierarchy developed to expedite the process)
Which evidence to choose?
Relevance to the question posed
* Consistency with contemporary practice
* Ranking on “Evidence Hierarchy” / Level of Evidence
* Peer-reviewed
* Credentials of authors, editors or journals
* Disclosure of funding sources or conflicts of interest
METHODS: Risk of bias in individual studies
(Assessing methodologic quality)
Intervention study (e.g., randomized controlled trials)
✓PEDro scale
✓Cochrane Risk of Bias (RoB) Tool
* Other types of study design
✓Diagnosis: QUADAS
Quality Assessment of Diagnostic Accuracy Studies
✓Prognosis: QUIPS
Quality In Prognostic Studies
GARBAGE in GARBAGE out
Pedro scale what is it ?
PEDro scale is a tool to quantitatively evaluate the methodologic rigor
and quality of an intervention study.
* There are a total of 11 criteria.
✓criterion 1: relates to the external validity (equivalent to “generalizability” or
“applicability” of the trial)
✓criteria 2-9: internal validity
✓criteria 10-11: statistical validity
Internal validity
the extent to which a study can confidently attribute its findings to the independent variable and not to other confounding or extraneous variables
threats to internal validity
selection bias - groups being compared are not equivalent at the start of the study
History effects - events outside the study that occur during the study period could affect the results
Maturation - changes in participants over time (aging/development) could be mistaken for the effect of the treatment
Attrition - participants dropping out of the study, potentially leading to a biased sample
Testing effects - participants responses on a test might be affected by having taken the test before
Probably more
How to improve internal validity
control groups, random assignment, blinding
external validity
refers to extent to which the findings of a study can be generalized or applied to other populations, settings, and situations
Its about if the results of a study are likely to be true or applicable in the real world, beyond the specific context of the study
threats to external validity
selection bias - the sample used in the study is not representative of the population to which the researchers want to generalize
Setting effects - the study is conducted in a unique or artificial setting, making it difficult to apply the results to other settings
Time effects - the study is conducted during a specific time period, and the results might not be applicable to other time periods
how to improve external validity
use representative samples, conduct studies in various settings, consider the real world relevance of the research question
PEDro scale questions , how many questions ?
11
PEDro scale has 11 questions, do we count all 11 ?
No, the first one relates to external validity
1. Eligibility criteria were specified
how does one assign points when grading on PEDro scale ?
Points are only awarded when a criterion
is clearly satisfied.
* If it is unclear or possible that a criterion
was not satisfied, a point should NOT be
awarded for that criterion.
* Assign point (Yes = 1 or No = 0) for all
criteria AND justify your assigned points
w/ “where”
PEDro # 9 deals with intention to ______ (ITT)
ITT analysis includes every subject who is randomized according to
randomized treatment assignment.
* ITT analysis maintains prognostic balance generated from the original
random treatment allocation.
* It ignores noncompliance, protocol deviations, withdrawal, and
anything that happens after randomization.
* ITT analysis is usually described as “once randomized, always analyzed”.
PEDro scale should be used with caution, why ?
- The PEDro scale should not be used as a measure of the “validity” of
a study’s conclusions. - In particular, users are cautioned that
✓studies which show significant treatment effects and which score highly on
the PEDro scale do not necessarily provide evidence that the treatment is
clinically useful.
In summary,
✓Note that statistical significance ≠ clinical significance AND
✓Treat PEDro scale as a screening tool
METHODS: Risk of bias in individual studies
(Assessing methodologic quality)
* Intervention study (e.g., randomized controlled trials)
✓PEDro scale
✓Cochrane Risk of Bias (RoB) Tool
https://www.bmj.com/content/bmj/343/bmj.d5928.full.pdf
Revised version (RoB 2) is available;
* Other types of study design
✓Diagnosis: QUADAS
Quality Assessment of Diagnostic Accuracy Studies
✓Prognosis: QUIPS
Quality In Prognostic Studies
METHODS: Risk of bias in individual studies
(Assessing methodologic quality)
* Intervention study (e.g., randomized controlled trials)
✓PEDro scale
✓Cochrane Risk of Bias (RoB) Tool
https://www.bmj.com/content/bmj/343/bmj.d5928.full.pdf
Revised version (RoB 2) is available;
* Other types of study design
✓Diagnosis: QUADAS
Quality Assessment of Diagnostic Accuracy Studies
✓Prognosis: QUIPS
Quality In Prognostic Studies
Risk of bias: Cochrane Risk of Bias Tool (RoB)
Multiple domains and sources of biases:
✓selection,
✓performance,
✓detection,
✓attrition,
✓reporting, and
✓other biases
Meta analysis is an extension of the systematic review that incorporates a statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the finding, T or F
T
When studies meet criteria for __________, a meta-analysis can be
appropriate.
homogeneity
all studies must be similar in the study characteristics (design and
methodology; PICOS components)
✓the outcomes are measured in similar ways
✓statistic used to evaluate this criterion is chi-square: a non-significant test
result indicates that there is common treatment effect across studies.
✓otherwise, systematic review may be sufficient to qualitatively synthesize the
information (thus a qualitative SR).
Homogeneity: the consistency of results of individual studies included in SRs.
RESULTS: Summary of “study/subject*” selection
* Studies of SRs are also called the “subject(s)” in SRs.
RESULTS: Summary of “study/subject*” selection
* Studies of SRs are also called the “subject(s)” in SRs.
In a SR and meta analysis, what is usually in the methods ?
eligibility criteria, search strategy/study selection, review of individual
studies (Quality Assessment), and data extraction/analysis (evidence
synthesis
In a SR and meta analysis, what is usually in the results ?
Flowchart on included studies; characteristics of included studies (aka
subjects= not human participants)
* presented in tables & graphs (meta-analysis)
Results
has some tables like the PRISMA flow diagram and PEDro scale (results of risk bias) and Cochrane RoB
The GRADE system stands for
Grade of Recommendation, Assessment, Development and Evaluation (GRADE) system
What is the GRADE system?
Four levels are generated based on factors of study design, bias etc.
1. High
2. Moderate
3. Low
4. Very low
One can even have a GRAPHIC Forest plot , typically only for meta analysis, and this would be in which part of the report ?
RESULTS
It is used to illustrate treatment effect size of the studies
The Cochrane Handbook recommends the
following organization to the discussion section
for SRs
Summary of main results (benefits and harms)
✓Overall completeness and applicability of evidence
✓Description of overall quality of the evidence
✓Potential biases in the review process
✓Explain/interpret agreements and disagreements
with other studies or reviews.
