Critical Inquiry and EBP Flashcards
On the hierarchy of evidence what is level 1 evidence? (highest level)
Evidence obtained from systematic reviews, high-quality diagnostic studies, prospective studies, or randomized controlled trials
On the hierarchy of evidence what is level 2 evidence? (second highest level)
Evidence obtained from lesser-quality diagnostic studies, systematic reviews, prospective studies, or randomized controlled trials (eg, weaker diagnostic criteria and reference standards, improper randomization, no blinding, less than 80% follow-up)
On the hierarchy of evidence what is level 3 evidence? (third highest level)
Case-control studies or retrospective studies
On the hierarchy of evidence what is level 4 evidence? (second lowest level)
Case series
On the hierarchy of evidence what is level 5 evidence? (lowest level)
Expert opinion
On CPGs, what justifies an A level for research?
A preponderance of level I and/or level II
studies support the recommendation. This
must include at least one level I study
On CPGs, what justifies a B level for research?
A single high-quality randomized controlled
trial or a preponderance of level II studies
support the recommendation
On CPGs, what justifies a C level for research?
A single level II study or a preponderance of
level III and IV studies, including statements
of consensus by content experts, support the recommendation
On CPGs, what justifies a D level for research?
Higher-quality studies conducted on this
topic disagree with respect to their conclusions. The recommendation is based on these conflicting studies
On CPGs, what justifies an E level for research?
A preponderance of evidence from animal or cadaver studies, from conceptual models/principles, or from basic science/bench research support this conclusion
On CPGs, what justifies an F level for research?
Best practice based on the clinical experience of the guidelines development team
What is the difference between a dependent and independent variable?
Dependent- what is being measured
Independent-What is being manipulated
If a study is trying to decide is manual therapy is better at decreasing pain levels in patients compared to ultrasound, what are the dependent variables and what are the independent variables?
Dependent-pain levels
Independent-Manual therapy and Ultrasound
What is a P-value?
What is an alpha value?
The chance that the results of a study are due to chance
alpha value is the pre-set P-value that the study puts forward to make if results are statistically significant
The P value has to be lower than the alpha value to be significant
If a study sets an alpha value at .05 and the P value they get after running their numbers comes back at .08 what should be the conclusion?
Results were not statistically significant
What is a type 1 error?
When a study decides that there IS a statistical difference when there is in fact not one (backing a loser)
What is a type 2 error?
What is the best way to avoid type 2 errors?
when a study says there IS NOT a difference when there actually is one (missing a winner)
Increasing the N (number of subjects)
What is an effect size?
What level constitutes a large effect?
Moderate?
Small?
Trivial?
how much better an independent variable is compared to another
Large=.8 or greater
Mod=.5-.7999
small=.2-.4999
trivial=smaller than .2
What would your conclusion be if, in a study, you found an effect size of IV-1 compared to IV-2 to be .456?
There is a small effect size, IV-1 is slightly better than IV-2
What would your conclusion be if, in a study, you found an effect size of IV-1 compared to IV-2 to be .98?
There is a large effect size, IV-1 is much better than IV-2
What would your conclusion be if, in a study, you found an effect size of IV-1 compared to IV-2 to be .03?
There is a trivial effect size, IV-1 may or may not be better than IV-2
What would your conclusion be if, in a study, you found an effect size of IV-1 compared to IV-2 to be .65?
There is a moderate effect size, IV-1 is better than IV-2
What is interrater reliability?
Intrarater reliability?
Inter=reliability between clinicians
Intra=reliability between one single clinician
What is Coen’s Kappa?
What levels equal excellent, good, fair, and poor?
A statistic that measures reliability where 0.0=no reliability and 1.0=perfect reliability
below .4=poor reliability
.4-.6=fair reliability
.6-.75=good reliability
above .75=excellent reliability
If a test has a Coen’s Kappa of .45, how would you interpret this result?
this test has fair reliability
If a test has a Coen’s Kappa of .8, how would you interpret this result?
This test has excellent reliability
If a test has a Coen’s Kappa of .224, how would you interpret this result?
this test has poor reliability
What are likelihood ratios?
How are they interpreted?
a statistic that tells you what to do once you have your test results whether you should rule up the likelihood or rule down
If over 1.0 you should rule up your suspicion
If under 1.0 you should rule down your suspicion
For positive likelihood ratios, what levels indicate no change should be made in suspicion?
What level suggests a small increase in suspicion?
Moderate increase?
Large increase?
1.0=no change in suspicion
1-5.0=small increase
5-10=moderate increase
10+=large increase
For Negative Likelihood ratios, what level suggests a small shift away from your suspicion?
moderate shift away?
Large?
above .2=small
.1-.2=moderate
.1 or below=large shift away
The likelihood ratio associated with your findings after performing tests that are part of a clinical prediction rule is 3.2, how do you interpret this result?
This is a small increase in likelihood so you should raise your suspicion a mild amount
The likelihood ratio associated with your findings after performing tests that are part of a clinical prediction rule is 20, how do you interpret this result?
This is a large positive likelihood ratio, your suspicion should sway in favor if the suspicion by a large order
The likelihood ratio associated with your findings after performing tests that are part of a clinical prediction rule is .42, how do you interpret this result?
This is a small negative likelihood ratio, you should decrease your suspicion but only a small amount
The likelihood ratio associated with your findings after performing tests that are part of a clinical prediction rule is .002, how do you interpret this result?
This is a large negative likelihood ratio, you should sway your suspicion away from your initial hypothesis by a large order
The likelihood ratio associated with your findings after performing tests that are part of a clinical prediction rule is .15, how do you interpret this result?
This is a moderate negative likelihood ratio, decrease your suspicion moderately
The likelihood ratio associated with your findings after performing tests that are part of a clinical prediction rule is 7.8, how do you interpret this result?
This is a moderate positive likelihood ratio, increase your suspicion by a moderate amount
What is the placebo effect?
How can you decrease this effect in a study?
When a person believes something will work and so it does
By having a sham test that looks and feels similar to the real thing
What is the nocebo effect?
When a person has negative beliefs on something and so it causes harm to their care
What is the Hawthorne Effect?
People who know they are being researched naturally try to perform better
What is the Observer Effect?
people who know they are being watched try to perform better
What is the John Henry Effect?
How can you decrease this effect for your research?
a control group believes they are at a disadvantage compared to the test group so they try to work harder
Blind the subjects into what group they are in
What is the Pygmalion Effect? (a.k.a Rosenthal Effect)
How can you decrease this effect for your research?
Expectation of those conducting research effect the outcomes
blind the researchers to decrease this effect
What are predictive values and what do they tell us?
How are predictive values limited in their usefulness?
a statistic that tells us how likely it is that a person who tests positive has the disorder, and how likely it is that a person who tests negative does not have the disorder.
only apply when the
clinical prevalence is identical to that reported in the study
How is sensitivity calculated?
(True Positives)/(true positives + False Negatives)
How is specificity calculated?
(True Negatives)/(False Positives + True Negatives)
How can you calculate the Type 1 error rate?
1-specificity
(If a test has high specificity the type 1 error rate is low)
How can you calculate the type 2 error rate?
1-sensitivity
(if a test has high sensitivity the type 2 error rate is low)
What does the ICF model stand for?
International Classification of Functioning, Disability and Health
According to the ICF model, what are the 3 levels of human function?
In these levels, what are the 3 domains of human function?
functioning at the level of body or body parts, the whole person, and the whole person in their complete environment.
-body functions and structures
-activities
-participation
What percentage of a normal bell curve should fall within one standard deviation?
Two standard deviation?
Three?
68% (or 34.1% +/- from mean)
95% (or 45% +/- from mean)
99% (or 49.9% +/- from mean)
What is Internal Validity?
The degree to which your study tests what it says it’s testing and isn’t influenced by other factors or variables
What are threats to internal Validity?
history (some event—like a pandemic or car crash—happens that changes the outcome), testing (all subjects get better at hop testing because they get practice every time it’s performed), participant selection (the groups aren’t similar), attrition (people drop out of the study)
What is External Validity?
The degree to which the results of a study can be applied to real life
What are threats to external validity?
Hawthorne/Observer/John Henry/Pygmalion effects,
sampling bias (the sample is different from the target population), unrealistic design (expensive equipment or treatment routines no one in a normal clinic can use)
How can you reduce the risk of a type 1 error?
Decrease alpha level
When would you use a one sample T test?
when you are Comparing the mean of a single sample to an expected value
When would you use independent samples T test?
when you are Comparing means of two independent samples
When would you use a paired samples T test?
when you are Comparing means of two dependent
samples (i.e., repeated samples from the same people)
When would you use ANOVA?
when you are Comparing means of three or more
groups
When would you use Pearson’s R or linear regression?
when you are Determining a relationship between
two continuous (quantitative) variables
