Interpretating Quantitative Findings and Evaluating Clinical Signficance Flashcards
DO stat results of a study allow for use in the real world?
No the stat results of a study in and of itself do not convey much meaning but the statistical results must be interpreted to be of use to the clinician and other researchers for us int eh real world
What are the 6 important considerations when itnerpreting statistical results
Credibility
Precision
Magnitude
Meaning
Generalizability
Implications
6 Considerations: Credibility
the credibility and accuracy of the results
6 Considerations: Precision
Precision of the estimate of effects
6 Considerations: Magnitude
magnitude of the effects and improtance of the results
6 Considerations: Generalizability
generalizability of the results t the greater population
6 Considerations: Implications
implications of the results for practice, theory, or further research
Interpreting research involves making a …
a series of inferences
Inference
involves drawing conclusions based on limited information, using logical reasoning
What do we infer from study results
“truths in the real world”
Statistical findings are what in regard to the true state of affairs in the real world
stand ins/ proxies
Through inference, study results…
are inferred to be truths in the real world
EBP involves doing what to research evidence when making clinical decisions
integrating the evidence with your decision making
Approach the task of interpretation with what kind of mindset
a critical - and even skeptical one
The onus of the responsibility for providing strong evidence that is credible is on who
the researcher - they should acknolwedge limitations adn discuss good choices
they should be the ones stating and showing that the null hypothesis has no merit
Credibility of quantitative results relies on what
validity
biases
corroboration
proxies and the interpretation
Every part of a study acts as…
a proxy for a real life situation / a representation of reality
Steps of using a sample as a proxy for the real world
Population Construct –Delineationg–> Target population –ID–> Accessible Population –Selection–> Recruited sample–refusal/attrition –> Actual sample
From all of this we infer
What happens as we go from population construct of interest toward our actual sample in regard to inferring from research results
Each step reduces the possibility the same we end up with is truly representative but we want the best possible methods for true inference of population in question
CONSORT
Consolidated Standards of Reporting Trials
Reporting guidelines that have been developed so readers can better evaluate methodological decisions and outcomes
Includes a flowchart for documenting participant flow in a study - is included as a table in most studies nowadays
CONSORT Table
Shows the flow of participants throughout a study
Will show who met inclusion requirements and then who was excluded and then who was left for enrollment into the groups
Randomize into groups and then follow the flow - look who left/attrition and can get attrition rates and see who was not participating in the research
How are credibility and validity related
They link with inference
so it is linked to the amount you can actually infer from the results about the real world
Includes: external, internal, construct, and statistical conclusion validity
Statistical Conclusion Validity
Power
Has to do with the methods and such like sample size and sampling
Construct Validity
how accurately are you measuring the construct of interest
is it an accurate proxy for the thing you want to measure - like BP and stress
External Validity
generalizability - will it replicate through time and space
Hawthorne Effect
change in behavior if being watched - selection, hx, etc
We want to protect against these threats with a control group, baseline testing, randomization, etc
What is the research’s job regarding credibility and bias
- Translate abstracts into appropriate proxies
- Eliminate, reduce, or control biases
- Look out for biases and factor them into assessment about the credibility of the results
What is important to consider regarding biases
what types may be present and how extensive, sizeable, and systematic they are
What is credibility in reference to corroboration
seeking evidence to disconfirm the “null hypothesis”
Do this by determining quality of the proxies that stand in for abstractions, ruling out biases, and seeking corroboration for the results via replication
The best way to corroborate results is…
replication
It will check for type I and II error, rule out biases, corroborate results, and show how good researcher choices were
Results from statistical hypothesis tests indicate whether…
a relationship or group difference is probably “real”
How helpful is the p-value with looking how precise/precision of results
p-value in hypothesis testing offers information that is important (whether the null hypothesis is probably false) but it is incomplete
What is the value to observe in order to determine precision of study results
Confidence Intervals (CIs)
they communicate information about how precise the study results are
Results support the researcher’s hypothesis are described as _____, however…
significant; that does not meant they are clinically significant - they need evaluation on top of that on whether the effects are large and useable in a clinical setting
Any interpretation of meaning of a credible and precise study result requires what
understanding not only the methodological issues but also theoretical and substantive ones
Interpreting stat results are easiest when hypotheses are…
supported (when they are positive results)
Meaning and Causality in Study Results
great caution is needed when drawing causal inferences - especially when the study is non-experimental (and cross sectional)
The critical maxim is…
correlation does not prove causality
What are the greatest challenges to interpreting the meaning of research results
- nonsignificant results
- serendipitous significant results
- results contrary to the hypotheses
Because statistical procedures are designed to provide support for research hypotheses through the rejection of the null hypotheses, it is very hard to…
test a research hypothesis that is a null hypothesis
Clinical Significance
The practical importance of research results in terms of whether they have genuine, palpable effects on the daily lives of patients or on the health care decisions made on their behalf
do the research results have meaning to use in the practice environment
Practical Significance (Group Level Clinical Significance)
Typically involves using stat information other than p values to draw conclusions about the ufefulness of research findings
What are the 3 most widely used statistics for group level clinical significance (as well as being unique to EBP)
- Effect Size (ES) indexes
- Confidence Intervals (CIs)
- Number needed to treat (NNT)
Clinical significance at the individual level involves..
ESTABLISHING A BENCHMARK (or threshold) that designates the score value on a measure (or the value of a change score) taht would be considered clinically important
This is a situation where we look at scores over time rather than compare two groups - so we need conceptual definitions of clinical significance and operationalization of clinical significance establishing the MIC benchmark
The focus of clinical significance at the individual level is on…
individual change scores (over time) rather than differences between groups
T/F: Statistical results provide the most meaningful means of communication about a study’s results
False
Rationale: The stat results do not in and of themselves communicate much meaning- they must be interpreted to be of use to others
T/F: A researcher supports inference that he or she wishes other to make, based on the research results, by ensuring study validity
True
Rationale: Inferences of the type the researcher wishes people to make are supported by rigorous methodological decisions, minimization of threats to study validity, good proxies or stand-ins for abstract constructs, elimination or reduction of bias, and efforts to find corroborating evidence.
T/F: In a nonexperimental study, correlation and causation are the same
False
Rationale: In a nonexperimental study correlation does not prove causation
