76. Statistics – errors in the interpretation of data from clinical trials Flashcards
What do we mean by evidence-based medicine?
Decisions regarding the care of patients must be made through the diligent,
unambiguous and thoughtful use of current best evidence. Evidence-based
medicine is an exhortation to integrate individual clinical proficiency with the
best available evidence from systematic research.
What is the null hypothesis?
A hypothesis is a statement of belief about how something occurs.
The hypothesis is generated from the research question,
can only be disproved and cannot be proved with certainty,
e.g. there is no difference between drug X and placebo
What are alpha and beta errors?
Type I or alpha:
A difference is found statistically where none exists
The null
hypothesis is wrongly rejected (false-positive). There is no
difference between drug X and placebo, but a statistical
difference is found.
Maximum p value is usually taken to be 0.05. Whatever the
value of p, however, there will always be a random chance of
making a Type I error (although the lower the p-value is, the
less likely this becomes). Confidence level is (1-alpha).
Type II or beta:
The null hypothesis is false in reality but the p-value
obtained is ≥0.05. We have incorrectly concluded that the
sample groups are similar – we have missed a real difference
The main cause of Type II errors is inadequate sample size.
This is a false-negative, e.g. drug X is found not to be superior to placebo when it is.
Power (1-beta) is a measure of the trial detecting a difference
if one exists. Most editors of scientific journals require the
power of a study to be at least 80% and sometimes 90%.
How do we judge the usefulness of a clinical test?
To be truly useful, a clinical test must positively identify those who have a
disease as well as positively exclude those who do not.
The methods of quantifying these measures are called the sensitivity and
specificity.
sensitivity
The sensitivity is a measure of how good the test is at correctly identifying
those patients afflicted with the disease. It is defined as the number of
patients who test positive as a fraction of those who really have the
abnormality, i.e. the proportion of positives that are correctly identified by
the test.
True positives Sensitivity = \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
True positives + False-negatives
specificity
The specificity is a measure of how good the test is at excluding those
patients who do not have the condition.
It is defined as the number of patients who test negative as a fraction of those who do not have the
abnormality, i.e. the proportion of negatives that are correctly identified by
the test
True negatives Specificity = \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
True negatives + False-positives
What are the positive predictive values of a test?
The positive predictive value quantifies how an abnormal result of a test
predicts a true abnormality.
It is defined as the number of patients who both test positive and who
really are positive as a fraction of the total with a positive test, i.e. the
proportion of those with a positive test who are correctly diagnosed
The negative predictive value
The negative predictive value quantifies how a normal result of a test
excludes an abnormality.
It is defined as the number of patients who both test negative and who
really are negative as a fraction of the total with a negative test, i.e. the
proportion of those with a negative test who are correctly diagnosed.
What is the difference between a systematic review and a meta-analysis?
A systematic review is the formal process of identification, appraisal and
evaluation of primary studies and other relevant research to draw
conclusions about a specific issue.
A meta-analysis
A meta-analysis is the statistical discipline of assimilating data from similar
smaller studies to measure an overall effect size with improved precision.
Commonly, it is invoked as part of a systematic review of the available
literature.
Problems with meta-analysis
Significance
Greater propensity for studies with positive or statistically significant results to be published by scientific journals
Replication
Occurs when the same data are published in multiple articles.
Language Occurs due to failure to search for articles other than
in English.
Selection Occurs when citations are specifically derived from
articles such as narrative reviews or expert opinion
Statistical heterogeneity
For studies to be combinable,
they should demonstrate homogeneity or similarity particularly
with respect to the subjects,
pre-test variables and methodology.
Combining heterogeneous studies may lead to irrelevant and erroneous conclusions
Sensitivity analysis
Sensitivity analysis
This involves checking to see whether alterations of the analyses by the
omission of trials originally included in the meta-analysis materially affect the overall result.
