Stats Flashcards
EBM
the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. The practice of EBM means integrating individual clinical expertise with the best available external clinical evidence from systematic research
Advantages of EBM
improves clinical appraisal skills
leads to the abandonment of unhelpful practices
makes the process of decision making transparent to patients and colleagues
leads to a better appreciation of uncertaintly in clinical practice
Disadvantages of EBM
sometimes is impossible
time consuming and expensive
excessively rigorous criteria may lead to useful papers being disregarded
Evidence can be manipulated
evidence is frequently out of date
RCTs are treated a gold standard, but in some settings other designs are more appropriate
Good clinical practice
n interntational standard for the design, conduct, performance,
monitoring, auditing, recording, analyses, and reporting of clinical trials that provides
assurance that the data and reported results are credible and accurate, and that the rights,
integrity, and confidentiality of trial subjects are protected
Levels of evidence (NHMRC)
1 - systematic review of all relevant RCTs
2 - RCT
3/1 - pseudorandomised trial of high quality
3/2 - cohort study or case control study - with a control group
3/3 - cohort study with historical controls or no control group
4 - case series
Grades of recommendation
A - consistent with level 1 studies
B - consistnet with level 2 or 3 studies, or extrapolations from level 1
C - level 4 studies or extrapolations from level 2 or 3
D - level 5 (oxford definitions - expert opinion) or inconsistent studies of any level
CONSORT
consolidated standards for reporting trials
- governs the reporting of trial data
- designed to produce literature with the highest degree of transparency
Absolute risk
Actual event rate in the group
Essentially is the incidence rate
Number of cases in group/total number of group
Absolute risk reduction
AR in exposed - AR in unexposed
NNT
1/ARR
relative risk
the difference in event rates between 2 groups, expressed as proportion of the event rate in the untreated group
AR in treatment group/AR in control group
relative risk reduction
1 - relative risk
attributable risk
a measure of the absolute effect of the risk of those exposed compared to the unexposed -
INcidence (exposed) - incidence( unexposed)
sensitivity
the ability of a test to correctly identify those with the disease (true positive rate)
true positives / (true positives + false negatives)
specificity
the ability of the test to correctly identify those without the disease (true negative rate).
true negatives / (true negatives + false positives)
POsitive predictive value
true positives / total positives
Negative predictive value
true negatives / total negatives
positive likihood ratio
sensitivity / (1- specificity)
negative liklihood ratio
specificity / (1 - sensitivity)
power
the probobility that a statistical test correctly rejects the null hypothesis when the null hypothesis is false
1 - false positive rate
or
1 - beta error
prevelance
number of affected individuals / total number in population
incidence
number of affected individuals / total exposed population
phases of clinical trial
in vitro activity
Animal model
Phase 1 - healthy volunteers
Phase 2 - patients with the disease of interest
Phase 3 - large scale trial on the patients with disease
Pahse 4 - post marketing experience
intention to treat analysis
once randomised, always randomised
preserves the bias-protective effect of randomisation
minimises type 1 error (false positives)
confidence interval
the range of values between which the actual result is found
gives an indication of precision of the sample mean as an estimate of the true population mean
p value
probability of the observed result occuring by change
Type 1 error
false positive
the null hypothesis is incorrectly rejected (there is no treatment effect but the study finds one)
alpha value determines the risk of this happening
Type 2 error
false negative
the null hypotesis incorrectly accepted (there is a treatment affect but the study doesnt detect it)
beta value
usually set at 0.8 (20% chance of a T 2 error)
determinants of sample size
alpha value
beta value
statistical test you plan to use
varience of the population (greater varience, larger sample needed)
effect size (smaller effect size, larger sample needed)
Methods of maintaining balance in several prognostic variables (ie way to ensure that pre-identified confounding factors are equally distributed
stratification
- separate randomisation list is generated for each prognostic subgroup.
- Usually limited to 23 variables
Minimisation algorithm
- maintains a running total of the prognostic variables in patients that have already been randomised and then subsequent patients are assigned using a weighting system that minimizes imbalance in those prognostic variables.
Sources of symmetry in funnel plots
Reporting biases
- Delayed publication (also known as time lag or pipeline) bias
- Location biases (eg, language bias, citation bias, multiple publication bias)
- Selective outcome reporting
- Selective analysis reporting
Poor methodological quality i.e. smalle studies inflated the effect size
- Poor methodological design
- Inadequate analysis
- Fraud
True heterogeneity
- Size of effect differs according to study size
(eg, in smaller studies the intervention was less intense: eg. PROSEVA trial)
Artefactual
- In some circumstances, sampling variation can lead to an association between the intervention effect and its standard error
Chance
- Asymmetry may occur by chance, which motivates the use of asymmetry tests
What is a funnel plot
a simple scatter plot of the intervention effect estimates from individual studies against some measure of each study’s size or precision (effect vs study size)
It is used to identify publication bias
What do the outer dashed lines and solid vertical line of a funnel plot mean
Outer dashed lines-triangular region where 95% of studies are expected to lie
Solid vertical line- no intervention effect, corresponds to an OR of 1.00.
External validity:
the extent to which the study results can be generalised to the greater population
Factors that affect external validity
The setting and the population from which the sample was selected
The inclusion and exclusion criteria
The “randomness” of the sample, and the baseline chacteristics of the patients
The difference between the trial control group and the routine practice
The changes in practice since the publication of the trial
The use of patient centered outcomes
The degree to which the surrogate outcome measures are related to patient-centered outcomes
Bias
a systematic error which distorts study findings
It is caused by flaws in study design, data collection or analysis
It is not altered by sample size (increasing sample size only decreases random variations and the influence of chance)
It can creep in at any stage in research, from the literature search to publishing of the results.
Selection bias:
The selection of specific patients which results in a sample group which is not random, and which is not representative of a population.
This can be avoided by randomisation, and allocation concealment.
Advantages of meta analysis over review of single studies
↑ Statistical power by ↑ sample size.
Resolve uncertainty when studies disagree.
Improve estimates of effect size.
Inconsistency of results across studies can be quantified and analysed e.g. heterogeneity of studies, sampling error.
Presence of publication bias can be investigated.
Establish questions for future RCTs.
May provide information regarding generalisability of results.
A good meta-analysis has:
Research questions clearly defined
Transparent search strategy
Thorough search protocol
Authors contacted and unpublished data collected
Definition of inclusion and exclusion criteria for studies
Sensible exclusion and inclusion criteria
Assessment of methodological quality of the included studies
Transparent methodology of assessment
Calculation of a pooled estimate
Plot of the results (Forest Plot)
Measurement of heterogeneity
Assessment of publication bias (Funnel Plot)
Reproduceable meta-analysis strategy (eg. multiple reviewers perform the same meta-analysis, according to the same methods)
NHMRC levels:
Level I: systematic review of RCTs
Level II: RCT
Level III-1: pseudorandomised trial of high quality
Level III-2: cohort studies or case control studies - but with a control group
Level III-3: cohort studies with historical controls, or no control group
Level IV: case series
Confidence interval:
CI gives a range of results and the percentage chance that the same experimental design would produce results within this range if the experiment were repeated.
What is a ROC curve and what does it show
The ROC curve is a plot of sensitivity versus false positive rate (1-specificity) for all observed values of a diagnostic test.
It is a graphical representation of a tests’ diagnostic accuracy
It allows the comparison of accuracy between tests
It allows the determination of cutoff values
It can be used to generate confidence intervals for sensitivity and specificity and likelihood ratios.
Advantages of a ROC curve
- Simple and graphical
- Represents accuracy over the entire range of the test
- It is independent of prevalence
- Tests may be compared on the same scale
- Allows comparison of accuracy between several tests.
What is the role of meta-analysis in evidence based medicine?
It offers an objective quantitative appraisal of evidence
It reduces the probability of false negative results
The combination of samples leads to an improvement of statistical power
Increased sample size may increase the accuracy of the estimate
It may explain heterogeneity between the results of different studies
Inconsistencies among trials may be quantified and analysed
Significance of intention to treat analysis:
All enrolled patients have to be a part of the final analysis
Maintains prognostic balance generated from the original random treatment allocation (preserving the bias- reducing effects of randomisation)
Avoids overoptimistic estimates of the treatment’s efficacy
Accurately models the effect of noncompliance and protocol deviations in clinical practice
Prevents bias introduced due to outcome-associated dropouts
Prevents bias by resisting the post-hoc manipulation of data to eliminate inconvenient outcomes
Preserves the sample size, thus preserving the statistical power
Minimises Type 1 error
Allows for the greatest external validity
Supported by the CONSORT statement
Essential for a superiority trial
Significance of randomisation:
Minimises selection bias
Minimises group heterogeneity
Controls unknown confounders, which should be randomly and evenly distributed among the groups
Allows probability theory to be used to express the likelihood that chance is responsible for the diffences in outcome among groups.
Failure to use random allocation and concealment of allocation were associated with relative increases in estimates of effects of 150% or more.
parametric and non-parametric tests use
To determine P value
parametric test uses
Parametric tests are more accurate, but require assumptions to be made about the data, eg. that the data is normally distributed (in a bell curve). If the data deviate strongly from the assumptions, the parametric test could lead to incorrect conclusions.
Non parametric test uses
Non-parametric tests make no assumptions about the distribution of the data. If the assumptions for a parametric test are not met (eg. the distribution has a lot of skew in it), one may be able to use an analogous non-parametric tests.
Non-parametric tests are particularly good for small sample sizes (<30). However, non-parametric tests have less power.
Examples of parametric tests:
Students T Test
Analysis of variance
Pearson correlation coefficient
Regression or multiple regression
Examples of non-parametric tests:
Mann-Whitney U test Wilcoxon sum test Wilcoxon signed-rank test Kruskal-Wallis test Friedman's test Spearman's rank order
Dimensions of evidence to be assessed in systematic review (as per NHMRC)
Strength of evidence
- level of evidence
- quality of evidence
- statistical precision
Size of effect
Relevance of evidence
Bland-Altman plot
a difference plot
a graphical method to compare two measurements techniques.
the ratios between the two techniques (test 1 - Test 2) are plotted against the averages of the two techniques.
Types of bias
Selection - selection of patients not random, avoided by randomisation
Detection - results pursued more diligently in internvention, avoided by blinding
Observer - investigator makes subjective decisions about outcome, avoid by blinding Ix
Response - if patients self enroll in trial - avoid by random sampling
Recall - pts report different outcomes knowing their allocation, avoid by blinding Pt
Publication
Hawthorne effect
Interpretive - there are man y types, but it refers to the way the reader interprets the findings of the trial, usually to confirm ones own beliefs
Propensity score analysis
statistical method to control for selection bias in observational studies
Allows observational studies to be conducted to mimic some characteristics of RCTs
If a group of subjects have the same propensity score, they should all have same baseline characteristics
Ways to randomise
Simple
Block
stratification - pre-identified confounding factors are equally distributed between groups
Minimisation algorithm - an alternatie to stratification to maintain the balance of confounders/prognostic variables between groups
- the algorithm maintians a running totoal of the variables of patients that have already been randomised and uses a weighting system to allocate subsequent patients
Blinding vs allocation concealment
Allocation concealment occurs before randomisation- it ensures that no one can predict which group the next patient withh be randomised to
Blinding prevents the Ix and the patients from knowing who is getting the treatment, after the patient has been allocated
Sources of asymmetry in funnel plots
reporting bias
Poor methodological quality
True heterogeneity
Chance
