EBM Flashcards
External Validity
The confidence with which the results can be accurately generalized to the population
What do Inclusion and Exclusion Criteria do to validity?
Improves internal validity by controlling variability
Decreases external validity by limiting generalizability
Probability Sampling
When each member of the population has a known chance to be included in the research
Stratified random sampling
To ensure that critical variables are represented in the sample, subjects are stratified by traits and then randomly sampled
Systematic sampling
A type of random sampling done when subjects are put in order. For example, the 3rd person from each letter of the alphabet can be chosen. Can incorporate stratification if there is a meaningful order (i.e. Weight or age)
Purposive sampling
Hand-picked patients. Purpose might be to gather different perspectives
Quota sampling
Convenience sampling, with specific quotas included. This is like stratification
Healthy-User Bias
May occur with convenience samples because healthier people are more likely to volunteer, or be seeking healthcare
Berkson’s Bias
Population selected from an impaired or diseased group (like hospitalized pts)
Exclusion Bias
Excluding patients with certain characteristics. I.e comorbidities, racial groups, socioeconomic groups, people who have dropped out of a study
Internal validity
Internal health of the study. How free it is from bias and chance.
External validity
The degree to which you can generalize the results of the study back to the population of interest.
How to prevent selection bias
Random or stratified random assignment
Equivalence can be double-checked for suspicious factors by obtaining relevant demographic or baseline data
Extraneous factors
Other variables in the study that may affect the relationship between the independent and dependent variables
Example: Researchers examined whether relaxation therapy was better than drug therapy for reducing the number of tension headaches in medical students.
Extraneous Factors to control or account for
1. Diet
2. Other medications
3. Severity of tension headaches
Measurement bias
Occurs when measurements are made unequally between treatment groups
Confounding bias
Occurs when 2 factors are associated and the effect of one is confused with or distorted by the effect of the other
Investigator bias
When the investigator ‘knows’ the expected results, so the investigator treats groups differently
Allocation concealment
Prevents investigator bias. The ppl randomizing individuals into groups are blinded as to which subjects go into which group. E.g. Subjects are given numbers (de-identified from researcher) or a 3rd party interacts with subjects during allocation
Investigator blinding
Prevents investigator bias. E.g. Investigator who is providing treatment or making measurements is blinded as to the group.
Hawthorne effect
Subject bias where people change behavior in a study. Effects both internal and external validity
Subject blinding
Solution to hawthorne effect
Placebo group
Subjects don’t know what group they’re in
BUT blinding doesn’t work for long if there is an obvious physical change
Absolute risk
Incidence # of ppl with disease/total number of people at risk for getting disease
Relative Risk/Risk Ratio
‘How many times more likely are exposed persons to get the disease relative to non-exposed persons?’
- Incidence in exposed/incidence in unexposed
- Common metric in studies with similar risk factors but with different baseline incidence rates.
Cross-over trials
Advantages:
- Subjects are their own controls, which reduces inter-subject variability
- Randomization removes order effect
- Statistical advantages
Disadvantages:
- Inappropriate when time has an effect (e.g. Recovery from surgery)
- Inappropriate when order effect is lasting
Absolute Risk Difference/attributable risk
How much extra disease is actually caused by an exposure or risk factor
Subtract absolute risk of unexposed from absolute risk of exposed
Relative Risk
Ratio between absolute risk of exposed and absolute risk of non exposed
I.e. Ratio between incidence of exposed and incidence of non exposed
“Big Data” Studies
Subtype of retrospective cohort
Very large N
Data from various sources, though gathered usually without medical intent (billing)
-BEST method for looking for rare side effects or outcomes
Case Control Studies
- generally involves interviewing
- Cases: newly-incident (optimally) cases of target disease
- Controls: ppl without target disease who ‘had the opportunity’ to get exposed, and also in other ways similar to pts.
- Controls are the hardest parts of case-control design
- Case control outcomes are usually odds ratio
- ratio bw odds of exposure in cases/odds of exposure in controls
- very close to relative risk if rate in unexposed people is <1%
Odds ratio
Ratio between the odds of exposure in cases and the odds of exposure in controls
Case series
No controls
NOT population-based
Best for describing emergent diseases, rare outcomes, odd exposures
Sometimes the best evidence available
Misclassification bias
Example: Overworked medical assistants often forget to ask patients whether or not they use alternative medicines and simply check “No” in the electronic medical record. You are studying the effect of alternative medicines on the later development of liver failure.
Pts that are actually taking these medications are classified as not taking them, so somebody could be taking these meds, be misclassified, then develop liver disease, so this bias is therefore increasing likelihood for type II error, where you say there isn’t a difference, then there is one
Compliance bias
People that are compliant with one thing tend to be compliant with other things that are thought to be good for them, causing confounding things
Lead-time bias
Common problem in screening studies: if you LOOK for something, you’ll tend to find it sooner, than if you wait for it to cause symptoms
-pt will live longer with diagnosis even if intervention had no effect
Prevalence
Number of people in a population who have a disease
at a given time
-If illness is chronic, over time prevalence will rise even if incidence is small
-Effective treatment can delay mortality and thus increase
prevalence
-Diseases with high and rapid mortality (even if disease has high
frequency) may have low prevalence because most die quickly
-High prevalence could reflect better treatment and longer survival
Absolute risk reduction
Difference in disease between 2 groups; but usually used in a treatment sense as opposed to an exposure sense.
-How much disease can be prevented/reduced with a given treatment?
NNT= 1/ARR
Number Needed to Treat
1/Absolute Risk Reduction OR 100/ARR
Ex: If a cancer treatment reduces mortality from 50% in untreated group to 40% in treated group, the Number needed to treat would be: 1/ (0.5 - 0.4) = 1/0.1 = 10. You need to treat 10 people with the drug to prevent 1 cancer death.
If RR<1
Adverse outcome is decreased by treatment compared to control
If RR>1
Adverse outcome is increased by the treatment compared to control
If RR=0.5
The chance of a bad outcome is half as likely to occur with treatment as without it
RR=1
Risk is same in both treatment and control groups
Relative Risk Reduction
How much the treatment reduced the risk of bad outcomes relative to the control group who did not have the treatment
[[[[[
Control event rate (#not exposed with disease/total not exposed)
MINUS
Experimental event rate (#exposed w. Disease/total exposed)
]]]]]
OVER
Control event rate
Number needed to harm
100/Absolute risk %
Sensitivity
How well can test detect disease
-True positive/Total with disease
Specificity
How well can the test detect the absence of disease
-True negative/Total people without disease
Pre-test probability
Prevalence
Varies with population/patient
Positive Predictive Value
Of all ppl with positive test, which % actually have disease?
Likelihood Ratio
Likelihood of certain test result in pt with disease
OVER
Likelihood of same test result in pt without disease
Likelihood ratio for a + test
Likelihood of + result in diseased pop/likelihood of same in non-diseased pop
=
Sensitivity/1-specificity
Likelihood ratio for a - test
Likelihood of - result in diseased pop/likelihood of same in non-diseased pop
1-sensitivity/specificity
Hazard ratios
Evaluate one prognostic factor at a time (i.e. How big was the primary tumor?)
- Similar to relative risk: Ppl with prognostic factor Y are X times more likely to die than ppl without this factor
- Assumes that ratio between 2 prognostic curves is fairly stable over time
Primary Prevention
- before exposure
- sanitation
- nutrition
- immunization
- education
Secondary Prevention
- after exposure
- early detection (Screening)
- early intervention
Tertiary Prevention
- after disease process begins
- reverse course of disease
- rehabilitation
- treatment
When is it okay to screen?
- If disease has significant morbidity/mortality
- Prolonged asymptomatic phase
- Effective treatment available
- High prevalence in particular pt population
- If pt accepts test
- If pt can comply to treatment after result of test
- If test has good sensitivity/specificity
- If test is valid and reliable
- If it’s cost-effective
Length-time bias
Slower growing tumors that are less likely to kill are more likely to be detected by one time/periodic/interval screening
68-95-99. Rule
-shorthand used to remember the percentage of values that lie within a band around the mean in a normal distribution with a width of one, two and three standard deviations, respectively; more accurately, 68.27%, 95.45% and 99.73% of the values lie within one, two and three standard deviations of the mean, respectively.
Variance and it’s relationship to standard deviation
Variance is the square of SD
S2= Sum of(X-M)2 / N-1
Where X=value and M=mean
Standard Error
A measure of how far the sample mean is away from the population mean
SEM= SD/sqrt(N)
Used to show variability caused by experimental imprecision
WHEREAS
SD is used to show biological variability
Relationship bw Alpha and Type 1 Error
As alpha decreases (say, from 0.05 to 0.01 —> 95% confidence to 99% confidence), chance of type 1 error DECREASES
Relationship between Alpha and Type II Error
As alpha decreases, the chance for Type II errors increases (I.e., as you make your confidence interval wider, you’re more likely to say there isn’t a difference when there is indeed a difference.)
Random Error
Caused by inherently unpredictable fluctuations in the readings of a measurement apparatus or in the experimenters interpretation of the instrumental reading. This error can occur in either direction.
Systematic Error
Is predictable
Typically consistent or proportional to the true value.
Causes by imperfect calibration of measurement instruments or imperfect methods of observation.
Usually occurs only in 1 direction.
Student T Test
Assumptions: N<30 Random sampling Equal sample size Most useful when comparing 2 sample means
T= M1-M2 / sqrt[(SE1)sq + (SE2)sq]
Want the highest # T as possible, because T represents difference between the means. Large T indicates small variability and high signal.
If T>critical T, then there is a clinically significant difference.
1 Tailed T Test vs 2 Tailed T Test
1 Tailed: A one-tailed test will test either if the mean of exp group is significantly greater than control group (x) or if the mean is significantly less than x, but not both. The one-tailed test provides more power to detect an effect in one direction by not testing the effect in the other direction.
2 Tailed: A two-tailed test will test both if the mean is significantly
greater than x and if the mean significantly less than x. The mean is considered significantly different from x if the test statistic is in the top 2.5% or bottom 2.5% of its probability distribution, resulting in a p-value less than 0.05. 2-tailed is more robust.
Paired vs Unpaired T Test
Paired
-The observed data are from the same subject or from a matched subject and are
drawn from a population with a normal distribution.
Example: Measuring glucose
concentration in diabetic patients before and after insulin injection.
Unpaired
The observed data are from two independent, random samples from a population with a normal distribution.
Example: Measuring glucose concentration of diabetic
patients versus non- diabetics.
ANOVA
Used to compare 3 or more means. Analyzes by using sum of squares.
-tells us that the smallest and largest means likely differ from each other
Mann Whitney U Test
- Nonparametric alternative to 2-sample T-test
- Actual measurements not used- instead, ranks are used
- USE THE LOWER OF THE 2 U STATISTICS
Kruskal-Wallis Test
- Nonparametric equivalent of a 1-way ANOVA on ranks
- used for comparing 2 or more independent samples of equal or different sample sizes
Calculating Chi Square
Expected value for box=
(Total in row * Total in column) / N Participants
Degrees of freedom= (rows-1)(columns-1)
Calculated X2 should be > Critical value
X2= sum of[(obs-exp)sq /expected]
