Review Slides plus class review Flashcards
A narrative in the professional literature that identifies a single incident and discusses pertinent factors related to the patient
Brings a novel or unusual patient to the attention of colleagues
Information is preliminary and unrefined in terms of research methodology
Important nonetheless
Case Report
Case Report
single incident and discusses pertinent factors related to the patient
novel or unusual patient
info preliminary / unrefined
this type of study analyzes a number of individual cases that share a commonality
-Usually relatively low numbers of subjects
Case Series
Case Series are used to
Examine adverse events or effects
Catalog new diseases or outbreaks
Determine the feasibility or safety of a new treatment or intervention
Discuss the potential efficacy of a new treatment
Data does not necessarily extrapolate to larger populations
Evidence may be circumstantial
Confounding factors may be present
Case reports and case series lack “sufficient methodological rigor”
But – both typically indicate the need for further study
Examine the relationship between exposures and diseases as measured in a population rather than in individuals.
Ecologic Studies
Can often be done by utilizing data from surveys or registries without having to interview, examine, or even identify individual subjects.
Ecologic Studies
After describing an association at the population level, the next step would be to do a an analytic study to see if the association holds true in individuals.
Ecologic Studies
Is a type of bias specific to ecological studies. Occurs when relationships that exist for groups are assumed to also be true for individuals.
Ecological Fallacy
Examines the relationship between outcomes and other variables of interest as they exist in a defined population at one particular time.
Determines prevalence (% of population) not incidence (rate)
Enrolls a large number of individuals
“The chicken or the egg?”: cannot show causality, does not separate cause/effect
Does not establish a temporal relationship between risk factors and disease because they are measured at the same time
Cross-sectional studies
Examines the relationship between outcomes and other variables of interest as they exist in a defined population at one particular time.
Cross-sectional studies
Determines prevalence (% of population) not incidence (rate)
Cross-sectional studies
can a cross-sectional study show causality?
NO!!! does not separate cause and effect
Look at slide 8 for the flow chart of the cross-sectional study
Boy you betta!
Strengths-
Can assess multiple outcomes and exposures simultaneously
Can be completed quickly
Data generated can lead to further studies
Can generate prevalence
Cross-sectional studies
Strengths of Cross-sectional studies?
Can assess multiple outcomes and exposures simultaneously
Can be completed quickly
Data generated can lead to further studies
Can generate prevalence
Limitations-
No time reference
-“Snapshot In Time”- like looking at a photograph
Only useful for common conditions
Cannot calculate incidence, it is a prevalence study
Results are dependent on the study population
We assume that the exposure rate is constant over time.
Cross-sectional studies
Cross-sectional studies Limitations?
Limitations-
No time reference
-“Snapshot In Time”- like looking at a photograph
Only useful for common conditions
Cannot calculate incidence, it is a prevalence study
Results are dependent on the study population
We assume that the exposure rate is constant over time.
Studies in which patients who already have a specific condition (cases) are compared with people who do not have the condition (controls).
The researcher looks back to identify factors or exposures that might be associated with the illness.
This type of study design may follow a case-series (as a retrospective look at causes).
Tries to capture the cause and effect relationship by comparing frequency of a risk factor among those how are exposed and not-exposed.
Case-control studies
Case-control studies
Studies in which patients who already have a specific condition (cases) are compared with people who do not have the condition (controls).
The researcher looks back to identify factors or exposures that might be associated with the illness.
This type of study design may follow a case-series (as a retrospective look at causes).
Tries to capture the cause and effect relationship by comparing frequency of a risk factor among those how are exposed and not-exposed.
Studies in which patients who already have a specific condition (cases) are compared with people who do not have the condition (controls).
Case-control studies
The researcher looks back to identify factors or exposures that might be associated with the illness.
This type of study design may follow a case-series (as a retrospective look at causes).
Case-control studies
Tries to capture the cause and effect relationship by comparing frequency of a risk factor among those how are exposed and not-exposed.
Case-control studies
Look at review slide 12/13 for flow chart
don’t be a slacker
Strengths-
Good for studying rare outcomes
Can evaluate many exposures
Ideal for initial, explanatory idea
Simple & fast – we already know the outcomes
Efficient-no waiting for outcome to occur
Inexpensive
Case-control studies
Case-control studies Strengths
Strengths-
Good for studying _____
Can evaluate many ____
Ideal for ___ idea
Simple & fast – we already know the ____
_____-no waiting for outcome to occur
cost?
Case-control studies
Strengths-
Good for studying rare outcomes
Can evaluate many exposures
Ideal for initial, explanatory idea
Simple & fast – we already know the outcomes
Efficient-no waiting for outcome to occur
Inexpensive
Limitations-
Single outcome
High risk for bias
High risk for confounding variables
Other factors may exist that influence outcomes
Can’t determine prevalence
Temporality
- –Can’t make causal interpretations
- –Can’t determine incidence
- –Can’t calculate Relative risk
Case-control studies
Case-control studies Limitations
___ outcome
High risk for ___
High risk for ____ variables
Other factors may exist that influence outcomes
Can’t determine ____
Temporality
- –Can’t make ___ interpretations
- –Can’t determine ___
- –Can’t calculate ____
Limitations-
Single outcome
High risk for bias
High risk for confounding variables
Other factors may exist that influence outcomes
Can’t determine prevalence
Temporality
- –Can’t make causal interpretations
- –Can’t determine incidence
- –Can’t calculate Relative risk
Potential Biases in Case-Control Studies?
Selection Bias
Information Bias
Researcher/Observer Bias
Voluntary Response Bias`
Case-Control Studies
Selection Bias?
Selection Bias: inappropriate selection of cases or controls.
Cases: Can be selected from a variety of sources: Hospitals, Clinics, Registries. If cases are selected from a single source, and risk factors from that facility may not be generalizable to all patients with that disease.
Controls: Ideally, you want controls to come from the same reference population that cases are derived from. An inappropriate control group can have the opposite effect and obscure an important link between disease and its cause.
Case-Control Studies
Information Bias?
Information Bias
Recall Bias (Subject Bias) is the main form of information bias in case-control studies. Occurs when there is a differential recall of exposure between cases and controls.
Case-Control Studies
Researcher/Observer Bias:
Researcher/Observer Bias:
Occurs when the researcher/observer evaluates cases vs controls differentially.
Case-Control Studies
Voluntary Reponses Bias
Voluntary Reponses Bias
Arises when case subjects who think they have been exposed to responds at a higher rate to controls.
case may not be generalizable to all patients with that disease
Selection Bias of the CASE
Potential Biases in Case-Control Studies
where do you want your controls to come from ideally?
Why?
Ideally, you want controls to come from the same reference population that cases are derived from. An inappropriate control group can have the opposite effect and obscure an important link between disease and its cause.
What is the main form of information bias in case-control studies?
Recall Bias (Subject Bias)
Information bias with a recall bias (subject bias) occurs when…
Occurs when there is a differential recall of exposure between cases and controls.
Potential Biases in Case-Control Studies:
Occurs when the researcher/observer evaluates cases vs controls differentially.
Researcher/Observer Bias:
Potential Biases in Case-Control Studies:
Arises when case subjects who think they have been exposed to responds at a higher rate to controls.
Voluntary Reponses Bias
Control Biases:
Process of selecting the controls so they are similar to the cases in certain characteristics, such as age, race, sex, socioeconomic status, and occupation.
Matching:
two subcategories of matching (a type of control bias)?
Individual: For each case selected for the study, a control is selected who is similar to the case in terms of the specific variable.
Group-based: Select controls with a certain characteristic that is identical to the proportion of cases with same characteristic.
Problems with matching:
If you select too many matching characteristics it is difficulty to find an appropriate control.
You lose the ability to study a matched variable.
Offers independent estimates of exposure among different samples of non-cases. Increases strength of the study.
Multiple Controls: Employ multiple control groups
Other Types of Case-Control Studies?
Case-crossover
Nested Case-Control
Case-Cohort
A variant of a case-control study
Each case becomes their own individual control
Used for transient exposures during a discrete occurrence
Case-crossover
A case control study within a large cohort
Typically seen with large enrollment studies
Controls are a sample of individuals who are at risk for the disease/outcome at the time each case of the disease develops.
Nested Case-Control
Same as nested case-control design, expect controls are randomly chosen from the cohort at the beginning of the study.
Case-Cohort
Cohort studies
Cohort – a group of people who share a common characteristic or experience and all remain in the group for a period of time
Cohort Study – an epidemiologic investigation that follows groups with common characteristics, strongest observational study
Prospective – identify a group of patients who are already taking a particular treatment or have an exposure, follow them forward over time, and then compare their outcomes with a similar group that has not been affected by the treatment or exposure being studied
Retrospective – start with a cohort and go back in time to evaluate past exposures to risk factors
– a group of people who share a common characteristic or experience and all remain in the group for a period of time
Cohort
– an epidemiologic investigation that follows groups with common characteristics, strongest observational study
Cohort Study
– identify a group of patients who are already taking a particular treatment or have an exposure, follow them forward over time, and then compare their outcomes with a similar group that has not been affected by the treatment or exposure being studied
Prospective
– start with a cohort and go back in time to evaluate past exposures to risk factors
Retrospective
Look at slide 20 to reinforce cohort studies
Job well done
Potential Biases in Cohort Studies
Selection Bias - (lost to follow up)
Information Bias - quality of info between exposed vs non-exposed AND Observer bias
Potential Biases in Cohort Studies
Selection Bias -
Lost to follow up ?
People with disease are selectively lost to follow-up, and those lost to follow-up differ from those not lost to follow-up.
Potential Biases in Cohort Studies
Information Bias?
Quality and extent of information is different for exposed person than for non-exposed person, a significant bias can be introduced.
OBSERVER BIAS – Occurs when the observer decides whether the disease has developed in each subject also knows whether that subject was exposed.
Strengths of cohort studies?
Temporal relationships identified
Confirm Cause and Effect (& magnitude of effect)
Measures Incidence Rate
- Can calc Relative Risk
- HIGHEST VALIDITY OF OBSERVATIONAL STUDY DESIGN**
Highest validity of observational study design?
measure of the incidence (rate) of disease
Strengths-
May study multiple effects of a single exposure
Can identify a temporal relationship between exposure and disease (outcome)
Help confirm cause and effect of disease and the magnitude of the effect
Can measure incidence (rate) of disease
- –Can calculate Relative Risk
- –Highest validity of observational study design
Cohort studies
Limitations-
Expensive and time consuming
Inefficient for studying rare diseases
Case-control more appropriate for rare diseases
Lose participants to follow-up
Risk of confounding variables
Retrospective studies require presence of records or recall
Cohort studies
Limitations of Cohort Studies?
Expensive and time consuming
Inefficient (rare diseases)
Lose participants (to follow up)
Risk of confounding variables
Require presence of records or recall
Which study is better for rare diseases… cohort or case control?
CASE-CONTROL
Study comparison
Cohort vs. Case Control
Cohort studies:
Start with exposure, look for disease
Prospective or retrospective
Common diseases
High risk for drop out
$$$
Case-Control studies:
Start with disease, look for exposure
Retrospective
Rare disease
Recall and selection bias
$
Randomized control studies
“Randomized” Allocation/Assignment
The main purpose of randomization is to prevent any potential biases on the part of…
the investigators from the influencing the assignment of participants into different treatment groups.
May be done by assigning random numbers or by a program that generates random assignments
Randomized control studies
Each subject has an equal chance of being assigned to each group (control or intervention)
Randomized control studies
Randomized control studies
Randomization strives for comparability of the different treatment groups; however…
its not guaranteed.
Randomized control studies
“Controlled” implies predefined:
Specified hypotheses
Primary and secondary endpoints to address hypotheses
Methods for enrollment and follow up
Eligibility/Exclusionary criteria
Rigorous monitoring
Analysis plans and stopping rules
Randomized control studies
Why Controlled?
Seek to eliminate confounding variables
Attempt to minimize bias
Look at table on slide 28
Did ya?
As far as enrollment,
Criteria for determining selection must be specified
before the study is begun.
As far as enrollment,
Want to ensure that participants actually
have the disease of interest.
As far as enrollment,
Carefully select sample based on
a reference population.
As far as allocation,
______ is the best approach in the design of a trial, and the critical element of _____ is the unpredictability of the next assignment.
Randomization
the critical element of _____ is the unpredictability of the next assignment.
Randomization
As far as allocation,
If conducted properly we don’t have to worry that any….
subjective biases of the investigator, either overt or covert, may be in introduced into the process of selecting patients.
How is randomization accomplished:
Computer programs
Envelope System – The treatment assignment that is designated by a random number is written on a card, and this card is placed inside an envelope. Each envelope is labeled on the outside: Patient 1, Patient 2, Patient 3…..etc. When the first patient is enrolled and consented the investigator opens the envelope and the treatment assignment is determined.
Only open the enveloped after a subject is consented and meets eligibility criteria!
When do we open the envelope in the randomization process of the envelope system?
Only open the enveloped after a subject is consented and meets eligibility criteria!
We hope that randomization achieves comparability of characteristics between the treatment groups
however, this not guaranteed!
– utilized when we are concerned about the comparability of the groups in terms of one or a few important characteristics. This is conducted by stratifying our study population by each variable that we consider important, and then randomize participants to treatment groups within each stratum.
Stratified Randomization
Treatment (Assigned vs Received)
Important to know if the patient was assigned to receive treatment A, but did not comply.
A subject may agree to be randomized, but may later change his or her mind and refuse to comply.
Conversely, it is also clearly important to know whether a patient who was not assigned to receive treatment A may have taken treatment A on his or her own, often without realizing.
As far as Outcome,
Comparable measurements in all study groups.
Improvement
Side Effects or Adverse Reaction
As far as Outcome,
___ ___ ____ for all outcomes to be measured in a study
Explicitly stated criteria
As far as Outcome,
Potential pitfall is outcomes being measured more carefully in…
How is this prevented?
those receiving a new drug than in those receiving currently available therapy must be avoided.
Blinding can prevent much of this problem; however, blinding is not always possible.
Behavioral Interventions
As far as Randomized control studies,
What is blinding?
The concealment of group allocation from one or more individuals involved in a clinical research study.
As far as Randomized control studies,
Usually is used in research studies that compare two or more types of interventions.
Blinding
As far as Randomized control studies,
Blinding is Used to make sure that knowing the type of treatment does not affect:
A participant’s response to the treatment
A health care provider’s behavior
The assessment of the treatment effects
After being observed for a certain period of time on one therapy; any changes are measured; patients are switched to the other therapy.
Planned Crossover
Planned Crossover
Each patient can serve as his or her own ____
control
Planned Crossover
holding constant the variation between individuals in many characteristics that could potentially affect…
a comparison of effectiveness of two agents.
Planned Crossover
Must have a
washout period!
Unplanned crossover
Occurs when subjects who are randomized
cross-over to the other group.
Unplanned crossover
If we analyze according to treatment that the patient actually receive, we will have
broken and therefore lost the benefits of randomization.
Unplanned crossover
Current practice to perform the analysis by ____ _ ___-according to the original randomized assignment.
What happens if bias occurs?
intention to treat
If bias occurs typically biases towards the null; typically provides a more conservative estimate
If bias occurs typically biases towards the null; typically provides a more conservative estimate
Unplanned crossover
Randomized control studies
Blinding types?
Single
Double
Triple
Allocation is concealed from only one group (researchers or subjects)
Single blinding
Randomized control studies
Allocation is concealed from both groups (researchers and subjects)
Double blinding
Randomized control studies
Allocation is unknown to the subjects, the individuals who administer the treatment or intervention, and the individuals who assess the outcomes.
Triple blinding
Randomized control studies
Subjects may agree to be randomized, but following randomization they may not comply with the assignment treatment.
May be Overt or Covert!
Noncompliance
The net effect of non-compliance on the study results will be to reduce __________, because the treatment group will include some who did not receive the therapy, and the no-treatment group may ____________
any observed differences
include some who received the treatment.
Randomized control studies
Strengths?
When combined-
Double-blinded Randomized Control Trial is typically referred to as the Gold-standard
Minimizes the chance for bias if randomization and blinding are done correctly
what is the gold standard of randomized control studies?
Double-blinded Randomized Control Trial is typically referred to as the Gold-standard
Randomized control studies
Limitations-
Large trials (may affect statistical power) Long term follow-up (possible losses) Compliance Expensive Possible ethical questions Primum Non Nocere / ‘First Do No Harm’
– Attempted to learn if the drug, surgical procedure, or administrative program works under ideal circumstance.
Efficacy Trial
– Within the confines of the study, results appear to be accurate and the interpretation of the investigators I supported.
Internal Validity
– Ability to apply results obtained from a study population to a broader population. Also called generalizability.
External Validity
Also called generalizability.
External Validity
External Validity also called…
generalizability.
Look at slide 41 to understand external vs. internal validity
It’s a PHENOMENAL visual reference
Designs that summarize the work of other studies
—-Takes the results of a large numbers of primary research studies and combines them into one
Systematic Reviews and Meta-analyses
The top two levels of the EBM pyramid
Generally represents the strongest evidence
Systematic Reviews and Meta-analyses
Both are subject to bias based on the inclusion and/or exclusion criteria
Systematic Reviews and Meta-analyses
What is the difference between a “systematic review” and a “meta-analysis”?
A “systematic review” is a thorough, comprehensive, and explicit way of interpreting the medical literature
A “meta-analysis” is a statistical approach to combine the data derived from several selected studies
A “_______” is a thorough, comprehensive, and explicit way of interpreting the medical literature
systematic review
A “_______” is a statistical approach to combine the data derived from several selected studies
meta-analysis
Both are used for the development of Clinical Practice Guidelines (CPGs)
Systematic reviews and Meta-analyses
Appraise and synthesize all the empirical evidence to answer a given research question!
Systematic reviews
Systematic reviews
Methodology? (9 Steps)
State objectives and outline eligibility criteria
Search for trials that meet criteria
Establish methods for assessing methodological quality of each study
Apply eligibility criteria and justify exclusions
Assemble the most complete collection possible
Analyze results using synthesis of data
Compare alternate analyses, if necessary
Prepare a critical summary of the review
Restate aims, methods, and results
Systematic reviews
Strengths
Exhaustive review of the current literature and other sources
Less costly to review prior studies than to create a new study
Less time required than conducting a new study
Results can be generalized and extrapolated into the general population more broadly than individual studies
More reliable and accurate than individual studies
Considered an evidence-based resource
Strengths systematic reviews…. things in bold only
can be generalized
evidence-based resource
Systematic reviews
Limitations?
Limitations
Very time and labor consuming
May not be easy to combine studies
greater statistical power
Meta-analyses
A method for combining pertinent study data from several selected studies that are similar enough to justify a quantitative summary to develop a single conclusion that has greater statistical power
Meta-analyses
A statistical synthesis of the numerical results of several trials which all addressed the same research question
Meta-analyses
Meta-analyses
The conclusion is statistically stronger than any single study due to:
increased numbers of subjects
greater diversity among subjects
accumulated effects and results
Meta-analyses
Strengths
Greater statistical power
Confirmatory data analysis
Greater ability to extrapolate to the general population
Meta-analyses
Limitations
Difficult and time consuming to identify appropriate studies
Not all studies provide adequate data for inclusion and analysis
Requires advanced statistical techniques
Heterogeneity of study populations
Age, gender, etc.
Results are not always reproducible by other investigators
Subject publication Bias, Selection Bias, and Misclassification Bias
Can give a false sense of certantiy regarding the magnitude of risk
Difficult
Not all studies provide adequate data
advanced statistical techniques
Heterogeneity
publication Bias, Selection Bias, and Misclassification Bias
Meta-analyses
Limitations
(That were bolded)
Meta-analyses
Limitations
(That were bolded)
Difficult
Not all studies provide adequate data
advanced statistical techniques
Heterogeneity
publication Bias, Selection Bias, and Misclassification Bias
Measures of Central Tendency
Mean, median, and mode
- the “average” – sum of the set divided by the number in the set
Mean
– the middle point (arrange the data smallest to largest, then find the middle point)
Median
– the score that occurs most frequently in a set of data
Mode
May have two most common values = “bimodal distribution”
Mode
Variance compared to standard deviation
quantifies the amount of variability, or spread, around the mean of the measurements.
To calculate?
Variance (σ2 )
To calculate: take each difference from the mean, square it, and then average the result
a measure of variation of scores about the mean
To calculate?
Standard deviation (σ):
To calculate: take the √ of the variance
the “average distance” to the mean
Variance compared to standard deviation
more frequently used?
In practice, the standard deviation is used more frequently than the variance.
Primarily because the standard deviation has the same units as the measurements of the mean.
Variance compared to standard deviation
When comparing two groups, the group with the larger standard deviation exhibits….
… a greater amount of variability (heterogeneous) while the groups with smaller deviation has less variability (homogeneous).
a greater amount of variability
(heterogeneous)
while the groups with smaller deviation has less variability
(homogeneous)
A useful summary of a set of bivariate data (two continuous variables)
Scatterplots
Scatterplots
Gives a good visual picture of the relationship between…
and aids?
the two variables
and aids the interpretation of the correlation coefficient or regression model.
Evaluates the strength of linear relationships or associations between variables
Scatterplots
Scatterplots
How strongly is one variable related to another?
Are BP and weight correlated?
Is HIV risk and # of sexual partners correlated?
Scatterplots
Direct or inverse relationship?
X increases and Y increases = ?
X increases and Y decreases = ?
0 is no correlation
X increases and Y increases = positive correlation
X increases and Y decreases = negative correlation
0 is no correlation
Represented by “r ” (rho)
The absolute value of the coefficient (its size, not its sign) tells you how strong the relationship is between the variables.
Tells us how strongly two variables are related
“r” can not be > 1 or < -1
Closer to -1 or +1: the stronger the relationship
Closer to 0 : the weaker the relationship
Correlation Coefficient
The most common measure of association. Results can misleading if the relationship is non-linear.
Pearson Correlation:
Correlation Coefficient
Pearson’s correlation is very sensitive to?
outlying values.
non-parametric version of Pearson’s correlation. The calculation is based on the ranks of the data points of the x and y values
Spearman Correlation:
The statement that establishes a relationship between variables being assessed
Example: In a clinical trial the hypothesis states the new drug is better the placebo
Alternative hypothesis (Ha or H1)
The statement of no difference or no relationship between the variables
Example: In a clinical drug trial the null hypothesis states that the new drug is no better than placebo
Null hypothesis (Ho)
Hypothesis stating the expected relationship between independent and dependent variables.
If there IS a statistically significant difference, then the researchers “ACCEPT” the alternative hypothesis and “REJECT” the null hypothesis.
Alternative hypothesis
AKA the “research hypothesis”
Alternative hypothesis
AKA the
“research hypothesis”
If there IS a statistically significant difference, then the researchers “ACCEPT” the alternative hypothesis and “REJECT” the null hypothesis.
Alternative hypothesis
AKA the “research hypothesis”
Hypothesis stating the expected relationship between independent and dependent variables.
If there IS a statistically significant difference, then the researchers “ACCEPT” the alternative hypothesis and “REJECT” the null hypothesis.
Alternative hypothesis
AKA the “research hypothesis”
If there IS a statistically significant difference, then the researchers must “REJECT” the Null hypothesis
If there IS NOT a statically significant difference, then the researchers must “RETAIN” or “FAIL to REJECT” the Null hypothesis.
Null hypothesis
If there IS a statistically significant difference, then the researchers must _____ the Null hypothesis
“REJECT”
If there IS NOT a statically significant difference, then the researchers must _____ the Null hypothesis.
“RETAIN” or “FAIL to REJECT”
Two kinds of errors can be made when we conduct a test of hypothesis.
Type I error (α error)
Type II error (β error)
also known as a rejection error or an α error.
Type I error;
A type I error is made if we
reject the null hypothesis when null hypothesis is true.
The probability of make a type I error is determined by
the significance level of the test.
The second kind of error that can be made during a hypothesis test is a Type II error, also known as
an acceptance error or an β error.
A Type 2 error is made if we
fail to reject null hypothesis.
The probability of committing a type II error is represented by
the Greek letter β.
Look at the easy table on slide 60 for null hypothesis made EASY
Look at the easy table on slide 60 for null hypothesis made EASY
The probability of finding an effect
The probability of correctly rejecting the null hypothesis
The probability of seeing a true effect if one exists
Designers of studies typically aim for a power of 80% or 0.8
Implies there is an 80% chance of getting it right
Generally speaking: More people = more power
Statistical Power
A __ __ calculates the number of participants a study must have to draw accurate conclusions
Power Analysis
A power analysis calculates the number of ____ a study must have to draw accurate conclusions
Takes into consideration: estimated effect size, sample means, etc.
Participants
The probability of rejecting a true H0
Signifigance
α = .05 usually set, acceptable error
Chance that 5 times out of 100 the H0 would be falsely rejected
Signifigance
How do we determine if the study result happened by chance alone
Signifigance
What determines statistical “significance”?
Significance
The probability of rejecting a true H0
The likelihood that the difference observed between two interventions could have arisen by chance
Probability Level
If Accepted value is 5% risk (p = .05)
Means there is a 5% chance that the results happened by chance
Allows us to reject or accept the null hypothesis
p is the chance of ___ ___
random error
α is the ___ ___, usually = .05
acceptable error
p ≤ α
reject the H0
the results (are/are not)? statistically significant
p ≤ α
reject the H0
the results are statistically significant
p > α
fail reject the H0
the results (are/are not)? statistically significant
p > α
fail reject the H0
the results not statistically significant
More important than p value – a better determination of significance
Confidence Interval (CI)
Any statistic is simply an estimate of the true value of that statistic
___ ___ produces a range within which the true value most likely lies
Confidence Interval (CI)
95% CI states that we can be 95% certain that the “true” value is within the CI range
Confidence Interval (CI)
Is a wider or narrow CI better?
Narrower CI is better
If the Confidence Interval include 1 (null value) then the results is ___ ___
Clinical insignificant
A ___ ___ is used to separate from a large group of apparently well persons those who have a high probability of having the disease, so that they may be given a diagnostic work up, and if diseased can be treated.
Screening Test
A screening test is used to separate from a large group of apparently well persons those who have a ___ ___ of having the disease, so that they may be given a diagnostic work up, and if diseased can be treated.
High Probability
Screening Tests
In general, screening is performed only when the following conditions are met:
1)
2)
3)
1) The target disease is an important cause of mortality and morbidity.
2) A proven and acceptable test exists to detect individuals at an early stage of disease.
3) There is a treatment available to prevent mortality and morbidity once positives have been identified.
The proportion of people with the disease who have a positive test for the disease.
Sensitivity
The ability of the test to identify correctly those who have the test.
Sensitivity
The proportion of people without the disease who have a negative test.
Specificity
The ability of the test to identify correctly those who do not have the disease
Specificity
Tends to rule OUT the disease
Sensitivity
Tends to rule IN the disease
Specificity
High Sensitivity means low probability of __ ___
False Negative
High Specificity means low probability of __ ___
False Positive
Screening test’s ability to identify presence of disease
A test with high ___ will not miss many patients who have the disease
Sensitivity
Screening test’s ability to truly identify absence of disease
Specificity
A highly useful test when NEGATIVE
Sensitivity
A highly useful test when it is POSITIVE
Specificity
Sensitivity and Specificity
Recap slide 70
graphs on 71-73
Allows us to calculate the net sensitivity and net specificity of using both tests in sequence. After completing both tests there is a loss in net sensitivity and net gain in specificity
Sequential (two stage) testing
slide 74 or square chart
When multiple tests are used simultaneously to detect a specific disease, the individual is generally considered to have tested “positive” if he or she has a positive result on any one or more of the tests. The individual is considered to have tested “negative” if he or she tests negative on all of the tests
Simultaneous tests
some colors on slide 75 for you to peruse
proportion of patients who HAVE the disease and a positive test
Positive Predictive Value (PPV)
proportion of patients who DO NOT HAVE the disease, and have a negative test
Negative Predictive Value (NPV)
Negative Predictive Value (NPV) = proportion of patients who __ the disease, and have a negative test
DO NOT HAVE
Positive Predictive Value (PPV) = proportion of patients who ___ the disease and a positive test
HAVE
Assesses reliability of positive test
i.e. PPV 90% = positive test 90% of the time the test is correct
Positive Predictive Value
With low prevalence (% of population) of disease:
Lower PPV
False positives increase
Less reliable positive test result
PPV
Assesses reliability of a negative test
i.e. NPV 90% = negative 90% of the time the test is correct
Negative Predictive Value (NPV)
With low prevalence(% of pop) of disease :
Higher NPV
False negative test decreased
A negative test result is more reliable
NPV
the occurrence, rate, or frequency of a disease
Incidence
Obtained from cohort studies
Must follow a cohort through time
Incidence
the number of occurrences at one particular time
Prevalence
Obtained from cross-sectional studies
No time line, only a snap shot
Prevalence
Incidence is the occurrence, ___, or frequency of a disease
Rate
Incidence calculations …
slide 80
Relationship between incidence and prevalence
slides 81 to 84
Nominal and Ordinal are:
Categorical Data
Interval and Ratio are:
Continuous Data
named categories with no implied order
Gender, race, ABO blood type, group
Nominal (categorical data)
sequenced or ranked data
Smallest to largest, lightest to heaviest, easiest to most difficult
Ordinal (categorical data)
intervals along the scale are equal to one another (i.e. integers)
Interval (continuous data)
characterized by the presence of absolute zero on the scale
Ratio (continuous data)
are statistical significance and clinical significance the same
NOT the same thing
Once we determine the difference between the expected outcome and the actual outcome was NOT due to chance (statistical significance), we have to decide on ___ ___
Clinical Significance
Clinically unimportant effects may be statistically significant if a study is large
Pay attention to effect size and confidence intervals.
___ ___ addresses How much more likely are we to find that a test is positive among patients with disease compared with those without disease?
Likelihood Ration (LR)
Summarizes the same kind of information sensitivity and specificity and can be used to calculate the probability of disease in a low prevalence setting.
Likelihood Ratio (LR)
Low prevalence = Less reliable positive test result; therefore, use __ __
Likelihood Ratio (LR)
LR provides indication of the test’s discriminatory power.
Predictive values are lower with a low prevalence
LR can be defined for the entire range of test result values
Low prevalence = Less reliable positive test result; therefore, use LR
Likelihood Ratio (LR)
Mnemonic - W/WO
With / Without
Likelihood Ration (LR)
Likelihood of a particular result in someone WITH the disease / Likelihood of the same result in someone WITHOUT the desiease
Likelihood Ratio
A ___ is the ratio of the proportion of diseased people with a positive test result (sensitivity) to the proportion of non-diseased people with a positive result (1-specificity).
positive LR (LR+)
How good the test is at “Ruling in” disease!
Range: 1.0 to infinity; Null value: 1.0 (no difference)
The bigger the better (Desirable: 5 or more)
A negative LR (LR-) is the proportion of diseased people with a negative test result (1-sensitivity) divided by the proportion of non-diseased people with negative test results (specificity)
Negative LR (LR-)
How good the test is at “Ruling out” disease
Range: 0.0 to 1.0; Null value: 1.0 (no difference)
The smaller the better (Desirable: 0.2 or less)
Is one of the most common ways to examine relationships between two or more categorical variables.
Chi-Square
Does Chi-square give any information about the strength of the relationship.
Does Not
Odd ration interpretation
slide 94
More on incidence
slide 95
Basic risk statements express the likelihood that a particular event will occur within a particular population
What exposure is responsible for an illness or other outcome?
Identifies what in our environment can lead to beneficial or adverse medical outcomes
Relative Risk
___ ___ measures the magnitude of an association between an exposed and non-exposed (control) group.
Relative Risk
___ ___ is calculated using cumulative incidence data to measure the probability of developing disease
- Must have incidence information to calculate
- Cohort or clinical trials are conducted over time
Relative Risk
Relative risk formula:
Experimental Event Rate (EER) /
Control Event Rate (CER)
NNT= 1/ARR
Number needed to treat
Expresses the likelihood of the treatment to benefit an individual patient
Number needed to treat
There is NO absolute value for NNT that defines whether something is effective or not.
NNTs for treatments are usually low because we expect large effects in small numbers of people
NNT
When an experimental treatment is detrimental, the term __ __ __ is often used.
The equations and approach are similar to those described above, except that NNH will have a negative absolute risk reduction
Number needed to harm
AKA Student’s t-test
Generally is used to analyze ___ ___
Compares the means and standard deviations of two populations
Continuous data
T-test Computes a ___ to test the null hypothesis
p-value
is that the difference between the two group means is 0 or no difference
Null hypothesis
The difference between the two group means is >0 or there is a difference.
Alternative Hypothesis
what type of Error:
finding an effect that isn’t real
Rejecting the null when the association isn’t real
“Convicting an innocent man to prison”
Type 1 Error
What type of Error:
Type II error: missing an effect that does exist
“Retaining” or “Failing to Reject” the null hypothesis when the association is real
“Not convicting a guilty man”
Type II Error
What type of error is considered worse, Type I or Type II?
Type I error is considered worse than Type II, therefore more important avoid
within the confines of the study results appear to be accurate and interpretation of the results are supported.
Internal Validity
results and interpretations of the study apply outside the studied population. Also called generalizability
External Validity
results and interpretations of the study apply outside the studied population. Also called generalizability
Validity of Data
Degree to which the measurements are reproducible
Example:
How closely do repeated measurements on the same subject agree
Reliability
Errors:
slide 105
Condition, intervention or characteristic that will predict or cause an outcome
Age, gender, & marital status of participants are independent from the outcome
–the experimental treatment doesn’t change these values
Independent Variables
AKA outcome variable
Response or effect that is presumed to vary depending on the independent variable
The measurable “outcome” variable
Example: blood pressure/rate in response to treatment
Dependent Variable
Variables that correlates directly or indirectly with the dependent and independent variables.
Confounding Variables
Confounding factors AKA..
AKA confounding factor, hidden variable, lurking variable, a confound, or confounder
An extraneous variable in a statistical model that correlates (positively or negatively) with both the dependent variable and the independent variable
Confounding factors
- Optimal clinical decision making requires awareness of the best available evidence, which ideally will come from systematic summaries of that evidence.
- EBM provides guidance to decide whether evidence is more or less trustworthy—that is, how confident can we be of the properties of diagnostic test, or our patient’s prognosis, or of the impact of our therapeutic options?
- Evidence alone is never sufficient to make a clinical decision.
3 Fundamental Principles of Evidence Based Medicine (EBM)
5 A’s of EBM
- Assess
- Ask
- Acquire
- Appraise
- Apply
5 A’s of EBM
- Assess
- Ask
- Acquire
- Appraise
- Apply
evidence pyramid, top down…
Filtered
unfiltered
background
see slide 113
Five types of clinical questions:
Therapy Harm Differential Diagnosis Diagnosis Prognosis
Research involving formal, objective information about the world, with mathematical quantification; it can be used to describe test relationships and to examine cause and effect relationships.
Quantitative Research
Research dealing with phenomena that are difficult or impossible to quantify mathematically, such as beliefs, meanings, attributes, and symbols; it may involve content analysis
Qualitative Research
Comparison of design
Qualitative vs Quantitative?
QUALITATIVE
Subjectivity is expected
Descriptive
What is this phenomena?
Low Control
QUANTITATIVE
Objectivity is critical
Experimental
To what extent does A affect or cause B?
High Control
Research Paradigms:Know that qualitative research is not less rigorous or easier –
just different!
Comparison of questions:
Qualitative vs Quantitative:
Comparison of questions
Qualitative: Why do you…? What do you think of…? When is it important…? How does that make you feel…?
QUANTITATIVE Who is at risk for…? What is the effect of…? How many will benefit from…? How much improvement…?
Comparison of methods:
Qualitative vs Quantitative:
Qualitative: Focus Groups Interviews Surveys Self-reports Observations Document analysis Sampling: Purposeful
Quantitative: Observational Experimental Mixed Sampling: Random
Basic level: a descriptive account of the data (i.e. this is what was said, but no comments or theories as to why or how)
Manifest Level
Higher level: a more interpretive analysis that is concerned with the response as well as what may have been inferred or implied
Latent Level
absolute risk reduction?
Absolute risk reduction (ARR) – also called risk difference (RD) – is the most useful way of presenting research results to help your decision-making. In this example, the ARR is 8 per cent (20 per cent - 12 per cent = 8 per cent). This means that, if 100 children were treated, 8 would be prevented from developing bad outcomes.
Another way of expressing this is the number needed to treat (NNT). If 8 children out of 100 benefit from treatment, the NNT for one child to benefit is about 13 (100 ÷ 8 = 12.5).
Absolute risk reduction (ARR) – also called risk difference (RD) – is the most useful way of presenting research results to help your decision-making. In this example, the ARR is 8 per cent (20 per cent - 12 per cent = 8 per cent). This means that, if 100 children were treated, 8 would be prevented from developing bad outcomes.
Another way of expressing this is the number needed to treat (NNT). If 8 children out of 100 benefit from treatment, the NNT for one child to benefit is about 13 (100 ÷ 8 = 12.5).
How do you calculate NNT?
NNT = 1 / ARR
Busy clinicians have an obligation to understand primary (original) research in order to maintain relevant, up to date practice.
Need to utilize information management to find:
Clinically applicable primary sources and appraise them critically.
Appropriate secondary sources that summarize the relevant literature and deliver a useful, actionable bottom line.
Evolution
Clinicians must be able to decipher which studies are useful to their practice
—-Must be able to critically evaluate articles fairly quickly and efficiently.
Clinicians must also be able to sift through which articles have been evaluated with sufficient academic rigor.
Peer-reviewed?
Academic journal vs. “throwaway”
Sponsorship? (pharmaceutical company, etc.)
True
New Definition of EBM
“The revised and improved definition of evidence-based medicine is a systematic approach to clinical problem solving which allows the integration of the best available research evidence with clinical expertise and patient values.”
Evaluate how recently the summary was updated or revised
Not all topics are covered by filtered information resources
Meta-analyses, Cochrane Database of Systematic Reviews.
Clinical Practice Guidelines
Filtered Resources
Examples of Filtered Resources?
Systematic Reviews
Meta Analysis
Evidence Summaries
Evidence Guidelines
Better known as “primary literature”
It’s up to YOU to assess quality, validity and applicability to your patient
Requires time and experience
Unfiltered Resources
Examples of Unfiltered Resources?
RCT’s
To learn about a new topic or refresh knowledge
Provide a comprehensive overview of a disease, condition, or concept
Usually in a textbooks, Medline, Medscape, UpToDate
Background Resources
Examples of Background Resources?
Background info, expert opinion
Research Paradigms: Know that qualitative research is not less rigorous or easier – just different!
Cool
