Unit 2 Flashcards
What are the factors and examples of a quantitative design?
Numbers are used to represent data
ex: scales, surveys, pain scales
What are the factors and examples of a qualitative design?
Words are used to represent data
ex: word clouds, small group open ended commentary session
What are the 2 types of quantitative designs?
Interventional and Observational
Interventional Design
Forced allocation into groups
- researchers intervene when assigning groups
Experimental design
Investigator selects the intervention
The only research design that can be used to show causation
Types: Phase 0, Phase 1, Phase 2, Phase 3, Phase 4
Observational Design
No forced allocation into groups
Natural design
- can do experimental type things
Researchers observe subjects/ elements occurring naturally or selected naturally by the individual
Used when it could be unethical to assign people to groups based on what you’re studying
Cannot be used to prove causation
Types: Cross-sectional, Case- control, Cohort
When would one use an observational design over an interventional design?
When it could be unethical to assign people to groups based on what you’re studying
When cost needs to be considered
What is the most useful and appropriate study design?
The one that answers the research question
Depends on question being asked and the desired perspective
What are the elements of a study design?
Research Question
Research Hypothesis
Selecting study subjects
Sampling Schemes
Research Question
“I wonder if …” statement
Frames study intent
Directs researcher to selecting and developing an effective study design to answer a question
Null Hypothesis
Research perspective that states there will be no true difference between the groups being compared
- says there is no association - will either reject or not reject this perspective based on results
Most conservative and most commonly used hypothesis
Statistical Perspectives = Superiority vs Noninferiority vs Equivalency
What are the 3 statistical perspectives of the Null Hypothesis?
Superiority
Noninferiority
Equivalency
Studies usually only look at 1 of these
Superiority
Used in drug vs placebo experiments
Asks “is this better?”
Null = this drug will not be superior to the placebo
Noninferiority
Used to compare drug to an efficient, gold standard drug already on the market
Asks “am I at least as good?”
Null = I am worse than the other.
Equivalency
Null = I am not equal
How to select study design?
Perspective of research question (hypothesis)
Ability/ desire of researcher to force group allocation (randomization)
Ethics of methodology
Efficiency and practicality (time and resource commitment)
Costs
Validity of acquired information (internal validity)
Applicability of acquired information to non-study patients (external validity)
Population
All individuals making up a common group
Can be divided into a smaller set (sample)
Sample
Subset or portion of the full, complete population
- representatives
Useful when studying the complete population is not feasible
How is the study population different from the population?
SP = the final group of individuals selected for a study
SP is a sample of the larger population
What is study subject selection based on?
Research hypothesis/ question
Population of interest
- people who are most useful and applicable to answer the research question
Group selection criteria
- Inclusion and exclusion group
- Case and control group
- Exposed and non-exposed group
- Desired vs logical vs plausible selection criteria
- Impacts generalizabilityWhat are the two broader examples of Sampling Schemes?
Probability Samples
Non-probability Samples
Probability Samples + examples
Every element in the population has a known probability of being included in sample
- non-zero probability
Simple Random, Systematic Random, Stratified Simple Random, Stratified Disproportionate Random, Multi-stage Random, Cluster Multi-stage Random
Simple Random Sampling
- Assign random numbers then take randomly- selected numbers to get desired sample size
- Assign random numbers then sequentially list numbers and take desired sample size from top/ bottom of listed numbers
Systematic Random Sampling
Systematically sampling within groups
Assign random numbers then randomly sort the numbers and select the highest or lowest number
- systematically and by a pre-determined sampling interval take every Nth number to get desired sample size
Stratified Simple Random Sampling
Stratify sampling frame by desired characteristic
- use simple random sampling to select desired sample size
Desired characteristic may be a confounder
Have an equal number of strata in each group
Stratified Disproportionate Random Sampling
Disproportionately uses stratified simple random sampling when baseline population is not at the desired promotional percentages to the referent population
- stratified sample is weighted to return the sample population back to baseline population
Used for over- sampling
Multi-stage Random Sampling
Uses simple random sampling at multiple stages towards patient selection
- SRS at different levels
Regions/ counties = primary sampling unit
City blocks/ zip codes = secondary sampling unit
Clinic/ hospital/ household
Individual/ occurrence
Cluster Multi-Stage Random Sampling
Same as multi-stage random sampling but all elements clustered together (at any stage)
- ex: all clinics in a zip code, all households in a community
Non-probability Sampling
Quasi-systematic/ Convenience Sampling
Not completely random or fully probabilistic
Researchers decide what fraction of the population is to be sampled and how they will be sampled
What is a concern with using non-probability sampling?
There is some known or unknown order to the sample generated by the selected scheme which may introduce bias
- selection bias
Outcomes of Study
Patient oriented vs disease oriented
PO is more important and useful
Patients want to know what an influence will be on them (don’t care about numbers)
What are the easiest outcomes to generate?
Physiological Outcomes (numbers, levels, etc)
What do are the characteristics of the assessments we want to use to ensure internal validity?
Scientifically rigorous and standardized
Objective assessments are between than subjective assessments
Accurate, reproducible, and scientifically
What is the study population selection based on?
Ethics
Principles of bioethics must be met
What is equipoise?
Genuine confidence that an intervention may be worthwhile in order to use it in humans
Worthwhile = risk vs benefit
What are the 4 key principles of bioethics?
Autonomy
Beneficence
Justice
Nonmaleficence
Autonomy
Self-rule/ self- determination
Patients must decide for ones-self without outside influences
- no coercion, reprisal, financial manipulation
Patients need to have full and complete understanding of risks and benefits
- no misinformation, incomplete information, or ineffectively- conveyed information - need to account for language and education level
Beneficence
To benefit or do good for the patient
Not society
Justice
Equal and fair inclusion and treatment regardless of patient characteristics
Nonmaleficence
Do no harm
Researchers must no withhold information, provide false information, exhibit professional incompetence
Belmont Report
Issued in 1978 by National Commission for Protection of Human Subjects of Biomedical and Behavioral Research
Based on Tuskegee Syphilis Study
Principles:
- Respect for persons
- research should be voluntary and subjects should remain anonymous
- Beneficence
- research risks are justified by potential benefits
- Justice
- risks and benefits of the research are equally distributed
Consent
Agreement to participate, based on being fully and completely informed
Given by mentally-capable individuals of legal consenting age
- adults; age 18
Assent
Agreement to participate, based on being fully and completely informed, given by mentally- capable individuals not able to give legal consent
- ex: children and adolescents
Children or adults not capable fo giving consent requires the consent of the parent or legal guardian and the assent of the potential study subject
IRB
Institutional Review Board
Determines if a study is ethical and safe
Role = protect human subjects from undue risk
All human subject studies must be reviewed by an IRB prior to study initiation
- observational and interventional studies
Regulated by federal statutes developed Department of Health and Human Services (DHHS)
Rules referred to by Common Federal Rules (CFR)
Applies to all studies funded by federal government
Regulations enforced by Office of Human Research Protections
Office of Human Research Protections (OHRP)
The agency that administers and enforces the regulations
Can sanction/ close down/ stop
The teeth of the IRB
What are the levels of IRB review and what are the main differences between the levels?
Full Board
Expedited
Exempt
Number of members for committee review/ approval
Time for committee review/ approval
Level of detail in documentation needed for review
Full Board
Used for all interventional trials with more than minimal risk to patients
Needs more time and is labor intensive
Used when researchers are interacting with people
Expedited
Used for minimal risk and when there are no patient identifiers
Risk/ trauma can be triggered
- ex: survey brings up a traumatic past experience
Exempt
Used when there are no patient identifiers, low/ no risk, de-identified dataset analysis, environmental studies, use of existing data/ specimens
Data already exists in records
- could be preexisting data
- may not have contact with patient
Who determines the level of review?
Data Safety and Monitoring Board (DSMB)
Semi-independent committee not involved with the conduct of the study but charged with reviewing study data as study progresses to assess for undue risk/ benefit between groups
Use pre-determined review periods
Can stop study early, for overly positive or overly negative findings in 1+ groups compared to the others
This was the group that shut down women’s postmenopausal study in estrogen/ testosterone group
What is the key difference between interventional and observational studies?
Investigator selects interventions and allocates study subjects to forced intervention groups
More rigorous in ability to show cause- and- effect
- can demonstrate causation
What differs between each phase of interventional studies?
Purpose/ Focus
Population studied (healthy/ diseased)
Sample Size
Duration
Pre- Clinical Stage
Prior- to human Investigation
Bench or animal research
Occurs before human receives intervention
Phase 0
Exploratory, Investigational New Drug
Not to see if drug is effective or safety
Does it do what we said it did?
Most phase 0 studies are used for oncology
Purpose/ Focus:
- assess drug- target actions and possibly pharmacokinetics in single or a few doses - first in human use
Population studied:
- healthy or disease patients (oncology) volunteers
Sample Size:
- very small N ( < 20)
Duration:
- very short duration (single dose to just a few days)
Do all interventional studies start at phase 0?
No
Phase 1
Investigational New Drug
Purpose/ Focus:
- assess safety/ tolerance and pharmacokinetics of 1+ dosages - can be first in human use/ early in human use - primary purpose is not to look at the efficacy of disease
Population studied:
- healthy or disease patients volunteers - depends on the disease
Sample Size:
- small N (20 - 80)
Duration:
- short duration (few weeks) - variable
What does pharmacokinetics look at?
How does the body handle the drug?
How does the drug get in?
How long does it take to get in?
Where does it go?
Phase 2
Investigational New Drug
Purpose/ Focus:
- assess effectiveness - continues to assess for safety/ tolerance but this isn’t the primary purpose - Need to use placebo/ comparison group
Population studied:
- diseased volunteers - may have narrow inclusion criteria for isolation of effects
Sample Size:
- larger N ( 100 - 300)
Duration:
- short to medium duration (few weeks to a few months)
Phase 3
Investigational new drug or indication/ population study
Last phase before FDA approval
- need minimum of 3 out of 5 trials to be positive to show that it wasn’t any better than the equivalent or comparison
Purpose/ Focus:
- assess effectiveness - continues to assess for safety and tolerability
Population studied:
- diseased volunteers
- may expand inclusion criteria and comparison groups for delineation of effects
- can use different statistical procedures
- superiority vs noninferiority vs equivalency
Sample Size:
- larger N (500 - 3000)
Duration:
- longer duration (a few months to a year or more)
Why can various statistical perspectives be taken in phase 3 studies?
In some studies, it is unethical to give placebo vs something else. In this case, both groups would get a baseline pharmacological drug
Ex: you wouldn’t ask someone with asthma to stop taking their medication
Phase 4
Post marketing and FDA approval
Purpose/ Focus:
- assess long-term safety, effectiveness, optimal use (risk/ benefits)
Population studied:
- diseased volunteers - expand use criteria (comorbidities/ concomitant medication) for delineation of long-term safety/ effects
Sample Size:
- Population N (few hundred to a few thousand)
Duration:
- wide range of durations (few weeks to several years) - ongoing - used in interventional/ observations designs - FDA may make companies use a registry in order to follow patients/ effects/ outcomes
Advantage of Interventional Trials
Cause precedes effect (can prove causation)
Only designs used for FDA approval process
Controlling exam environment
Can avoid certain biases
Disadvantages of Interventional Trials
Cost
Complexity/ time
Ethical considerations (risk vs benefit)
Generalizability
Can be over controlling and not true clinical practice
Very regimented and prescriptive in what we do
Pragmatic Studies
Explanatory
Intervention- like but tells us how to treat patient and disease
Gives us flexibility to change dose/ care to treat each patient as needed
- more flexible in exam environment and makes it closer to clinical practice
What are limitations of pragmatic studies?
Makes it hard to compare groups at the end
Flexibility can introduce extraneous factors/ confounds/ etc
What are differences between simple and factorial interventional designs based on?
Differ based on number of randomization steps patient goes through before being put into final study group
Comes down to how much control should we really give to researchers
Simple Interventional Design
Randomizes subjects exclusively into 2+ groups
1 randomization process (no subsequent randomization)
Commonly used to test a single hypothesis at a time
Asks if drug A is better/ worse/ equal to drug B
Factorial Interventional Design
Randomizes subjects into 2+ groups and then further randomizes each of the groups into 2+ additional subgroups
Allows us to ask more research questions
- is drug A better/ worse/ equal to drug B? - is drug A alone better? - are additional combinations better than 1 drug alone?
Used to test multiple hypotheses at the same time
What does testing multiple hypotheses at the same time do for a factorial interventional design?
Improves efficiency for answering clinical questions
Increases study population sample size
- due to increased group number
Increases complexity
Increases risk of drop outs
May restrict generalizability of results
Parallel Interventional Design
Regardless of simple/ factorial, once patients get into final study group, they do not change groups
Groups are simultaneously and exclusively managed
No switching of intervention groups after initial randomization
Cross- Over Interventional Designs
Also known as self- control
Groups serve as their own control by crossing over from 1 intervention to another during the study
- subjects get to be their own control - works because they are matched on demographics because it is themselves
Allows for smaller total sample size
- caveat: participants have longer participation time - each participant contributes additional data
Uses wash-out and lead- in
Wash- out
Period of time where we don’t give patients drugs
Allows drug to wash out from system so they can start subsequent trials/ treatment
Washes out pharmacological/ psychological effects of study before subsequent treatments
Lead- in
Wash out period that occurs before the study starts
Ex: when patient is on drugs but cannot necessarily take them during the study. This allows drug to leave the system before the study starts
Way to test patients ability to follow directions and see if they meet requirements
- if cannot meet requirements, they can be dropped from the study
Disadvantages of Cross- over designs
Only suitable for long-term conditions which are not curable or which treatment provides short-term relief
Duration of study for each subject is longer
Carry-over effects during cross- over
- wash out is required when prolongs study duration
Treatment- by- period interaction
- differences in effects of treatments during different time periods
Smaller N requirement only applicable if within-subject variation is less than between- subject variation
Complexity in data analysis
Run in/ lead in phase
All study subjects blindly given 1+ placebos for initial therapy (defined time period) to determine baseline of new disease
Standardization
Can assess study protocol compliance
Can wash out existing medication
- reduces at least 1 possible common exclusion criteria
Can determine amount of placebo- effect
What is the difference between primary and secondary outcomes?
Primary are the most important/ key outcomes.
-main research question used for developing/ conducting study
Secondary/ tertiary are less important and can be used for future hypothesis generation
What is a composite outcome?
Combines multiple endpoints into single outcome
Patient- Oriented Endpoints
Most clinically relevant
Death
Stroke/ Myocardial Infarction
Hospitalization
Preventing need for dialysis
Disease- oriented endpoints
Surrogate markers
- elements used in place of evaluating patient- oriented end points
Blood pressure - for risk of stroke
Cholesterol - for risk of heart attack
Change in SCr- for worsening of renal function
Non- Random Group Allocation
Subjects don’t have an equal probability of being selected or assigned to each intervention group
Ex: the first 100 patients admitted to the hospital
- patients attending morning clinic = group 1, patients attending evening clinic assigned to group 2
Random Group Allocation
Most commonly utilized
Subjects have an equal probability of being assigned to each intervention group
Ex: random number generating program
Randomization
Purpose: to make groups as equal as possible based on known and unknown important factors/ confounders
Attempts to reduce systematic differences (bias) between groups which could impact results/ outcomes
Equality of groups is not guaranteed
Documentation of equality of groups reported as p values
Only used in interventional studies, not observational
We want groups to be as equal as possible except on 1 thing
Simple Randomization
Equal probability for allocation within one of the study groups
Blocked Randomization
Ensures balance within each intervention group
Used when researchers want to assure that all groups are equal in size
Used when you can’t run the risk of having unequal groups
Equality is usually assessed for in blocks of 10
Stratified Randomization
Ensures balance with known confounding variables
Want groups to be equal based on a characteristic
- ex: gender, age, disease severity
Can pre-select levels to be balanced within each interfering factor
Masking - Single- blind
Study subjects not informed which intervention group they are in
Investigators are permitted to know
Masking- Double Blind
Neither investigators nor study subjects are informed which intervention group subjects are in
Post-study surveys used to assess adequacy of blinding
Masking- Open label
Unmasked/ unblinded
Study subjects and researchers know what intervention is being received
Masking- Placebo
Dummy Therapy
Inserts treatments made to look identical in all aspects to the active treatments
Improvement in condition by power of suggestion of being treated
Post- Hoc sub-group analysis
Not accepted by most when not prospectively planned
Is accepted when it is prospectively planned for or performed for hypothesis generation and development of future studies
Managing Drop outs/ lost to follow ups
Want drop outs to be equal between groups
Intention to treat
- most conservative decision
Ignore them
- per protocol or efficacy analysis
- compliance must be pre-defined
Treating them as treated
- ignores group assignments
- allows subjects to switch groups and be evaluated in groups they moved to, end in, or stay in most
Impact of Drop out decisions
Intention to treat results in:
- preserved randomization process
- preserves baseline characteristics and group balance at baseline which controls for known and unknown confounders
- maintains statistical power
Per- protocol results:
- biases estimates of effect
- reduces generalizability
How to assess for adherence (compliance)?
Drug levels
Pill counts at each visit
Bottle counter tops
Methods of improving adherence
Frequent follow-up visits/ communications
Treatment alarms/ notifications
Medication blister packs or dosage containers
What is a case- control study?
Observational study
Allows researcher to be a passive observer of natural events occurring in individuals with the disease/ condition of interest compared to people who do not have the condition of interest
How are people assigned to groups in case- control studies?
Put into groups based on disease status
No forced allocation into groups
Case = diseased Control = non- diseased
What information do controls in case- control studies provide us?
Gives information about the expected baseline risk factor profile in the population from which cases are drawn
What caveat to randomization do case-control studies create?
In any observational study, the pool of people to draw from may be bigger than needed so we randomly select people from this
We only need some of these people on order to represent the full population
Why should we use case- control study design?
Unable to force group allocation
- due to ethics/ feasibility
Limited resources
- time/ money/ subjects
- time to completion is shorter because we have data that has already been collected
Disease of interest is rare in occurrence
- little is known about its associations/ causes
- case- control studies directly assess perspective of the hypothesis
Prospective exposure data is difficult/ expensive to obtain and/ or time inappropriate
Case- control studies are usually conducted in what kind of fashion? Why?
Retrospective
- going back to look at exposure
We already know the outcomes so we want to look at the exposures
Strengths of Case- Control Studies
Good for assessing multiple exposures of 1 outcome
Useful when diseases are rare
Useful in determining associations
Less expensive (money/time) than interventional trials and prospective studies
Useful when ethical issues limit interventional studies
Useful when disease has long induction/ latent period (ex: cancer)
Weaknesses of Case- control studies
Can’t show causation
Can be impacted by unassessed confounder
Retrospective
- can’t control for other exposures or potential changes in amount of study- exposure during study frame
Can be impacted by biases
- especially selection and recall/ assessment bias
Limited by available data
Selection of cases in case- control study
Author needs to tell how they picked patients/ cases
Defined by investigator using accurate, medically- reliable, efficient data sources
Selection must be made objectively, consistently, accurately, and with validity
Must use clinically supportable and definable criteria
Classifying patients is ideal but can be misclassified
- prefer non-differential misclassification
- balanced error - moves OR closer to 1.0 (no association)
Control selection in case- control studies
Most difficult part
Expectation: control represents baseline risk of exposure in general or referenced population
Way controls are selected is a major determinant in whether any conclusion is valid
Want groups to be as equal as possible except the presence of disease of interest
Must be selected irrespective of exposure status
- cannot look at past exposure prior to picking controls
What are the three control group sources?
Population
- state/ community/ neighborhood
- general, brand
- can be obtained several ways (randomly)
Institutional/ Organizational/ Provider
- illnesses of controls should be unrelated to exposures being studied
Spouses/ Relatives/ Friends
- genetics, environment, SES, etc
Can be specific or general
Describe selecting control study population via Outbreak sources of control
Choose people who participated in the same event as the cases but did not get the outcome
ex: people at picnic who did vs did not get food poisoning
Can someone be exposed and unexposed individual in the same study? Explain.
Yes
Can be both exposed and unexposed in studies where looking at different exposures of a single outcome
Associated with an outbreak investigation with multiple exposures or in a situation of brief change in risk of the outcome of interest
Case- Crossover Design
Used in a situation of brief change in risk of the outcome of interest
Observational Design
Subjects are their own controls during the other times they don’t have the acute change in risk
- comparing people to themselves
The only case- control design able to adequately attempt to address issue of temporality
Nested Case- Control Studies
Case- control study conducted after or out of a prospective previous study type
Subjects in cohort study, ultimately developing the disease/ outcome, are defined as cases for the subsequent case- control study
- diseased used in a new/ different study
- used to evaluate other exposures
A subsequent study that comes from a different, already completed case control study
A secondary outcome from one study that leads to the development of this subsequent study
What are the sampling techniques for controls used in nesting case- control studies?
Survivor Sampling
Base Sampling
Risk- set Sampling
Survivor Sampling
Sample of non-diseased individuals at end of study period
The “survivors” of the study who didn’t get the same outcome as previous cases
Most commonly used technique
Base Sampling
Sample of non-diseased individuals at start of study period
Go back to the beginning
Risk- Set Sampling
Sample of non-diseased individuals during study period at the same time when case was diagnosed
Taken from a time based during the study
Describe the 2 common biases found in Case- Control Studies
Selection Bias
- Related to the way subjects are chosen for the study
- less of a concern during case- crossover study designs
Recall Bias
- Related to the amount/ specificity that cases or controls recall past events differently
- more common that cases are more likely to recall past exposures and levels of exposure or their timing
Matching- Case- Control Studies
Cases are matched to controls in 1:1 or higher ratio
Cannot match on anything that might be a risk factor
Individual
- matches individuals based on specific patient- based characteristics
- useful for controlling confounding characteristics
Group
- proportion of cases and proportion of controls with identical characteristics are matched
- requires cases to be selected first
What observational study design gives us the strongest evidence?
Cohort Studies
What are cohort studies?
Observational studies that allow the researcher to be a passive observer of natural events occurring in naturally exposed and unexposed groups
Used when studying a rare exposure
Trying to determine of the exposed people, what number of them will get the outcome or not?
- we know the total exposed
Generates risk of disease/ outcome for each group
What are other names for cohort studies?
Incidence studies or longitudinal studies
What is group allocation in cohort studies based on?
Based on exposure status or group membership (having something in common with others)
What is a cohort?
A group of people that have something in common
We can allocate groups after we have our cohort
Birth Cohort
Individuals assembled based on being born in a geographic region in a given time period
ex: everyone born in KC city limits in 2014
Inception Cohort
Individuals assembled at a given point based on some common factor
- common factor examples: where people live/ work
Useful for single- group assessments for incidence rate determination
- ex: single healthcare system
Ex: Framingham Heart Study
- began in 1948
- selected on being a stable population with updated annual population lists and other unique attributes
Ex: Nurses’ Health Study, COB class of 2019
Exposure Cohort
Individuals assembled based on some common exposure
Frequently connected to environmental or other one time events
- usually one time exposures or single events
Ex: 9/11
Can cohort size change over time?
Yes - depends on if fixed vs closed vs open/ dynamic cohort
Fixed Cohort
Cohort is derived from an irrevocable event and can’t gain members but can have loss-to-follow-ups
Long evaluation time
Closed Cohort
A fixed cohort with no loss-to-follow-ups
Short evaluation time
Open/ Dynamic Cohort
Cohort with new additions and some loss- to- follow- ups
Cohorts can increase or decrease over time as people immigrate or emigrate in and out of the population being studied
When should we use cohort studies?
Unable to force group allocation (randomize)
- unethical/ not feasible
Limited resources
- time/ money/ subjects
Exposure of interest is rare in occurrence and little is known about its associations/ outcomes
- directly assesses the perspective of the hypothesis
More interested in incidence rates or risks of outcome of interest
- more than effects of interventions
Can any type of cohort study be used to prove causation?
No
Prospective studies that are well- controlled can approximate causation
Prospective Cohort Study
Exposure group is selected on basis of a past or current exposure
Both groups are followed into future to assess for outcomes of interest which have yet to occur
Retrospective Cohort Study
At start of study, both exposure and outcome of interest have already occurred
- groups are still allocated based on past history of exposure
Retrospectively start at time of exposure and follow forward to the point of outcome occurrence in the present
- exposure still has to occur before outcome of interest
Ambidirectional Cohort Study
Uses retrospective design to assess past differences up to present while also adding future data that is collected prospectively from start of study
Ex: Vietnam War and Agent Orange Exposure
How to select an exposed study population?
Allocate subjects base on pre-defined criteria of exposure
Criteria needs to be scientifically and consistently determined
How to select an unexposed study population?
Make the groups as close as possible
- coming from same cohort/ population yet not exposed
- if exposure truly has no effect, risk will be exactly the same for both groups and RR = 1.0
Unexposed group sources: internal, general population, and comparison cohort
Internal unexposed Study population source
Best option, if feasible
Patients from the same cohort yet are unexposed
If there are only levels of exposure, you may have to use lowest exposure group as comparator
General Population unexposed study population source
Used as a second choice when the best possible comparison group is not realistically possible
Ex: everyone is exposed or exposure subjects were drawn from general population
Comparison Cohort unexposed study population source
Least acceptable group
Attempt to match groups as close as possible on numerous personal characteristics
Cannot control for other potentially harmful exposures in comparison cohort
Strengths of Cohort Studies in general
Good for assessing multiple outcomes of 1 exposure
Useful when exposures are rare
Useful in calculating risk and risk ratios
Less expensive than interventional trials
Good when ethical issues limit use of interventional study
Good for long induction/ latent periods
- retrospective
Able to represent temporality
- prospective
Weaknesses of Cohort Studies in general
Can’t demonstrate causation
Hard to control for other exposures if more than 1 is plausible for being associated with an outcome
Can be impacted by unassessed confounders (retro)
Can be impacted by various biases (retro)
Limited by available data
Advantages of Prospective Cohort Studies
Can obtain a greater amount of study- important information from patients
More control over specific data collection process
Follow- up and tracking of patients may be easier if planned for
Gives better answer to temporality
May look at multiple outcomes from a supposed single exposure
Can calculate incidence and incidence rates
Disadvantages of Prospective Cohort Studies
Time
Expense
Lost- to- follow- ups
Not efficient for rare diseases
- use case- control studies instead
Not suited for long induction/ latency conditions
Exposure/ amount of exposure may change over time
Loss- To - Follow- Ups
Possible with prospective cohorts
Lowers sample size
- decreases power
- increases risk of type 2 error
- loss of study participation may not be equal between groups
Need to limit loss-to-follow-ups via time, energy, and resources
Advantages of Retrospective Cohort Studies
Best for long induction/ latency conditions
Able to study rare exposures
Useful if the data already exists
Saves time and money
Disadvantages of Retrospective Cohort Studies
Requires access to charts, databases, employment records which may not be complete/ thorough enough for study
Information may not factor in or control for other exposures to harmful elements during study period or over time
Patients may not be available for interview if contact is necessary for missing/ incomplete data
Exposure/ exposure amount may have changed over time
Matching- Cohort Studies
Way to strive to makes groups as equal as possible on known/ potential confounders
What are the biases that affect cohort studies?
Healthy- Worker Effect
- if health, you work, even if you are exposed
- if too ill to work, you may be unemployed
- looks at those that are healthy enough to work and ignores those that are dead/ sick
- concern because most of cohort studies are environmental in nature
Selection Bias
- How is exposure status defined/ determined?
What are Cross- Sectional Studies?
Observational studies that capture health/ disease and exposure statuses at the same time
Collects health records/ information at the same time
Prevalence study
- can be used to estimate elements of prevalence
Focuses simultaneously on disease and population characteristics
Why are designs called cross- sectional?
Information gathered represents what is occurring at a point in time or time- frame across a large population acquired without regard to exposure or disease outcome/ status
Looking at a snap shot in time of all elements
Who are the study subjects in a cross- over design?
The entire population or a sample of the population
By focusing simultaneously on disease and population characteristics, what are three things that are looked at in cross- sectional designs?
Looks at associations
Generates and tests hypotheses
By repetition in different time periods, can be used to measure change/ trends
Cross- sectional Approaches
Collect data on each member of the population
- more frequently utilized in city/ state-level evaluations if data is already tracked
- ongoing collection
Take a sample of the population and draw inferences to the remainder (generalizable)
- more frequent approach for large population data
What are the 2 broad approaches used to collect study data/ information in cross-sectional studies?
Questionnaires/ Surveys
- directly collected from patients/ caregivers or their medical records
Physical Assessments
- might involve laboratory, clinical, or psychological tests
- great for assessing health / disease in similar population as time changes
- not likely to be the same individuals year to year
- due to immigration vs emigration rate
Strengths of Cross- Sectional Designs
Quicker and easier for the researcher when using already collected data
- free and already available
Less expensive for researcher than any form of prospective study
Can be analyzed like a case- control or cohort study regarding group allocation
Useful for estimating prevalence rates
Useful for answering research questions about a myriad of exposures and diseases using the same data
Weaknesses of Cross- Sectional Designs
Prevalent cases may represent survivors
Difficult to study diseases of low frequency (may not select for people with it since it is so infrequent)
Unable to generate incidence rates
Problems in determining temporal relationship of presumed cause and effect
- due to fact that exposure and disease histories are taken at the same time
Where do we get cross-sectional surveys from?
National Center for Health Statistics
National Health and Nutrition Examination Survey
NHANES
Assesses the health and nutritional status of adults and children
Combines interviews and physical examinations
- interviews include demographics, SES, dietary, and health- related questions
- examination component consists of medical, dental, physiological measurements, and laboratory tests
Survey sample is selected to represent US population of all ages
- oversampled people who are > 60, blacks/ African Americans, and Hispanics
Worry about selection bias
National Health Interview Survey
NHIS
Principle source of information on health of the civilian, non-institutionalized population
- looks at global health of civilians outside of hospital
Survey sample is selected to represent the US population of all ages
Has central role in other surveys NSFG and NAMCS/ NHCS
Data collected through personal household interview (door to door)
Consists of a set of core questions that remain largely unchanged and a set of supplements used to respond to public health data needs as they arise
Bias: response bias
National Ambulatory Medical Care Survey
NAMCS
National survey designed to meet the need for objective, reliable information about the provision and use of ambulatory medical care services in US
Based on a sample of visits to non-federal, non-institutional physicians primarily engaged in direct patient care
National Hospital Care Survey
NHCS
Combined national survey designed to describe national patterns of healthcare delivery in non-federal hospital based settings
- discharges from inpatient departments and institutions
- visits to emergency departments, outpatient departments, and ambulatory surgery centers
Integrates other surveys
Behavioral Risk Factor Surveillance System
BRFSS
State- based system of telephone health surveys that collects information on health risk behaviors, preventative health practices, and health care access primarily related to chronic disease and injury
Interviews adults aged 18 and older who can give consent
Largest landline telephone health survey in world
Youth survey conducted in schools with parental consent and child assent
Worry about responder bias
What are weaknesses of the BRFSS?
Not everyone has landlines
Times of people calling
- may not answer during day bec they are at work, etc
Are people telling truth?
What questions should patients ask their physicians when a medical screening test is recommended?
How accurate is the screening test you are about to recommend for me?
When the test results are announced, how confident will you be in your prediction of whether I do or don’t have the disease?
What are the possible screening outcomes?
True Positive
True Negative
False Positive
False Negative
True Positive
Test correctly reports a positive result in a patient that actually does have the disease
Corresponds to A in the 2x2 table
True Negative
Test correctly reports a negative result in a patient that actually does not have the disease
Corresponds to D in the 2 x 2 table
False Positive
Test incorrectly reports a positive result in a patient that actually does not have the disease
Ex: telling a man they are pregnant
Corresponds to B in 2x2 table
False Negative
Test incorrectly reports a negative result in a patient that actually does have the disease
Ex: telling a pregnant women they are not pregnant
Corresponds to C in 2x2 table
What answers the question of “How accurate is the screening test you are about to recommend for me”?
Sensitivity and Specificity
Describes accuracy of test result based on a known disease status from a gold standard
Trying to find the accuracy of test in people who are already diseased/ have drug in their system
Sensitivity
How well a test can detect presence of disease when in fact disease is present
- how well test comes back positive when person has disease
- positivity of test in the diseased
Proportion of time that a test is positive in a patient that does have the disease
If test has high sensitivity, it has low false negative rate
What are the 3 equations used to calculate Sensitivity?
= [True positive / (True positive + false negative)] * 100%
= [True positive / all diseased] * 100%
= A / (A + C)
Specificity
How well a test can detect absence of disease when in fact the disease is absent
- negativity of test in the healthy
Proportion of time that a test is negative in a patient that does not have the disease
If a test has high specificity, it has a low false positive rate
What are the 3 equations used to calculate Specificity?
= [True negative / (True negative + false positive)] * 100%
= [True negative / all not diseased] * 100%
= D / (D + B)
What answers the question of “When the test results are announced, how confident will you be in your prediction of whether I do or don’t have the disease”?
Positive and Negative Predictive Value
Describes accuracy of prediction of disease based on known test results
Positive Predictive Value
How accurately a positive test predicts the presence of disease
- What percentage of the time do people really have disease?
Proportion of true positives in patients with a positive test
- correct prediction
What are the 3 equations used to calculate Positive Predictive Value?
= [True positives / (True positive + false positive)] * 100%
= [True positives / all positive tests] * 100%
= A / (A + B)
Negative Predictive Value
How accurately a negative test predicts the absence of disease
- What percentage of the time do people really not have disease?
Proportion of true negatives in patients with a negative test
- correct prediction
What are the 3 equations used to calculate Negative Predictive Value?
= [True negative / (True negative + false negative)] * 100%
= [True negative / all negative tests] * 100%
= D / (C + D)
What happens to the value of PPV when you approach prevalence of 100%?
PPV skyrockets toward 100% because most things will be associated with the disease
Why are specificity and sensitivity the same across all of the populations despite different prevalences (2% vs 20% vs 40%)?
We are using the same test to assess for both across all of the populations so the numbers aren’t going to change.
Specificity and Sensitivity reflect the test, not the community you use it on
Why does PPV and NPV change across all of the populations despite different prevalences (2% vs 20% vs 40%)?
Number of diseased people increases so the number of errors increases
False positives and false negatives are changing because specificity and sensitivity are not equal to 100 (and not equal to each other)
Prevalence is not 100%
What does a specificity of 97% mean?
Test is good at coming back negative when people don’t have the disease
Why are PPV’s low for X-Rays?
Any visualization on an X-Ray does not automatically mean cancer/ disease. The image could be something else
The more false positives = _______ the specificity = ______ the PPV. Why is this the case?
Lower
Lower
Due to the false positives being included in the denominator
The more false negatives = _____ the sensitivity = _____ the NPV.
Lower
Higher
When will PPV and NPV not change?
When the prevalence of disease or not disease = 100%
Diagnostic Accuracy
Also known as diagnostic precision
Proportion of total screenings that a patient is correctly identified as either having a disease (true positive) or not having disease (true negative) with either a positive or negative test, respectively
= [(True positives + true negatives) / (true positive + false positive + true negative + false negative)] * 100%
= [(A+D) / (A+B+C+D)] * 100%
Likelihood Probabilities
Ratio of 2 probabilities
Probability of a given test result for a person with disease / probability of the same test result for a person without disease
Probability of some test outcome in diseased / probability of some test outcome in non-diseased
Likelihood Ratio Positive
Probability of a positive test in the presence of disease / probability of a positive test in absence of disease
Sensitivity / (1 - specificity)
= [(A / (A + C)) / (B / (B + D))]
Want numerator to be large and denominator to be small
Looking at the probability of getting positive test in the diseased and not diseased
If test is good/ useful, LR+ should be large
Should be > 10 to demonstrate test is most beneficial
Likelihood Ratio Negative
Probability of a negative test in the presence of disease / probability of a negative test in absence of disease
(1 - Sensitivity) / Specificity
= [(C / (A + C)) / (D / (B + D))]
Looking at the probability of getting a negative test in the diseased and not diseased
If test is good/ useful, LR- should be small
Should be < 0.1 to demonstrate test is most beneficial
What is the likelihood that people who have disease get a negative result?
0%
What is the likelihood that people who don’t have disease get a negative result?
100%
What is the likelihood that people who have disease get a positive result?
100&
What is the likelihood that people who don’t have disease get a positive result?
0%
What does LR+ = 1.0 and LR- = 1.0 mean for the beneficiality of test?
LR+ = 1.0 means that test is just as likely to be positive in people with disease as it is in people who don’t have disease
LR- = 1.0 means that test is just as likely to be negative in people with disease as it is in people who don’t have disease
Validity
Ability to accurately discern between those that do and those that don’t have the disease
Precision in finding and reporting the truth
Internal or External
Internal Validity
Extent to which results accurately reflect what was being assessed
Occurs inside study design
True situation of study population
External Validity
Extent to which results are applicable to other populations
Populations are not those that were included in original study
Occurs outside study design
Generalizability
Reliability
Ability of a test to give same result on repeated uses
Reproducibility/ Consistency
A valid test is always ____.
A reliable test is not always ____.
Reliable
Valid
Cutoff values
Not always dichotomous
Depends on disease and big picture impact
Want least amount of false negatives and false positives
- want something in the middle because if you target 1 or the other, will have more of the one you didn’t correct for
Cutoff of 50% because it minimizes FN and FP
