Chapter 2- Research Methods Flashcards
Prospective Treatment Assignment?
Have some information on the person before we follow them into the future; whether or not the investigator designs the exposure; do they become involved before or after they start their treatment
Experimental study
Manipulated treatment variable to follow the effects
Observational
Don’t follow the treatment
Don’t intervene or change the participant’s course during the trials
Simply observing them
Is there a comparison group?
Yes- Analytical Study
No- Descriptive study
Tx → Outcome
Cohort Study
Identify a group that receives a treatment, and following them prospectively to see what the treatment is
Example- using new york state quit line to see who’s trying to quit smoking, and following them for another 6 months to see whether or not they quit and compare them to people who didn’t call the quitline
Advantages of Cohort study
Track progress over time
Not relying on people’s memory
Documenting the prevalence of relapses
Disadvantages of cohort study
Cannot tell if treatment caused the outcome (no cause and effect)
Are you manipulating the treatment? No → all you are doing is observing and if they are just observing, they are just correlations
Tx ← Outcome
Case control study
Case control study
Start from the outcome
Example- Find people who quit smoking in the past 6 months and see which people are more likely to call the quitline
Remembering backwards to determine whether or not they had a treatment
Example- someone who developed a disease and looking backward to see if they were exposed to a pathogen
Same Time
Cross-Sectional Study
Cross-Sectional Study
Most common studies seen
Only doing the survey once, and seeing if that at that moment in time, are the studies correlated
Cannot say that one thing caused/predicted the other
Use one time point to assess both outcome and treatment exposure
Can provide estimates of frequency or prevalence of an outcome or treatment
Can’t tell you which came first
Subject to recall bias
Cohort Study Strengths
Treatment comes before outcome (temporal precedence)
Less prone to recall bias
Provides estimates of incidence of outcomes over time
Cohort Study Limitations
Cost
Rare outcomes are hard to observe
Studies may need to be very long to observe outcomes
Case-Control Study Strengths
Useful for rare outcomes
Can save time and money
Case0Control Study Limitations
Difficult to select an appropriate control group
Recall bias
Cannot tell you how prevalent the outcome or treatment is - only the odds of experiencing both
More variables involved - more people might develop that outcome, they just haven’t yet
Rate
frequency of an event in the population over a defined period of time
Proportion
frequency of an event without a defined time period
Ratio
number of people in one condition, relative to the number in another
Rate Example
Number of people with depression in the US who saw a therapist last year / Total number of people in the US who had depression last year
Example- Rate of people who have depression seeing a therapist
Proportion Example
Number of adults in the US who see a therapist / Total number of adults in the US
More broad
Ratio Example
Number of people who attended therapy in 2022 / Number of people who did not attend therapy in 2022
Absolute risk = ??
probability of an outcome
Absolute risk = ??
probability of an outcome
Relative Risk = ??
ratio representing how often the outcome happens in the treatment group, relative to the untreated group
Absolute Risk Example
If you attempt to quit smoking, what is the probability that you will relapse?
Relative Risk Example
Some treatments show that people are 4x more likely to talk to their doctor about quitting compared to those who didn’t go through the treatment
Odd Ratio
Likelihood of membership in one group, given membership in another (also relative)
Odd Ratio Example
Example- given membership in the treatment group (vs. non-treatment group), how likely are you to also be in the outcome group
Example 2- what are the odds people went on to use additional treatment vs. not
Inclusion Criteria
Who gets to be included in the study
Define severity (example- diagnosis)
Example- can anyone who displays depression symptoms participate or do they have to be clinically defined as being depressed at east 1 month before the study?
What works for whom? Question being answered
People in the trial define the conclusions you can draw
Specific course or duration
Consider potential iatrogenic comorbidity
Staging and use of adaptive designs
Recruitment
How are participants identified?
Is the sample representative?
If we made compromises, how can we assume scientific studies and draw accurate conclusions
No treatment or waitlist
Minimal attention
Either receive the smoking intervention or get nothing
Problems- People have to believe in the treatment and the placebo effect aids in treatment effect sizes
If you have a no treatment group, how much of an impact your intervention had might be artificially inflated because you’re not comparing to a placebo effect, you are comparing it to nothing
Treatment as usual (control group)
What does someone do when they usually go to the doctor and compare it to whether their treatment works better than medical practice guidelines
Challenging to define, and often isn’t homogenous
Compared to someone who goes to a clinic and seeing what they get
Attention placebo control
Non-specific factors - patient-therapist rapport, contact time with the therapist, BUT no active treatment component
Allowing for the possibility of people getting better due to the placebo effect and due to positive patient-therapist rapport
Parallel treatment Design
Enrolling 100 people in the study, splitting them and, then observing them at the same time
Both groups are going through the same thing at the same time
Adaptive Design
Should people get a treatment that is tailored to them, or just a generalized treatment for everyone
“Just in time” interventions
Someone signs up for the interventions and we identify certain time and situations where they are in a setting and something is pushed to their cell phones to give adaptive content that differ within the group to help them
Dismantling Design
What works for whom in what context and why
What are the active ingredients of the treatment
What pieces of the treatment is causing the effect? Why? Do we need all of it? What treatment components do we keep and what do we try to get rid of
Treatment allocation should be blind and concealed from those administering assessment
How do we make these assessments and define our outcomes
Who is assessing them
Pre and post treatment, consider length of follow up
Short follow up usually
Showing something worked before or after
Don’t know long term outcomes because its hard to follow people for an extended amount of time
Define a priori
Before we started
“Moving the goalposts” - say you need to get a certain outcome and you don’t reach it so you just move what the outcome is supposed to be
What are the outcomes before you conduct the study?
When you report the results, did they actually report the outcomes they said they were going to, or did they move the goalposts?
Define level of improvement they consider acceptable
What level of improvement will mean the treatment worked?
Remission
Reduction of symptoms
Longevity of change
Efficacy
It worked in a controlled trial
If was efficacious
Given all the things we just talked about and the randomized control trial was controlled, all the things are super valid in the trial, BUT they could lack generalizability
Questions- for whom? Under what context? Often, people who participate in well-controlled, high-validity control trials are a homogenous group that is really different in subsequent ways
Valuable in that it tells if things could work and provides the experimental control that allows us to determine cause and effect BUT is extremely limited in how it advises
Therapist tape record session and researchers watch to make sure they’re doing what they need to be doing
Effectiveness
Have to let go of some of the control
Different situations that people use a technique in
Need to take efficacious studies and test them for effectiveness
Stringent Criteria for study designs
Must have active control, large sample sizes, support from more than one study conducted by different research groups
Efficacy? Large control/sample size? Different research groups?
Must be able to show different research groups have the same control
Subjects to limitations of RCTs
Efficacy vs. effectiveness
Manualized interventions
Evidence-based Practice
Going straight to the research
Look up what research has been done, and what seems to be effective
What is efficacious and integrate that with the patient’s values and preferences
Honest about the goals and treatments going into the therapy
Evidence-based practice-
Some research/evidence that shows it can be beneficial
Evaluating the Evidence base
There will always be strengths and limitations to designs
No such thing as a perfect study
Must make compromises
Making sure to balance strengths and limitations
Fallible- must make sure to be flexible
Example- PTSD- symptoms may get worse before they get better
Possibly shifting the course of treatment after monitoring progress and understanding challenges and limitations that research can provide
