4 - Therapys + RCTs Flashcards
1
Q
what is the most important consideration in the design of any experiment?
A
- to minimize error
2
Q
explain the association of power and error
A
- power = ability to statistically detect a diff btw groups when one exists (signal)
- error = unexpected variability within the outcome (noise)
- therefore finding a S.S. diff btw groups = signal/noise!
- little we can do about signal, much can be done about noise reduction!
3
Q
what is total error made up of?
A
- systematic and random error
4
Q
define systematic vs random error - how can it be prevented?
A
- systematic = bias, variability in the outcome that can be prevented or explained (threatens validity)
- random: variability in the outcome that cannot be explained (threatens precision and external validity)
- there will always be some amount of error
- prevented through design or removal during analysis
5
Q
how can we increase power? (3)
A
- decrease noise
- increase signal
- lower standards (increase willingness to accept t1 error)
6
Q
describe the conventional design of an RCT
A
- p 52
7
Q
describe importance of control groups and 2 types of control groups
A
- important for controlling threats to internal validity (whether change happened due to chance or it would have happened anyways?)
- no-treatment control and standard-of-care control
8
Q
what is the standard of care control group
A
- provides all medical treatment to all participants
- less statistically powerful (delta is smaller)
- may be more ecologically valid
- can’t compare with placebo if another valid standard of care exists (ethics)
9
Q
what is the no treatment control group
A
- may be limited to treatments with wait lists
- more statistically powerful than standard of care control group (delta is bigger)
- bigger signal, increased power, decreased n-size requirements
10
Q
name some features that limit bias in a design
A
balance of prognostic factors
- randomization
- allocation concealment
- blinding
- standardization of protocol
- intent-to-treat analysis
- completeness of follow-up
11
Q
describe the balance of prognostic factors
A
- prognostic means controlling for what you can up front or at the end through analytical methods
- look to make sure in the table (should be provided by researcher) that important characteristics are similar btw 2 groups
- note that there is no point in adding p values for comparing (redundant and use up our alpha)
- p 53
12
Q
what are the 4 types of randomization?
A
- simple
- stratified
- blocked
- minimization
13
Q
what is stratified randomization?
A
- separating samples into several subsamples to balance prognostic factors btw groups
- ie males and females - each group separated equally among tx and ct
- good for smaller studies
14
Q
what is blocked randomization?
A
- controls allocation of participants so there is an equal distribution of participants btw groups
- blocks are multiples of the number of groups you have (ie if you have 3 groups, block size can be 3, 6, or 9)
- good for smaller studies
15
Q
what is minimization randomization?
A
- calculated imbalance within each prognostic factor should the patient be allocated to a particular treatment group. various imbalances added together to get overall study imbalance. patient is assigned to treatment group that would minimize the imbalance.
- uses computer algorithm
16
Q
what is allocation concealment?
A
- person making decision about patient eligibility is unaware of which group they are assigned to until decision about eligibility has been made
- internal validity error!
17
Q
what is selection bias?
A
- systematic errors in the measurement of the effect of treatment due to differences btw those who are selected and those who are not
- internal validity error!
18
Q
how do we implement allocation concealment? how does inadequate allocation concealment affect results?
A
- note that allocation concealment can ALWAYS be done!
- safeguard allocation procedure before and until allocation is done
- use web-based (best), central call-in, independent source, envelopes, etc
- trails where AC was inadequate demonstrate on average larger treatment effect (30-40%)
19
Q
what is simple randomization?
A
- like flipping a coin
- good for larger studies
20
Q
define blinding and how important it is
A
- study participants (including clinician, patients etc) are unaware of the group to which patients are assigned
- importance depends on subjectivity of the outcome (eg just life/death is very objective, but cause of death is becoming more subjective - as soon as we step away from extremely objective, need blinding)
- cannot always be implemented
- safeguards randomization sequence AFTER allocation
21
Q
how does lack of blinding affect results?
A
- trials where blinding was inadequate demonstrate on average 17% overestimation of treatment effect
22
Q
what is a detection bias? how to prevent?
A
- you find something bc you are expecting it
- prevent with tests and tet frequency being the same!
- in terms of clinician blinding, when different testing or frequency of tests/seeing the patient occurs
- p 58
23
Q
what is interviewer bias
A
- greater probing of an interviewer of some participants than others related to group
24
Q
what is expectation bias
A
- interviewers knowledge of group influences their expectation of finding an outcome (act surprised, etc)
25
Q
what is an observer-expectancy effect?
A
- when an experimenter subtly affects the participant, causing them to behave congruently with his behavior
26
Q
what is a subject-expectancy effect?
A
- when a participant acts in a certain way due to their expected outcome
27
Q
define placebo effect
A
- effect of treatment independent of its biological effect
28
Q
describe the demand characteristics
A
- faithful participant role: trying to follow instructions exactly
- good participant role: trying to determine the research hypothesis and confirm them
- negative participant role: trying to determine the research hypothesis and disconfirm them
- apprehensive participant role: participant is concerned of researcher’s opinion of them and changes behaviour accordingly
29
Q
look at blinding data analyst slide
A
- p 56
30
Q
what do we think about measuring blinding success?
A
- not really necessary bc even if blinding is successful participants/experimenters may still demonstrate expectancy bias
31
Q
what is contamination vs co-intervention?
A
- contamination: individuals within the control group obtain the treatment outside the experiment (increase noise, decrease power) - ie receive the other groups treatment (for one or both groups) - get diluted signal (ITT) or biased results (non-ITT)
- co-intervention: individuals within the control group obtain effective treatments other than the treatment under study (delusion of results, don’t know what caused what)
- prevent w rules about timing and types of co-interventions permitted, record keeping and post hoc adjustment for imbalances
- for both prevent with standardizing protocol, same freq and test type
32
Q
what is intention to treat analysis (ITT)?
A
- aka “as randomized” analysis
- patients analyzed within their allocated group (not according to what they received), whether or not they received, were adherent to, or completed the protocol
- minimizes t1e (preserves prognostic balance of group)
- can contribute to t2e bc of contamination
33
Q
what are exceptions to ITT? aka when is it less of a threat to validity to exclude patients from the analysis? and how are they removed?
A
- patients without the disorder, patients never eligible for participation (accidentally entered through error)
- must be removed w independent adjudication committee blind to group allocation before randomization!
34
Q
why does missing data threaten validity and does inflating n size fix the problem?
A
- data are rarely missing for trivial reasons
- no! - people who drop out cannot be replaced (bc you will get someone who is not the same as the people who dropped out)
- want to see less than 20% missing overall, if % missing is diff btw groups may be due to treatment!
35
Q
define the 3 types of missing data
A
- MCAR - does not depend on observed or unobserved outcome (ie car breaks down) - can threaten precision (smaller n-size) but not validity of study!
- MAR - depends on variables unrelated to outcome (storm keeps all patients from Toronto from coming)
- MNAR - related to the outcome (patient does not want to return bc they got better or worse) no imputation methods are appropriate for dealing w this type of data!
- note that data missing completely at random only influences precision and any other types of missing data threaten t1e AND precision
36
Q
describe excluding all patients w missing data method
A
- easy default to most statistical packages
- increase t2e
- threatens internal validity (does not follow ITT) - this increases possibility of t1e!
- decreases precision (bc of decreasing n-size)
37
Q
describe assuming worst/best-case scenario method
A
- assume worst for treatment/best for control (increasing t2e!)
- not appropriate for longitudinal data w missing mid-point data (bc there are points on either side of the missing data)
- may be overly conservative
- usually used when we don’t know what happened at the end of a study
38
Q
describe last outcome carried forward method
A
- may be too conservative
- not appropriate for longitudinal w missing mid-point
- trajectory of change is ignored
39
Q
describe growth curve analysis
A
- good for mid-point data
- requires at least 2 data points
- not appropriate for missing endpoint data
- decreases variability ie CI (increases chance of t1e)
40
Q
describe regression methods
A
- allows for examination of longitudinal trends
- decreases variability ie CI (increases chance of t1e)
- where did the avg person score at 4 weeks for example
- added precision, reduced noise
- more missing data = worse
41
Q
describe multiple imputation methods
A
- allows for longitudinal trend examination
- using more than one method, which adds come variability to data (decreasing t1e)
42
Q
describe mixed model analysis
A
- for when no actual time is assigned
- fixed and random
- time recorded as “days since intervention” not by fixed visit interval
- can’t be used for endpoint data
- this is recommended bc it leaves all patients in analysis without making assumptions does not reduce variability too much (ie by inputting avg values for missing data points)
43
Q
is the validity of the study threatened by missing data?
A
- imputation should only ever improve precision, validity should not change! if it does, this is an n-size issue (not big enough) - ie conclusion should not change!
- see p 61
