4 - Therapys + RCTs Flashcards

1
Q

what is the most important consideration in the design of any experiment?

A
  • to minimize error
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

explain the association of power and error

A
  • power = ability to statistically detect a diff btw groups when one exists (signal)
  • error = unexpected variability within the outcome (noise)
  • therefore finding a S.S. diff btw groups = signal/noise!
  • little we can do about signal, much can be done about noise reduction!
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

what is total error made up of?

A
  • systematic and random error
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

define systematic vs random error - how can it be prevented?

A
  • systematic = bias, variability in the outcome that can be prevented or explained (threatens validity)
  • random: variability in the outcome that cannot be explained (threatens precision and external validity)
  • there will always be some amount of error
  • prevented through design or removal during analysis
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

how can we increase power? (3)

A
  • decrease noise
  • increase signal
  • lower standards (increase willingness to accept t1 error)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

describe the conventional design of an RCT

A
  • p 52
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

describe importance of control groups and 2 types of control groups

A
  • important for controlling threats to internal validity (whether change happened due to chance or it would have happened anyways?)
  • no-treatment control and standard-of-care control
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

what is the standard of care control group

A
  • provides all medical treatment to all participants
  • less statistically powerful (delta is smaller)
  • may be more ecologically valid
  • can’t compare with placebo if another valid standard of care exists (ethics)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

what is the no treatment control group

A
  • may be limited to treatments with wait lists
  • more statistically powerful than standard of care control group (delta is bigger)
  • bigger signal, increased power, decreased n-size requirements
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

name some features that limit bias in a design

A

balance of prognostic factors

  • randomization
  • allocation concealment
  • blinding
  • standardization of protocol
  • intent-to-treat analysis
  • completeness of follow-up
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

describe the balance of prognostic factors

A
  • prognostic means controlling for what you can up front or at the end through analytical methods
  • look to make sure in the table (should be provided by researcher) that important characteristics are similar btw 2 groups
  • note that there is no point in adding p values for comparing (redundant and use up our alpha)
  • p 53
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

what are the 4 types of randomization?

A
  • simple
  • stratified
  • blocked
  • minimization
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

what is stratified randomization?

A
  • separating samples into several subsamples to balance prognostic factors btw groups
  • ie males and females - each group separated equally among tx and ct
  • good for smaller studies
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

what is blocked randomization?

A
  • controls allocation of participants so there is an equal distribution of participants btw groups
  • blocks are multiples of the number of groups you have (ie if you have 3 groups, block size can be 3, 6, or 9)
  • good for smaller studies
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

what is minimization randomization?

A
  • calculated imbalance within each prognostic factor should the patient be allocated to a particular treatment group. various imbalances added together to get overall study imbalance. patient is assigned to treatment group that would minimize the imbalance.
  • uses computer algorithm
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

what is allocation concealment?

A
  • person making decision about patient eligibility is unaware of which group they are assigned to until decision about eligibility has been made
  • internal validity error!
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

what is selection bias?

A
  • systematic errors in the measurement of the effect of treatment due to differences btw those who are selected and those who are not
  • internal validity error!
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

how do we implement allocation concealment? how does inadequate allocation concealment affect results?

A
  • note that allocation concealment can ALWAYS be done!
  • safeguard allocation procedure before and until allocation is done
  • use web-based (best), central call-in, independent source, envelopes, etc
  • trails where AC was inadequate demonstrate on average larger treatment effect (30-40%)
19
Q

what is simple randomization?

A
  • like flipping a coin

- good for larger studies

20
Q

define blinding and how important it is

A
  • study participants (including clinician, patients etc) are unaware of the group to which patients are assigned
  • importance depends on subjectivity of the outcome (eg just life/death is very objective, but cause of death is becoming more subjective - as soon as we step away from extremely objective, need blinding)
  • cannot always be implemented
  • safeguards randomization sequence AFTER allocation
21
Q

how does lack of blinding affect results?

A
  • trials where blinding was inadequate demonstrate on average 17% overestimation of treatment effect
22
Q

what is a detection bias? how to prevent?

A
  • you find something bc you are expecting it
  • prevent with tests and tet frequency being the same!
  • in terms of clinician blinding, when different testing or frequency of tests/seeing the patient occurs
  • p 58
23
Q

what is interviewer bias

A
  • greater probing of an interviewer of some participants than others related to group
24
Q

what is expectation bias

A
  • interviewers knowledge of group influences their expectation of finding an outcome (act surprised, etc)
25
Q

what is an observer-expectancy effect?

A
  • when an experimenter subtly affects the participant, causing them to behave congruently with his behavior
26
Q

what is a subject-expectancy effect?

A
  • when a participant acts in a certain way due to their expected outcome
27
Q

define placebo effect

A
  • effect of treatment independent of its biological effect
28
Q

describe the demand characteristics

A
  • faithful participant role: trying to follow instructions exactly
  • good participant role: trying to determine the research hypothesis and confirm them
  • negative participant role: trying to determine the research hypothesis and disconfirm them
  • apprehensive participant role: participant is concerned of researcher’s opinion of them and changes behaviour accordingly
29
Q

look at blinding data analyst slide

A
  • p 56
30
Q

what do we think about measuring blinding success?

A
  • not really necessary bc even if blinding is successful participants/experimenters may still demonstrate expectancy bias
31
Q

what is contamination vs co-intervention?

A
  • contamination: individuals within the control group obtain the treatment outside the experiment (increase noise, decrease power) - ie receive the other groups treatment (for one or both groups) - get diluted signal (ITT) or biased results (non-ITT)
  • co-intervention: individuals within the control group obtain effective treatments other than the treatment under study (delusion of results, don’t know what caused what)
  • prevent w rules about timing and types of co-interventions permitted, record keeping and post hoc adjustment for imbalances
  • for both prevent with standardizing protocol, same freq and test type
32
Q

what is intention to treat analysis (ITT)?

A
  • aka “as randomized” analysis
  • patients analyzed within their allocated group (not according to what they received), whether or not they received, were adherent to, or completed the protocol
  • minimizes t1e (preserves prognostic balance of group)
  • can contribute to t2e bc of contamination
33
Q

what are exceptions to ITT? aka when is it less of a threat to validity to exclude patients from the analysis? and how are they removed?

A
  • patients without the disorder, patients never eligible for participation (accidentally entered through error)
  • must be removed w independent adjudication committee blind to group allocation before randomization!
34
Q

why does missing data threaten validity and does inflating n size fix the problem?

A
  • data are rarely missing for trivial reasons
  • no! - people who drop out cannot be replaced (bc you will get someone who is not the same as the people who dropped out)
  • want to see less than 20% missing overall, if % missing is diff btw groups may be due to treatment!
35
Q

define the 3 types of missing data

A
  • MCAR - does not depend on observed or unobserved outcome (ie car breaks down) - can threaten precision (smaller n-size) but not validity of study!
  • MAR - depends on variables unrelated to outcome (storm keeps all patients from Toronto from coming)
  • MNAR - related to the outcome (patient does not want to return bc they got better or worse) no imputation methods are appropriate for dealing w this type of data!
    • note that data missing completely at random only influences precision and any other types of missing data threaten t1e AND precision
36
Q

describe excluding all patients w missing data method

A
  • easy default to most statistical packages
  • increase t2e
  • threatens internal validity (does not follow ITT) - this increases possibility of t1e!
  • decreases precision (bc of decreasing n-size)
37
Q

describe assuming worst/best-case scenario method

A
  • assume worst for treatment/best for control (increasing t2e!)
  • not appropriate for longitudinal data w missing mid-point data (bc there are points on either side of the missing data)
  • may be overly conservative
  • usually used when we don’t know what happened at the end of a study
38
Q

describe last outcome carried forward method

A
  • may be too conservative
  • not appropriate for longitudinal w missing mid-point
  • trajectory of change is ignored
39
Q

describe growth curve analysis

A
  • good for mid-point data
  • requires at least 2 data points
  • not appropriate for missing endpoint data
  • decreases variability ie CI (increases chance of t1e)
40
Q

describe regression methods

A
  • allows for examination of longitudinal trends
  • decreases variability ie CI (increases chance of t1e)
  • where did the avg person score at 4 weeks for example
  • added precision, reduced noise
  • more missing data = worse
41
Q

describe multiple imputation methods

A
  • allows for longitudinal trend examination

- using more than one method, which adds come variability to data (decreasing t1e)

42
Q

describe mixed model analysis

A
  • for when no actual time is assigned
  • fixed and random
  • time recorded as “days since intervention” not by fixed visit interval
  • can’t be used for endpoint data
  • this is recommended bc it leaves all patients in analysis without making assumptions does not reduce variability too much (ie by inputting avg values for missing data points)
43
Q

is the validity of the study threatened by missing data?

A
  • imputation should only ever improve precision, validity should not change! if it does, this is an n-size issue (not big enough) - ie conclusion should not change!
  • see p 61