Introduction to the design and evaluation of clinical trials for psychotherapy research Flashcards
What is the definition of psychotherapy by Campbell and colleagues (2012)?
“the informed and intentional application of clinical methods and interpersonal stances
- derived from established psychological principles
- for the purpose of assisting people to modify their behaviour, cognitions, emotions, and/or other personal characteristics
- in directions that the participants deem desirable”
What is the purpose and method to gather evidence for the foundational principles of psychology?
> Evidence-based therapy
> Evidence best derived from the application of robust and reliable empirical methods
What is the purpose of gathering evidence for the effectiveness of psychotherapy?
> Ethical
- use methods that we know are useful
> Economic
- use the most cost-effective approach
- acknowledging that one-size-fits-all approach is not appropriate
What are the 3 characteristics of evidence that can be used to demonstrate that a treatment is effective in healthcare (not specific to psychotherapy)?
- Robust
- Unbiased
- Objective
What is the problem of using anecdotal reports, expert opinions and testimonials for evidence?
Subjective and highly prone to bias
-> weak evidence
What is the problem of using observational studies, uncontrolled case studies and case series for evidence?
> Very subjective
Subject to possible bias from multiple sources
-> weak evidence
When are controlled case studies useful for evidence?
In the early stages of therapy development
- prelude to clinical trial
What is the problem of cohort studies?
> Observational
> Do not control for biases that might have led to one patient receiving a treatment and another not
What is the ‘gold standard’ for evaluating therapy outcomes?
Randomised controlled trials (RCTs)
- general paradigm for treatment evaluation
What can be considered as the strongest source of evidence for treatment effectiveness? Why?
Systematic reviews and meta-analyses
- combined RCTs
What are the steps of building a randomised controlled trial (RCT)?
- Feasibility
- demonstrate the the full RCT is feasible - Pilot
- demonstrate and refine methods
- estimate size of the treatment effect (plan number of patients needed for RCT) - Efficacy (most RCTs are efficacy trials)
- assess the treatment in restrictive context of trials - Effectiveness
- assess the treatment in real-world clinical setting
How do you demonstrate the feasibility of an randomised controlled trial (RCT)?
Feasibility trial (mini-RCT)
- run with some conditions of RCT
- clear operational plans, objectives
When and how do you assess the effectiveness of a treatment?
> When treatment is shown to be effective (efficacy trial)
> Multiple factors
What is the limit of effectiveness randomised controlled trials (RCTs)?
They are the most expensive (hundreds of thousands of pounds)
- assess the treatment in real-world clinical setting
- requires time and a lot of participants
-> they are the least commonly used
What are the 5 characteristics of a randomised controlled trial (RCT)?
- Careful selection of cases
- Two or more ‘treatments’
- Randomisation of cases to treatments
- Random allocation to condition is ‘blind’
- Repeated assessment of outcome measures
What is the process to carefully select cases in a randomised controlled trial (RCT)?
> Inclusion criteria: take part
Exclusion criteria: ruled out
e. g.:
- reduce between-patient variability
- remove obstacles to the sage and effective delivery of treatment
- remove clinical confounding factors
What are the disadvantages of restrictive criteria in a randomised controlled trial (RCT)?
> Problems recruiting sufficient numbers
> Unrepresentativeness and lack of generalisability of results (due to small number of participants)
Why does a randomised controlled trial (RCT) require at least two or more ‘treatments’?
Fundamental aim: is a treatment effective
-> active treatment vs. control treatment
What are the two types of randomised controlled trials?
> Superiority trial
- new treatment is better than existing or no treatment
> Inferiority trial
- new treatment is no worse than the existing treatment
What are the types of control treatments used in a randomised controlled trial (RCT)?
> Placebo controlled
> Standard care, treatment as usual (TAU)
> Waiting list controlled
> Current ‘gold standard’ treatment
What is a placebo controlled RCT?
Control = dummy treatment without active ‘ingredients’
- placebo response (biological or psychological)
What is the attention effect observed in clinical trials?
Patients in clinical trials receive lots of attention
- researchers showing considerable interest in symptoms
- maintaining contact over couse of the study
-> attention effect, potent in disorders such as depression
What is a standard care or treatment as usual (TAU) controlled RCT?
> Control = treatment or care that patient would usually receive
> For psychological problems, TAU = no treatment
What is a waiting list controlled RCT? What are its advantages?
Patients receive active treatment after a delay
vs. patient receive it immediately
- practical and ethical advantages over standard care comparison (TAU)
What is a current ‘gold standard’ treatment controlled RCT?
Control = best available treatment currently in use
- new treatment may be as effective but easier, cheaper, quicker to deliver, or preferred by patients, or needs less highly trained staff
Why are curent ‘gold standard’ treatment controlled RCTs very expensive?
Because both active and control treatments are effective, it requires a large number of patients
What is the process of randomisation of cases of treatments in a randomised controlled trial (RCT)?
Which patient receives which treatment must be determined randomly
- adresses researcher, treating therapist, and patient
> Equalisation of groups
Prevention of allocation bias
-> enables ‘blinding’: unable to tell which condition a patient has been allocated to
What is the equalisation of groups in a randomised controlled trial (RCT)?
Ensures groups are matched for severity of symptoms, age, gender and other factors
What is the prevention of allocation bias in a randomised controlled trial (RCT)?
Investigators can not choose between active or control treatments for a patient
- would otherwise result in major systematic bias
What are the potential disadvantages of the randomisation of cases to treatment in randomised controlled trials?
> Deterrent to recruitment
> Some patients may want to know what treatment they are going to receive
- hoping for effective one, and to receive it as quickly as possible
-> need for assessment of the feasibility of recruitment before commencing full scale trial
Why is the random allocation to condition ‘blind’ in randomised controlled trials?
> Clinical trial = experiment where no-one knows the outcome
- there are multiple opportunities for bias from assessor, patient and statistician
> Minimise these biases:
- avoid possibility of unconscious bias from assessor (expectancy)
- stop patients giving response they think doctors want to hear
- avoid patients focussing more on improvement attributed to active treatment
- avoid bias in choice of analyses or interpretation of results
> Sometimes full blindness is impossible
Which bias from the assessor is purposefully minimised in an RCT with the ‘blind’ random allocation to condition?
Expectancy bias:
- based on knowledge of which treatment condition patient is in, might nudge the researcher to give a symptom score more indicative of improvement in patients receiving the active treatment
Which blindness of allocation to condition is an absolute minimum in a randomised controlled trial (RCT)?
Blindness of the statistician
Why is there a repeated assessment of outcome measures in a randomised controlled trial (RCT)?
All clinical trials require a robust and reliable way to measure the symptom of condition of interest
What are the types of outcome measures in a randomised controlled trial (RCT)?
> Primary outcome measure
- main index used to measure effectiveness
- accurate and reliable: measure of symptoms with minimal error
- the more reliable the measure, the easier it is to detect treatment effect from trial
> Secondary outcomes
- measure other outcomes of interest
What are the types of repeated assessments in a randomised controlled trial (RCT)?
> Multiple assessment points
- repeat at baseline before randomisation
- before and after treatment
- main indicator of efficacy
- potential intermediate assessment points during treatment period
> Follow-up assessment
- to measure continuation of benefit after treatment ends
- the larger the better
What is the limit of follow-up assessments in randomised controlled trials (RCTs)?
The larger the better, but the more costly, particularly if blindness is maintained during follow-up
What are the types of clinically meaningful outcomes?
> Minimal Clinically Important Change (MCIC)
- a response greater than expected by chance or normal recovery
- may be expressed on self-report scale (e.g. Beck Depression Inventory)
> Defined response criteria
> Short-term remission of symptoms (relative to baseline)
> Lasting recovery
What are the three main challenges to designing and conducting robust RCT in psychotherapy research?
- Psychotherapy treatment
- complex intervention
- need to standardise how it is delivered - Choice of control treatment
- Allocation blindness
What makes the psychotherapy treatment part of an RCT challenging?
Standardisation:
> User treatment manual
- therapists should be trained to deliver therapy as defined in the manual
> Ensure adherence to treatment (Fidelity)
- therapists regularly checked to ensure they follow the manual
- variance of therapists may need to be incorporated in data analysis
> Assess therapist effects
- they may vary in expertise and other characteristics
What is a user treatment manual in an RCT?
Restrictive description of treatment and delivery
What makes the choice of control treatment in a psychotherapy RCT challenging?
> No psychotherapeutic equivalent to dummy pill
> Effective placebo control treatment almost impossible
> Requires two near identical treatments
> Must be equally plausible and convincing
- without true match placebo condition, patients risk to know whether they’ve been allocated the active or control treatment
- > risk of expectancy bias
> Extremely difficult to achieve in practice
> Regular assessments of treatment fidelity necessary
- particularly if therapists are delivering an alternative and presume less effective control treatment in which they have no faith
What is the most common control in a psychotherapy RCT?
Another active treatment
- typically antidepressant medication
What is the advantage of using antidepressant medication as control treatment in a psychotherapy RCT?
Ethical and practical advantage for recruitment
- all patients receive an effective treatment
What did the study of Elkin and colleagues (1989) consist of?
Large multicenter study
- cognitive therapy compared to other active treatments
Randomisation:
- cognitive therapy
- vs. interpersonal therapy (IPT)
- vs. antidepressant medication (tricyclic - TCA)
- vs. pill placebo
- N = 250 (239)
- Duration: 16 weeks treatment
- 32% stopped early
- Outcome criterion: Remission
- Completer patients analysis (N = 155) -> not significant
- Intention-to-treat (N = 239) -> significant
What was the result and limit of the study of Elkin and colleagues (1989)?
> Support for interpersonal therapy (IPT) as treatment for depression, and at lesser extent CBT
> Limit: sample size to low to test for differences between active treatments
What were the research questions of the study of Rahmen and colleagues (2008)?
> Could CBT help depressed mothers with infant children in rural Pakistan?
> Would improving the mothers mental health improve the health and early development of her new born children?
What did the study of Rahmen and colleagues (2008) consist of?
Applying CBT for depression in a low-income country
- delivered by existing primary healthcare workers
- trained to deliver CBT in centres randomised to active treatment
- other centres delivered typical care (enhanced, closer than usual contact)
- CBT vs. Enhanced typical care
- N = 903
- Duration = 16 sessions over 10 months
- Assessment: baseline, 6 and 12 months
- Outcome criterion: diagnosis, mother-baby interaction, infant health
What makes the intention-to-treat analysis a preferred method of analysis in clinical trials?
Avoids possible biases such as patients dropping out at different rates in each treatment group and for different reasons
What was the result of Rahmen and colleagues’ study (2008)?
> Cost effective CBT intervention has effects that go beyond the mothers’ acute mental health
- > Other positive outcomes:
- fewer cases of stunted growth
- fewer cases of diarrhoea
- increased rates of infant immunisation
- increased use of maternal contraception
- increased play with the mother and father
-> wider and potential long-term impact of untreated maternal depression on infant health and social adjustment
