Week 5: Efficacy of Cognitive Therapy: Design and Evaluation of Clinical Trials for Psychotherapy Research Flashcards
Psychotherapy
Psychologically based approach that seeks to help a person change or overcome mental and/or physical problems
Psychotherapy (APA)
It is the informed and intentional application of clinical methods and interpersonal stances derived from established psychological principles to assist people in modifying their behaviors, cognitions, emotions, and/or other personal characteristics in directions that the participants deem desirable.
Psychotherapy Approaches
- Cognitive
- Behavioral
- Humanistic
- Psychoanalysis
Evidence-based therapy
Evidence is best derived from the application of robust and reliable empirical methods.
Evidence
Key to supporting both the underlying principles and the therapy’s effectiveness.
Robust, unbiased and objective.
Hierarchy of Evidence / Evidence Pyramid
Top to Bottom:
1. Systematic Reviews and Meta-analyses
2. Randomized Controlled Trials
3. Cohort Studies
4. Controlled Case Studies and Case Series
5. Observational Studies, uncontrolled case studies
6. Anecdotal reports, expert opinions, testimonials
Anecdotal reports, expert opinions, testimonials
Are personal stories or experiences that are highly subjective and prone to bias
Observational Studies and uncontrolled case studies
Very subjective and possibly biased from multiple sources
Case Studies
Detailed examinations of a single individual or a small group.
An attempt to control for at least some of the factors that may contribute to observable outcomes.
Useful in the early stages of therapy development.
Controlled Case Studies
Careful matching of the characteristics of the
patients in two groups, either overall or on a case-by-case basis
Cohort Studies
Another observational approach in which typically large
groups of patients are compared, one of which received the therapy and the other of which did not.
Do not control for biases that might have led to one patient receiving treatment and another not.
Randomized Controlled Trials
Gold standard for evaluating therapy outcomes.
Patients are randomly assigned to either a treatment group or a control group to minimize bias.
Aims to establish cause-and-effect relationships between a treatment and an outcome.
Systematic Reviews and Meta-analyses
Combining the evidence from multiple RCTs, and
indeed from all available evidence types.
Represents the top of the evidence hierarchy.
Aims of Randomized Controlled Trials (RCT)
- Feasibility: Determine practicality and feasibility
- Pilot: Demonstrate and refine methods. Estimate the size of the treatment effect.
- Efficacy: Assess the effectiveness of treatment in the restrictive context of trials
- Effectiveness: Assess the effectiveness of treatment in the real world. Least commonly done and most expensive.
Key Features of Randomized Controlled Trials (RCT)
- Careful selection of cases
- Two or more ‘treatments’
- Randomization of cases to treatments
- Random allocation to condition is ‘blind’
- Repeated assessment of outcome measures
Careful Selection of Cases
- Inclusion Criteria and Exclusion Criteria
- Reduce between-patient variability
- Remove obstacles for the safe and effective delivery of treatment
- Remove clinical confounding factors
Inclusion Criteria
Characteristics subjects must show to take part in the study
Exclusion Criteria
Characteristics that rule subjects out from taking part in the study
Disadvantages of Restrictive Criteria in Selection of Cases
- Narrow eligibility criteria (insufficient participants)
- Time and cost to expand eligibility criteria
- Limited representativeness
- Lack of generalizability results (External validity)
Internal Validity
Focuses on the accuracy of the study’s findings within the specific context of the study itself (i.e., IV affects DV and not other variables)
External Validity
Applicability of the study’s findings to other contexts or generalized to other populations, settings, or condition
Treatment Comparisons in RCTs
- Placebo-Controlled
- Standard Care / Treatment as Usual (TAU)
- Waiting List Controlled
- Current ‘Gold Standard’ treatment
Fundamental Aim of RCT
Determine whether a treatment is effective
Placebo Treatment / Control
Dummy treatment without active ingredients.
Isolate the effects of the actual treatment by accounting for the placebo effect.
Placebo Response / Effect
This is not just the patient imagining that they are benefiting.
Can produce biological changes in disease processes, as well as psychological changes.
This phenomenon shows a person experiencing improvement in their condition due to the belief that they are receiving treatment, even if the treatment is inactive.
Standard Care / Treatment as Usual (TAU)
Treatment or care that the patient will usually receive.
Often means no treatment.
Waiting List Controlled
Compare a group of patients who receive treatment immediately and those on a waiting list to be treated.
This has both practical and ethical advantages over a standard care comparison.
Types of Trials
- Superiority Trial
- Non-inferiority Trial
Superiority Trial
New treatment is BETTER than existing standard or no treatment
Non-inferiority Trial
New treatment is NO WORSE than the existing treatment
Attention Effects
Patients taking part in clinical trials receive lots of attention, with researchers showing considerable
interest in their symptoms, and maintaining contact throughout the study. In the context of depression, feeling understood, supported, and monitored can lead to improvements, even without specific treatment.
Randomization of Cases to Treatments
Which patient receives which treatment must be determined randomly.
Advantages of Randomization
- Equalization of Groups
- Prevention of Allocation Bias
- Enables ‘blinding’
Blinding
The ability to hide which condition a patient has been allocated to
Disadvantages of Randomization
Obstacles in Recruitment
1. Some patients want to receive the treatment ASAP
2. Some patients want to know what treatment they are going to receive
Randomization
Assigning participants to different treatment groups in a clinical trial by chance ensures that each participant has an equal probability of being assigned to any group.
Minimize bias and create comparable groups for evaluating treatment effectiveness.
Clinical Trial
A type of research study that tests a behavioral intervention in people to determine if a new intervention is safe and effective.
Random Allocation to Condition is ‘Blind’
Minimize multiple opportunities for:
- Assessor Bias
- Patient Bias
- Statistician Bias
- Other biases: Optimism, unconscious, and expectancy
For all of these reasons, every effort is made in trials to ensure that investigators, researchers, patients, and statisticians are blind to which treatment a patient is receiving, and blindness is maintained until after the trial is complete and the results are revealed.
Unconscious Bias (AKA Unintentional / Implicit Bias)
The influence of expectations and beliefs on our perceptions, memories, attitudes, and behaviour
Expectancy Bias
Based on knowledge of which treatment condition the patient is in, might nudge the researcher to give a score more indicative of improvement in the patients receiving the active treatment.
Optimism Bias
All of those involved will typically hope AND believe that it will show that the new treatment is effective or better than an existing treatment
Blinding
Minimizing knowledge of treatment conditions helps to reduce the potential for conscious or unconscious bias in data collection, analysis, and interpretation.
Blinding Techniques
Single-blind: Participants are unaware of their treatment group.
Double-blind: Both participants and researchers are unaware of treatment assignments.
Triple-blind: Participants, researchers, and data analysts are unaware of treatment assignments.
Outcome Measures
All clinical trials require a robust and reliable way to measure the symptoms or condition of interest, the target of the intervention.
Repeated Assessment of Outcome Measures
- Primary Outcome Measure
- Accurate and Reliable
- Secondary Outcomes
- Multiple Assessment Point
- Follow-up Assessment
- Statistically Significant Results
Primary Outcome Measure
Main index to measure effectiveness
Secondary Outcomes
Measure other/additional outcomes of interest
Accuracy and Reliability
The more reliable the measure, the easier it is to detect the accuracy of a treatment effect from the trial.
Multiple Assessment Point
Comparison of the outcome measure before and after
treatment is the main indicator of its efficacy.
Follow-up Assessment
Supports the hope that clinical benefits will persist after active treatment is stopped. The longer the period, the better, but also the more costly, particularly if blindness needs to be maintained during the follow-up period.
Statistically Significant Results
Clinical trials need to demonstrate that results are robust and not likely due to chance.
This does not mean that the results are necessarily clinically meaningful.
Clinical Significance / Clinically Meaningful Outcome
Clinical trials often have additional criteria for significance:
- Minimal Clinically Important Change (MCIC) - smallest change meaningful to patients
- Defined ‘response’ criteria (reduction of symptoms)
- Short-term remission (improvement in symptoms)
- Full recovery (lasting effects)
Challenges of the RCT in Psychotherapy Research
- Psychotherapy Treatment
- Choice of Control Treatment
- Allocation Blindness
Challenges of RCT: Psychotherapy Treatment
- Complex intervention
- Standardized measures are important for consistency but may limit therapist flexibility
Solutions to Psychotherapy Treatment (Challenges of RCT)
- User Treatment Manual
- Ensure Adherence to Treatment (Fidelity)
- Assess Therapist Effects
User Treatment Manual
Therapist’s guide to be trained in the description of treatment and delivery
Ensure Adherence to Treatment (Fidelity)
The therapist following the manual needs to be checked regularly to ensure they are adhering to it.
Therapist Effects
Therapists will vary in terms of personal and interpersonal characteristics, such as expressed empathy, warmth, and compassion.
Nonspecific Factors
Factors that can be measured and, while hard to control, may need to be incorporated into the data analysis as additional explanatory factors for outcome.
Often interpersonal and relational in nature.
Examples of Nonspecific Factors
Therapist characteristics: Empathy, warmth, genuineness, and competence.
Therapeutic alliance: The quality of the relationship between the therapist and the patient.
Patient expectations: The patient’s beliefs about the effectiveness of the therapy.
Challenges of RCT: Choice of Control Treatment
- No ‘dummy pill’ equivalent
- True match placebo is almost impossible
- True control: Requires two nearly identical treatments that must be equally plausible and convincing which is extremely difficult to achieve in practice
Solutions to Choice of Control Treatment (Challenges of RCT)
- Compare to alternative real treatment / another active treatment (i.e., antidepressants)
- Controls: Treatment as Usual (TAU) or Waiting List
- Attention placebo
Attention Placebo
Showing that the patients have a similar amount of contact with the therapist or other researchers without any of the active components of the therapy itself.
Can mitigate some of the potential placebo components in the active treatment, but not all.
Reporting Bias
Patients are aware of whether they are receiving the active treatment or the control condition. This knowledge can influence how they report their symptoms and experiences, potentially biasing the results.
True Match Placebo
n intervention that is indistinguishable from the active treatment in all respects (therapist-patient relationship, session structure, and attention given to the patient) except for the therapeutic component.
Challenges of RCT: Allocation Blindness
- The therapist will be aware of which treatment they are delivering
- Almost impossible to achieve patient and therapist blindness
Solutions to Allocation Blindness (Challenges of RCT)
- With care, the patients do not reveal which treatment they have been receiving
- Use of independent assessors who are unaware of treatment allocation
- Statisticians blind to allocation
Systematic Bias
Consistent, repeatable error that occurs during data collection or analysis, leading to an inaccurate or misleading result.
Hamilton Depression Rating Scale (HDRS / HAM-D)
A widely used instrument for assessing the severity of depression. It’s a clinician-administered scale.