5:4: Evaluating the efficacy of cognitive therapy

Question 1

List three aims of RCTs.

Answer 1

1. Feasibility: Trial is feasible, with resources that are available.
2. Pilot: Demonstrate and refine methods. Estimate the size and the treatment effects.
3. Efficacy: Assess the treatment in restrictive context of trials.

Question 2

What is a feasibility trial?

Answer 2

A mini-RCT

It is run under the same conditions as the planned full RCT, with clear operational plans and objectives.

Question 3

True or false: A feasibility RCT seeks to assess whether or not the treatment works.

Answer 3

False. It only seeks to assess that the planned trial can be delivered.

Question 4

What is a pilot RCT?

Answer 4

Incorporates some assessments of feasibility but aims are broader and are designed to inform the design and plans for the full RCT.

May include work on developing and refining the treatment and provide information on the likely size of any treatment effect.

Question 5

What is an efficacy trial?

Answer 5

Most full RCTs are efficacy trials.

Shows whether the active treatment shows a significant effect compared to a controlled treatment under the precise conditions of the clinical trial.

Question 6

What are effectiveness RCTs?

Answer 6

After a course of a number of separate RCTs, it seeks to answer whether the treatment works equally effectively in the real world, where there can multiple factors that can complicate outcome.

Question 7

RCTs define precise characteristics of patients that will take part. What two criteria are considered?

Answer 7

Inclusion criteria - includes the characteristics patients must show.

Exclusion criteria - any characteristics that rule them out from taking part.

Question 8

The less variability between patients, the easier it is to show:

Answer 8

A statistical effect of the treatment.

Question 9

List two disadvantages of carefully defined patient criteria.

Answer 9

1. The more restrictive the criteria, the narrow the range of patients that can take part. Leads to many failed RCTs.
2. Narrow criteria may mean that the sample of patients selected are not representative of the broad range of patients that might ultimately benefit from the treatment.

Question 10

List four controls in RCTs.

Answer 10

1. Placebo
2. Standard care, care as usual
3. Waiting list
4. Compared to current gold
   standard treatment

Question 11

Demonstrating that a new treatment is better than no treatment or an existing one is called a:

Answer 11

Superiority trial.

Question 12

Demonstrating that a new treatment is no worse than an existing treatment is called a:

Answer 12

Non-inferiority trial.

Question 13

What are three advantages of randomization of which patient receives which treatment in an RCT?

Answer 13

1. Equalization of groups: ensures groups are matched for severity of symptoms, age, gender, etc.
2. Prevention of allocation bias: doesn’t give the treatment to clients who might benefit most.
3. Enables blinding: Unable to tell which condition a patient has been allocated to.

Question 14

Four clinically meaningful outcomes of RCTs include:

Answer 14

1. Minimally Clinically Important Change (MCIC).
2. Defined “response” criteria.
3. Short-term remission of symptoms.
4. Lasting recovery.

Question 15

In RCTs, the more complex an intervention, the more important it is to:

Answer 15

Standardize how it is delivered to minimize:

• variability between patients receiving treatment
• variability between therapists delivering it
• variability between different centers that may be involved.

Question 16

What is a treatment fidelity check?

Answer 16

Regular checks on the degree to which the therapist is following the manual of the RCT.

• Ensures therapists don’t lapse back into prior habits as the trial progresses.

Question 17

In RCTs, any decision whether to recommend the use of a psychological treatment requires evidence that it is at least as effective as:

Answer 17

Medication.

Question 18

What is the preferred method of analysis in RCT clinical trials?

Answer 18

Intention-to-treat analysis.

It avoids possible biases that might occur by patients dropping out of the control group and other treatment groups at different rates and for different reasons.

Question 19

What are four purposes of systematic reviews?

Answer 19

1. Collate and summarize evidence.
2. Permit evidence-based recommendations about useful treatments.
3. Identify gaps and shortcomings in our evidence.
4. Generate new hypotheses that can be explored.

Question 20

Why do systematic reviews require a clearly stated set of objectives (questions)?

Answer 20

Needs a clear focus that can be specified as one or more specific questions.

Too broad or failure to define clearly will not be sufficient.

Question 21

Why do systematic reviews require pre-defined eligibility criteria for studies?

Answer 21

Sets the scope of their assessment.

Limits the generalizability of results.

Question 22

Why do systematic reviews require systematic literature search strategy?

Answer 22

Used to identify relevant papers for review and analysis.

Eligibility criteria are converted into standard search terms to be used when searching the various databases of published studies to identify relevant papers for analysis.

Question 23

Why do systematic reviews require explicit, reproducible methodology?

Answer 23

Increases the reliability of the review process itself and minimized bias on the part of the reviewer.

Question 24

Why do systematic reviews require an assessment of the quality of the study?

Answer 24

Greater importance or weight is attached to higher quality studies.

Question 25

Why do systematic reviews require systematic presentation and synthesis of studies and findings?

Answer 25

Findings are presented in a systematic way, often in tables summarizing details, or presented graphically.

Question 26

What does GRADE Quality Assessment assess?

Answer 26

Ratings of 1 (low) to 4 (highest) quality of a study.

Question 27

What is a standardized effect size?

Answer 27

A way of describing results of different studies in a standardized way before combining them in a meta-analysis.

Question 28

What is continuous data?

Answer 28

Outcomes that are assessed on a continuous scale, like scores on a depression rating scale.

Question 29

How do you standardize raw difference scores (results of two treatments, A and B, before and after therapy)?

Answer 29

Divide the mean difference by the standard deviation of all participants at baseline.

Question 30

What two statistics do standardized different scores typically use?

What mean differences are considered large, moderate or small?

Answer 30

Cohen’s d index

Hedges g index

A standardized mean difference of:

0.8 or above is regarded as large.

O.5 is moderate.

0.2 is small.

Question 31

What does the relative risk ratio measure?

Answer 31

Prevention of relapse. Relative risk that relapse will be higher or lower in the effective treatment.

Question 32

What are “events” in RCT tables?

Answer 32

They show the number, and the intervention, and the control group that show an outcome.

Question 33

What is a confidence interval?

Eg. a 95% confidence interval

Answer 33

It describes the range within which we would expect the true result to fall if we were to repeat the trial (95) times.

Question 34

What is a forest plot?

Explain it’s characteristics.

Answer 34

A graphical representation of the relative risk ratio.

• The relative risk is shown by a blue square (the bigger the square the bigger the sample size).
• Each square has two lines on either side.
• The wider the two lines, the less confident we are in the results.
• If one of the bars overlaps the midpoint line, the study will not have been statistically significant.
• The black diamond shape shows a pool of the evidence to give an overall estimate of the treatment effect.

Question 35

What are the 5 R’s of outcome?

Answer 35

Response
Remission
Recovery
Relapse
Recurrence

Question 36

At what percent of severity is the patient said to have shown a positive treatment response in the acute phase? Ie. Response to outcome.

Answer 36

50%

Question 37

Define remission

Answer 37

No or few symptoms for 1-2 months.

Question 38

Define recovery.

Answer 38

Sustained remission of between 6-12 months.

Question 39

Define relapse.

Answer 39

Another episode of depression after remission but before recovery.

Question 40

Define recurrence.

Answer 40

Episode of depression after recovery.

Question 41

What is a predictor?

Answer 41

A general term to describe any factor that on its own or in combination predicts clinical outcome.

Question 42

What are some examples of nonspecific predictors?

Answer 42

Eg: Characteristics of the patients at baseline that explain some of the variance and subsequent outcome regardless of which treatment they received.

Eg: The experience of those providing the treatment with less experienced therapists having poor outcomes regardless of the specific therapy they are delivering.

Question 43

What is a moderator?

Answer 43

It is also a predictor but the term is used to define those baseline or other characteristics that interact with the treatment and influence the size of the eventual treatment effect.

Question 44

The search into moderators addresses what question?

Answer 44

On whom and under what conditions do treatments have different effects.

Question 45

If patients who have more severe symptoms at baseline get better more quickly than patients with mild symptoms regardless of the treatment, severity =

Answer 45

Nonspecific PREDICTOR of outcome.

Question 46

If symptom severity only had an influence in those patients receiving the active treatment, severity =

Answer 46

MODERATOR of the treatment effect.