Clinical Assessment and Predictions

Question 1

Clinical Assessment is composed of: (3)

Answer 1

1. Tests (IQ/general mental ability, Personality, Neuropsychology)
2. Behavioural Assessment + Contextual/Environmental information
3. Clinical interview (Structured/Unstructured)

Question 2

E.g. of Neuropsychology tests

Answer 2

• Bender Gestalt
• Luria-Nebraska

Question 3

Challenge with clinical assessment

Answer 3

Integrating the Information for Decision Making

Question 4

Phases in Decision Making (2)

Answer 4

• Data collection
• Data integration and prediction

Question 5

(Phases in Decision Making): Data Collection Phase - Description (2)

Answer 5

1. “Mechanical” scores
   
   • E.g. Questionnaires
2. “Judgmental” scores
   
   • E.g. Clinical interview

Question 6

(Phases in Decision Making) Data Integration Phase and Prediction - Description (2)

Answer 6

1. Clinical: Integrating material, writing reports with recommendation
2. Statistical: Combining information statistically

Question 7

Myth of Experience

Answer 7

Beyond certain amount of training, more experience does NOT translate into more accurate diagnosis

Question 8

Myth of more information

Answer 8

More information does NOT necessarily give more accurate predictions

Question 9

Myth of configurability/patterns

Answer 9

Clinicians’ decisions CAN be modelled using a formula

Question 10

Study: Goldberg (1965). Comparison of clinicians with statistical combinations. Description of study + Findings

Answer 10

Participants:13 Ph.D. staff and 16 predoc trainees
- Material:861 MMPI profiles
- Decision: Was the patient neurotic or psychotic?
- Criterion: Official hospital discharge diagnosis (either neurosis or psychosis)
Findings:
(1) No effect of training
(2) All statistical procedures did better than the clinicians’ average
-> Models based on judges’ decision patterns surpassed judges in accuracy, due to their perfect reliability.

Question 11

Goldberg (1965): What’s the Goldberg rule and what role did it have in the experiment?

Answer 11

Goldberg rule = stat rule for diagnosis (Add scores from 3 MMPI scales, subtract 2 others; score ≥ 45 = psychotic, < 45 = neurotic)
=> Judges, even after 4000 practice trials, couldn’t surpass the Goldberg Rule.

Question 12

Goldberg (1965): Are these results isolated or general?

Answer 12

Meehl (1965) - Meta-Analysis
N = 51 studies in different domains
-> Statistical BETTER than clinical: 33
-> Statistical EQUAL than clinical: 17
-> Statistical WORSE than clinical: 0

Question 13

Meehl et al.: Introduced the ____ comparison

Answer 13

clinical vs. actuarial comparison

Question 14

Meehl et al. established two conditions for fair comparison between clinical and actuarial judgment

Answer 14

1. Same Data Basis: Both methods should use the same data for judgment, though their development may rely on different datasets.
2. Cross-Validation: Apply decisions to new contexts (not only known outcomes) to ensure it would work in a real world setting

Question 15

Yes, stat seem better than clinical judgment. HOWEVER, there seem to be a limit on the ____ of those studies

Answer 15

Generalizability
-> Many actuarial rules are context-specific, though some (e.g., Goldberg Rule) have broader applications.

Question 16

Meehl et al - Meta analysis: Are These Results Still True?

Answer 16

Grove et al. (2000):
Statistical prediction > Clinical prediction in 33-47% of studies
Clinical prediction > Statistical prediction in 6-16% of studies

Question 17

A hybrid of clinical and actuarial methods is possible but often impractical, why?

Answer 17

• Impractical for dichotomous decisions (e.g., prescribing medication, granting parole).
• Agreement between methods makes combination unnecessary; disagreement forces a choice between one or the other.

Question 18

Meehl et al - Meta analysis: Statistical prediction improves accuracy by about ____

Answer 18

10%

Question 19

Why Aren’tClinicians Better at Prediction? (3 errors)

Answer 19

(1) Do not apply their knowledge uniformly or reliably
(2) Overweight positive instances
(3) Similarity Heuristic

Question 20

(Why Aren’tClinicians Better at Prediction?) Do not apply their knowledge uniformly or reliably, explain. (3)

Answer 20

• Distraction
• Fatigue
• Mood

Question 21

[Why Aren’tClinicians Better at Prediction?] Overweight positive instances, explain. (3)

Answer 21

(1) Do NOT consider false positives
(2) Do NOT consider base rates
(3) Can be incorrect with an INVALID predictor to the extent that your guess is consistent with the base rate and the base rate is high.
-
E.g., If Marc receives a score indicating moderate depression, will he attempt suicide?
→ People remember the true positives and say yes
→ BUT other types of data are relevant: e.g. depressed individuals who DIDN’T commit suicide (FALSE positive) and those who weren’t depressed but who committed suicide (FALSE negatives).

Question 22

(Why Aren’tClinicians Better at Prediction?) Similarity Heuristic

Answer 22

I.e. Individuals predict future behaviour that RESEMBLES test information (E.g. The test-taker tells violent stories on TAT → Prediction = person will assault others)
-> Ignores base rates
-> Ignores VALIDITY of test information

Question 23

Even if they (wrongfully) use the Similarity heuristic, clinicians might still be correct about the diagnosis IF: _____

Answer 23

The BR for phenomena is HIGH/The test information is in fact VALID

Question 24

Conclusion: Why Aren’tClinicians Better at Prediction? (3 bad practices)

Answer 24

1. We overweight successful predictions & do NOT remember incorrect predictions
2. We do NOT think base rates
3. We do NOT use all the available information

Question 25

Improving Clinical (human) Judgment (3)

Answer 25

1. Systematically consider alternatives
2. Collect feedback about decisions/predictions
3. Think about statistical prediction issues

Question 26

What Can Clinicians (Humans) Do Well? (3)

Answer 26

1. Provide input into statistical model
2. Generate hypotheses
3. Provide prediction when no formula (yet) exists