Reliability of classification systems

Question 1

What is meant by ‘reliability of mental health diagnosis’?

Answer 1

The extent to which two or more different psychologists are in agreement and come to the same diagnosis for a mental health condition

Question 2

What is the clinical psychology definition of ‘reliability’?

Answer 2

The consistency of a clinical diagnosis when using classification systems

Question 3

What is the clinical psychology definition of ‘inter-rater reliability’?

Answer 3

The extent to which two or more different clinicians are in agreement and arrive at the same clinical diagnosis when diagnosing the same patient

Question 4

What is the clinical psychology definition of ‘INTRA-rater reliability’?

Answer 4

When the same clinician diagnoses the same patient with the same symptoms as having the same disorder at two different points in time

Question 5

What is the clinical psychology definition of ‘test-retest reliability’?

Answer 5

Two or more clinicians observing the same patients separately within a period of time where the clinical conditions of the patients are unlikely to have changed

Question 6

What was the purpose of the DSM-5 Field Trials?

Answer 6

To evaluate the test-retest reliability of mental diagnosis

Question 7

What was the procedure of the DSM-5 Field Trials? (3 points)

Answer 7

Took place in 11 academic centres in US & Canada

2 clinicians interviewed each patient

Clinicians were blind to previous diagnosis of patient to reduce researcher bias

Question 8

What were the results of the DSM-5 Field Trials? (4 points)

Answer 8

Kappa score for:
Sz = 0.46
PTSD = 0.67
Manic depressive disorder = 0.28

Diagnoses that showed unacceptable agreement in children:
+ Mixed anxiety disorder
+ Non-suicidal self-injury diagnosis in children

Question 9

What are the 4 supporting and 5 refuting points that support the reliability of the DSM classification system?

Answer 9

Supporting:
Regier et al (2013)
Brown et al (2001)
The DSM standing the test of time
The DSM-5 removing the culture-bound syndromes list

Refuting:
Spitzer and colleagues (2012)
Rosenhan (1973)
Cooper et al. (1972)
Co-morbidity
Loring and Powell (1988)

Question 10

How does Regier et al support the reliability of the DSM classification system? (2 points)

Answer 10

Found that the DSM-5 field trials show the reliability of the manual

Found some of the highest levels of agreement and reliability between clinicians resulting in kappa scores of 0.67. (very good) for disorders like PTSD

Question 11

How does Brown et al support the reliability of the DSM classification system? (2 points)

Answer 11

Found good reliability in clinical diagnosis when using DSM-4 in 1,127 outpatients

Good to excellent consistency for most of the DSM categories for generalised anxiety disorders and mood disorders

Question 12

How does the DSM standing the test of time support its reliability? (2 points)

Answer 12

Despite the revisions, it is still one of the most widely used classification systems in mental health

The DSM-5 fine-tunes diagnoses like ‘eating disorder’ to ‘binge eating disorder’ - more specific and reliable which enables clinicians to come to a common diagnosis

Question 13

How does the DSM-5 removing the culture-bound syndromes list support the reliability of the DSM classification system? (3 points)

Answer 13

“Culture-bound syndromes” list was replaced with advice on “Cultural Concepts of Distress’’ which has increased multicultural perspectives

Has resulted in positive contributions to cross-cultural diagnosis by making new conditions known outside of the USA

E.g. ADHD and Borderline Personality Disorder are now recognised in the UK

Question 14

How do Spitzer and colleagues (2012) refute the reliability of the DSM classification system? (4 points)

Answer 14

They argued that a kappa score below 0.60 would be concerning, even considering the DSM-5’s test-retest methodology

DSM-5 field trials show only 5 out of 23 diagnoses were found to be ‘very good’

This raises questions about the reliability of conditions like Sz, which has a kappa score of 0.46

Suggests there are many clinical diagnoses which remain unreliable among different clinicians

Question 15

How does Rosenhan (1973) refute the reliability of the DSM classification system? (2 points)

Answer 15

Further highlighted inconsistency in diagnosis - found a tendency for Type I errors in mental health hospitals

Suggests that diagnosing the clinically ‘insane’ from the ‘sane’ was unreliable using the DSM-4 in the USA

Question 16

How does Cooper et al (1972) refute the reliability of the DSM classification system? (2 points)

Answer 16

They showed American and British psychiatrists the same videotaped interview and asked them to make a diagnosis

New York psychiatrists said it was schizophrenia twice as often, whereas the London psychiatrists said it was depression twice as often

Question 17

How does the concept of comorbidity refute the reliability of the DSM classification system? (3 points)

Answer 17

Comorbidity describes people who suffer from two or more mental disorders
E.g. Sz and depression are often found together, making it more difficult to confidently diagnose Sz

Occurs because the symptoms of different disorders overlap
E.g. major depression and Sz both involve very low levels of motivation

Buckleyet al(2009)
Other disorders they found Sz patients were diagnosed with:
Depression: 50%
PTSD: 29%
OCD: 23%

Question 18

How do Loring and Powell (1988) refute the reliability of the DSM classification system? (2 points)

Answer 18

They found that some behaviour which was regarded as psychotic in males was not regarded as psychotic in females - gender and cultural bias present in the diagnosis of Sz

Men are far more likely to be diagnosed, potentially because:
+ Women experience the symptoms differently to men
+ Much of the clinical research remains androcentric

Question 19

What are the supporting and refuting points for the reliability of the ICD? (2 points)

Answer 19

Supporting: Jakobsen et al. (2005)

Refuting: Nicholls et al (2000)

Question 20

How does Jakobsen et al support the reliability of the ICD? (3 points)

Answer 20

Tested the reliability of the ICD-10 classification system in diagnosing Sz

Results showed a concordance rate of 98% in 100 Danish patients with a history of psychosis

Demonstrates the high reliability of the clinical diagnosis of Sz using up-to-date classification

Question 21

How does Nicholls et al refute the reliability of the ICD? (2 points)

Answer 21

Found a low level of agreement of 36% for eating disorders when using the ICD-10

Suggests the reliability of classification systems is a universal problem and is not limited to the DSM