Reliability and Validity in Diagnosis and Classification

Question 1

Describe reliability

Answer 1

• The consistency of a diagnostic measure
• e.g. the level of agreement on the diagnosis by a psychiatrist across time (test-retest)

Question 2

Describe validity

Answer 2

The extent to which SZ is a unique syndrome with characteristic and symptoms distinct from other conditions

Question 3

What is inter-rater reliability?

Answer 3

The extent to which different assessors agree on their assessment

Question 4

How is inter-rater reliability measured?

Answer 4

By a kappa score between 0-1, where greater than (>)0.7 is good

Question 5

Give evaluation for reliability of diagnosis (low inter-rater reliability)

Answer 5

• Whaley found that inter-rater reliability correlations in the diagnosis of SZ was as low as 0.11.
• However, in later years, Osorio et al reports increased inter-rater reliability at 0.97 and 0.92 for test-retest reliability
• Suggests that, now, we can be more confident that the diagnosis of SZ is consistently applied where needed.

Question 6

Give evaluation for reliability of diagnosis (unreliable symptoms)

Answer 6

• For a diagnosis of SZ, only 1 of the characteristics may be required, ‘if delusions are bizarre’, but this creates problems for reliability of diagnosis
• e.g. Mojtabi and Nicholson found when 50 psychiatrists were asked to differentiate between ‘bizarre’ and ‘non-bizarre’, they produced an inter-rater reliability of about 0.40
• Shows that even this central diagnosis lacks sufficient reliability for it to be a reliable method of distinguishing between schizophrenic and non-schizophrenic patients.

Question 7

Give evaluation for reliability of diagnosis (cultural bias)

Answer 7

• e.g. British Afro Caribbean people are up to 9 times more likely to be diagnosed than white British people
• Shows that there’s an overinterpreting of symptoms in black British people showing that they may be discriminated against by a culturally-biased diagnostic system

Question 8

What is criterion validtity?

Answer 8

Considers if different assessment systems arrive at the same diagnosis for the same patient

Question 9

Describe co-morbidity as an issue in the diagnosis of SZ

Answer 9

Occurs when 2 or more conditions share symptoms. If conditions share many symptoms, it questions the validity of classifying the 2 conditions separately.

Question 10

Describe symptom overlap as an issue for diagnosis of SZ

Answer 10

The occurrence of 2 illnesses of conditions together. When 2 conditions are often found together it questions the validity of classifying the 2 conditions separately

Question 11

Give evaluation for validity of diagnosis (co-morbidity)

Answer 11

• e.g. Buckley et al found that 50% of people with a SZ diagnosis also have depression, and 47% had substance abuse
• This is a problem for classification as it means SZ may not exist as a distinct condition, questioning the validity

Question 12

Give evaluation for validity of diagnosis (research for low validity)

Answer 12

• e.g. Cheniaux et al had 2 psychiatrists independently assess the same clients using the ICD-10 and DSM-IV and found that 68% were diagnosed with SZ under the ICD and 39% under DSM
• Suggests SZ is either over- or underdiagnosed, depending on the diagnostic system, but either way has low criterion validity