Reliability And Validity In Diagnosis And Classification❌

Question 1

What is reliability?

Answer 1

Consistency.

The consistency of measurements.
We would expect any measurement to produce the same data if taken on successive occasions.

Question 2

For a diagnosis of schizophrenia to be reliable, what must happen?

Answer 2

Has to be repeatable - clinicians must be able to reach the same conclusions at two different points in time (test-retest reliability), or different clinicians, must reach the same conclusions (inter-rater reliability).

Question 3

What does test-retest reliability mean?

Answer 3

Clinicians must be able to reach the same conclusions at two different points in time

Question 4

What does inter-rater reliability mean?

Answer 4

Different clinicians, must reach the same conclusions.

Question 5

How is inter-rater reliability measured?

Answer 5

By a statistic called a kappa score.

A score of 1 indicates perfect inter-rater agreement; a score of 0 indicates no agreement.
A score of 0.7 above is generally considered good.

Question 6

What is culture?

Answer 6

The rules, customs, morals, childrearing practices, etc, that bind a group of people together and define how they are likely to behave.

Question 7

What are the studies that suggest there is a cultural difference in diagnosis? (Reliability)

Answer 7

Copeland

Luhrmann

Question 8

What did Copeland do? (Reliability)

Answer 8

(1971)
Gave 134 US and 194 British psychiatrists a description of a patient.
69% of the US psychiatrists diagnosed schizophrenia, but only 2% of the British ones gave the same diagnosis.

Question 9

What did Luhrmann do? (Reliability)

Answer 9

(2015)
Interviewed 60 adults diagnosed with schizophrenia - 20 each in Ghana, India and the US.
Each was asked about the voices they heard.

While many of the African and Indian subjects reported more positive experiences with their voices, describing them as playful or offering advice, not one American did.
The US subjects were more likely to report the voices they heard as violent and hateful - and indicative of being ‘sick’.

Luhrmann suggests that the ‘harsh, violent voices so common in the West may not be an inevitable feature of schizophrenia.

Question 10

What is validity?

Answer 10

Refers to whether an observed effect is a genuine one.

Question 11

What does gender bias mean? (Validity)

Answer 11

Refers to the tendency to describe the behaviour of men and women in psychological theory and research in such as way that might not be seen to represent accurately the characteristics of either one of these genders.

Question 12

When does gender bias occur in schizophrenia?

Answer 12

When accuracy of diagnosis is dependent on the gender of an individual.

The accuracy of diagnostic judgments can vary for a number of reasons, including gender-biased diagnostic criteria or clinicians basing their judgments on stereotypical beliefs held about gender.

Question 13

What’s an example of gender bias?

Answer 13

Critics of the DSM diagnostic criteria argue that some diagnostic categories are biased towards pathologising one gender rather than the other.

Question 14

What’s a study about gender bias?

Answer 14

Broverman (1970) found that clinicians in the US equated mentally healthy ‘adult’ behaviour with mentally healthy ‘male’ behaviour.

As a result, there was a tendency for women to be perceived as less mentally healthy.

Question 15

What is symptom overlap? (Validity)

Answer 15

Refers to the fact that symptoms of a disorder may not be unique to that disorder but may also be found in other disorders, making accurate diagnosis difficult.

Question 16

What other disorders have the same symptoms as schizophrenia?

Answer 16

Bipolar disorder

Depression

Question 17

What’s a study about symptom overlap?

Answer 17

Ellason and Ross (1995) point out that people with dissociative identity disorder (DID) actually have more schizophrenic symptoms than people diagnosed as being schizophrenic.

Most people who are diagnosed with schizophrenia have sufficient symptoms of other disorders that they could also receive at least one other diagnosis (Read, 2004).

Question 18

What is co-morbidity? (Validity)

Answer 18

Refers to the extent that two (or more) conditions or diseases occur simultaneously in a patient, for example schizophrenia and depression.

Question 19

What are the evaluative points for reliability?

Answer 19

Lack of inter-rater reliability
Unreliable symptoms
A comment on cultural differences in the diagnosis of schizophrenia

Question 20

What is meant by lack of inter-rater reliability?

Answer 20

Despite the claims for increase reliability in DSM-III (and later revisions), over 30 years later there is still little evidence that DSM is routinely used with high reliability by mental health clinicians.

For example, Whaley (2001) found inter-rater reliability correlations in the diagnosis of schizophrenia as low as 0.11.
Further problems with the inter-rater reliability of the diagnosis of schizophrenia are illustrate in the Rosenhan study.

This suggests that, because psychiatrists diagnosis lacks some of the more objective measures enjoyed by other beaches of medicine, it inevitably faces additional challenges with inter-rater reliability.

Question 21

What is meant by unreliable symptoms?

Answer 21

For a diagnosis of ‘schizophrenia’, only one of the characteristic symptoms is required ‘if delusions are bizarre’. However, this creates problems for reliability of diagnosis.

When 50 senior psychiatrists in the US were asked to differentiate between ‘bizarre’ and ‘non-bizarre’ delusions, they produced inter-rather reliability correlations of only around 0.40 (Moijabi and Nicholson, 1995).

The researchers concluded that even this central diagnostic requirement lacks sufficient reliability for it to be a reliable method of distinguishing between schizophrenic and non-schizophrenic patients.

Question 21

What is meant by a comment on cultural differences in the diagnosis of schizophrenia?

Answer 21

Research (e.g. Barnes, 2004) has established cultural, and racial, differences in the diagnosis of schizophrenia. However, the prognosis for members of ethnic minority groups may be more positive than for majority group members.

The ethnic culture hypothesis predicts that ethnic minority groups experience less distress associated with mental disorders because of the protective characteristics and social structures that exist in these cultures.

Brekke and Barrio (1997) found evidence to support this hypothesis in a study of 184 individuals diagnosed with schizophrenia or a schizophrenia-spectrum disorder.
The sample was drawn from two non-white minority groups (African Americans and Latinos) and a majority group (white Americans).

Consistent with the predictions of the ethnic culture hypothesis, they found that non-minority group members were consistently more symptomatic than members of the two ethnic minority groups.

Question 23

What are the evaluative points for validity?

Answer 23

Research support for gender bias in diagnosis
The consequences of co-morbidity
Differences in prognosis

Question 24

What is meant by research support for gender bias in diagnosis?

Answer 24

Loring and Powell (1988) found evidence of gender bias among psychiatrists in the diagnosis of schizophrenia.

They randomly selected 290 male and female psychiatrists to read two case vignettes of patients behaviour. These psychiatrists were asked to offer their judgement using standard diagnostic criteria.
When the patients were described as ‘males’ or no information was given about their gender, 56% of the psychiatrists gave a diagnosis of schizophrenia.
However, when the patients were described as ‘female’, only 20% were given a diagnosis of schizophrenia.

This gender bias was not as evident among the female psychiatrists, suggesting that diagnosis is influenced not only by the gender of the patient but also the gender of the clinician.

Question 25

What is meant by the consequence of co-morbidity?

Answer 25

A number of studies have examined single co-morbidities with schizophrenia, but these studies have usually involved only relatively small sample sizes.

By contrast, Weber (2009) looked at nearly 6 million hospital discharge records, findings evidence of many co-morbid non-psychiatric diagnoses.
Many patients with a primary diagnosis of schizophrenia were also diagnosed with medical problems including asthma, hypertension and type 2 diabetes.

They concluded that the very nature of a diagnosis of a psychiatric disorder is that patients tend to receive a lower standard of medical care, which in turn adversely affects the prognosis for patients with schizophrenia.

Question 26

What is meant by differences in prognosis?

Answer 26

In the same way that people diagnosed as schizophrenic rarely share the same symptoms, likewise there is no evidence that they share the same outcomes.

The prognosis for patients diagnosed with schizophrenia varies with about 20% recovering their previous level of functioning, 10% achieving significant and lasting improvement, and about 30% showing some improvement with intermittent relapses.
A diagnosis of schizophrenia, therefore, has little predictive validity - some people never appear to recover from the disorder, but many do.

What does appear to influence outcome, therefore, is more to do with gender (Malmberg, 1998) and psychosocial factors such as social skills, academic achievement and family tolerance of schizophrenic behaviour (Harrison, 2011).