Reliability and Validity in Diagnosis Flashcards
what is reliability and validity in terms of diagnosis and classification of schizophrenia?
6
classification systems such as DSM-V are worthless unless they are reliable and valid
reliability = the consistency of a classification system (such as DSM) or a measuring instrument to assess particular symptoms of schizophrenia
we would expect any measurement to produce the same data if taken on successive occasions
validity = the extent that a diagnosis represents something that is real and distinct from other disorders and the extent that a classification system measures what it claims to measure
essentially refers to whether an observed effect is a genuine one
reliability and validity are inextricably linked because a diagnosis cannot be valid if it is not reliable and reliability alone counts for nothing unless the systems and scales are also valid
reliability
6
diagnostic reliability means that a diagnosis of schizophrenia must be repeatable
clinicians must be able to reach the same conclusion at two different points in time (test retest reliability) OR different clinicians must reach the same conclusion (inter rater reliability)
inter rated reliability is measured by a statistic called a kappa score
a score of 1 indicates perfect inter rater agreement while a score of 0 indicates zero agreement
a kappa score of 0.7 or above is generally considered good
in the DSM-V field trials, the diagnosis of schizophrenia had a kappa score of only 0.46
reliability: cultural differences in diagnosis
9
research suggests that there is a significant variation between countries when it comes to diagnosing schizophrenia
culture (the rules, customs, morals and child rearing practices that bind a group of people together and define how they are likely to behave) seems to influence the diagnostic process
Copeland (1971) gave 134 US and 194 British psychiatrists a description of a patient
69% of the US psychiatrists diagnosed the patient with schizophrenia but only 2% of the British psychiatrists did
furthermore, one of the main characteristics of schizophrenia is heating voices, which also appears to be influenced by the cultural environment
Luhrmann et al (2015) interviewed 60 adults diagnosed with schizophrenia (20 from Ghana, 20 from India and 20 from the US)
each was asked about the voices they heard and it was found that many of the African and Indian subjects reported positive experiences with their voices, describing them as playful or as offering advice
but not one American did — instead, the US subjects were more likely to report the voices they heard as violent, hateful and indicative of being sick
Luhrmann suggests that the harsh, violent voices so common in the West may not be an inevitable feature of schizophrenia globally, as in many cultures hearing voices is actually a positive experience
validity
1
includes…
• gender bias in diagnosis
• symptom overlap
• co-morbidity
validity: gender bias in diagnosis
6
gender bias refers to the tendency to describe the behaviour of men and women in a way that might not accurately represent the characteristics of either one of these genders
gender bias in the diagnosis of schizophrenia is said to occur when accuracy of diagnosis is dependent on the gender of an individual
the accuracy of diagnostic judgements can vary for a number of reasons — including gender biased diagnostic criteria or clinicians basing their judgements on stereotypical beliefs held about gender
for example, critics of the DSM diagnostic criteria argue that some diagnostic categories are biased towards pathologising one gender rather than the other
Broverman et al (1970) found that clinicians in the US equated mentally healthy adult behaviour with mentally healthy male behaviour, as a result there was a tendency for women to be perceived as less mentally healthy
parents tend to be less tolerant of schizophrenic sons than they are of schizophrenic daughters, which may account for their earlier diagnosis
validity: symptom overlap
5
symptom overlap refers to the fact that symptoms of a disorder may not be unique to that disorder but may also be found in other disorders, making accurate diagnosis difficult as patients may fulfil the criteria for multiple disorders
despite the claim that the classification of positive and negative symptoms would make for more valid diagnoses of schizophrenia, many of these symptoms are also found in other disorders such as depression and bipolar disorder
this problem is referred to as symptom overlap
Ellason and Ross (1995) point out that people with disassociative identity disorder (DID) actually have more schizophrenic symptoms than people diagnosed as being schizophrenic
most people who are diagnosed with schizophrenia have sufficient symptoms of other disorders that they could also receive at least one other diagnosis
validity: co-morbidity
8
co-morbidity is an important issue for the validity of the diagnosis of mental illness
it refers to the extent that two or more conditions or diseases occur simultaneously in a patient — for example, having both schizophrenia and depression
psychiatric co-morbidities are common among patients with schizophrenia — these include substance abuse, anxiety and symptoms of depression
Buckley et al (2009) estimate that co-morbid depression occurs in 50% of schizophrenic patients and 47% of patients also have a lifetime diagnosis of co-morbid substance abuse
schizophrenia and OCD are two distinct psychiatric conditions — roughly 1% of the population develop schizophrenia and roughly 2-3% develop OCD
since both are fairly uncommon, we would expect that only a few people with schizophrenia would develop OCD and vice versa
however, evidence suggests that the two conditions appear together more often than chance would suggest
a meta analysis by Swets at al (2014) found that at least 12% of patients with schizophrenia also fulfilled the diagnostic criteria for OCD and about 25% displayed significant obsessive-compulsive symptoms
x3 evaluation points for validity
research support for gender bias in the diagnosis of schizophrenia
research support for co-morbidity
a diagnosis of schizophrenia has little predictive validity
VALIDITY EVALUATION
research support for gender bias in the diagnosis of schizophrenia
5
Loring and Powell (1988) randomly selected 290 male and female psychiatrists to read to case descriptions of patients’ behaviour
the psychiatrists were then asked to offer their judgement on these individuals using standard diagnostic criteria
when the patients were described as males or no information was given about gender, 56% of the psychiatrists diagnosed them with schizophrenia
however, when the patients were described as female, only 20% were given a diagnosis of schizophrenia
this gender bias was not as evident among the female psychiatrists which suggests that diagnosis is influenced not only by the gender of the patient also by the gender of the clinician giving the diagnosis
VALIDITY EVALUATION
research support for co-morbidity
6
a number of studies have examined single co-morbidities with schizophrenia, but these studies have usually involved only relatively small sample sizes
by contrast, a US study by Weber et al (2009) looked at nearly 6 million hospital discharge records to calculate co-morbidity rates
psychiatric and behaviour related diagnoses accounted for 45% of comorbidity
however, they also found evidence of many co-morbid non-psychiatric diagnoses
many patients with a primary diagnosis of schizophrenia were also diagnosed with medical problems including hypothyroidism, asthma, hypertension and type two diabetes
they concluded that the very nature of a diagnosis of a psychiatric disorder is that patients tend to receive a lower standard of medical care, which in turn adversely affects the prognosis for patients with schizophrenia
VALIDITY EVALUATION
a diagnosis of schizophrenia has little predictive validity
4
this is because in the same way that people diagnosed with schizophrenic rarely share the same symptoms, there is no evidence that they share the same outcomes
the prognosis for patients diagnosed with schizophrenia varies with about 20% recovering their previous level of functioning, 10% achieving significant and lasting improvement and about 30% showing some improvement with intermittent relapses
therefore, a diagnosis of schizophrenia has little predictive validity with some people never appearing to recover from the disorder, while others do
what does appear to influence outcome is more to do with gender and psychosocial factors such as social skills, academic achievement and family tolerance of schizophrenic behaviour
x3 evaluation points for reliability
lack of inter rater reliability
unreliable symptoms
cultural differences in the diagnosis of schizophrenia
RELIABILITY EVALUATION
lack of inter rater reliability
7
despite the claims for increased reliability in DSM-lll and later revisions, over 30 years later there is little evidence that DSM is routinely used with high reliability by mental health clinicians
Whaley (2001) found inter rater reliability correlations in the diagnosis of schizophrenia as low as 0.11
further problems with the inter-rater reliability of the diagnosis of schizophrenia are illustrated in the Rosenhan study
Rosehan’s famous study highlighted the unreliability of diagnosis
in the study, normal people presented themselves to psychiatric hospitals in the US claiming they heard an unfamiliar voice in their head saying the words empty, hollow and thud
they were all diagnosed as having schizophrenia and admitted to hospital
throughout their stay, none of the staff recognised that they were actually normal
RELIABILITY EVALUATION
unreliable symptoms
4
for a diagnosis of schizophrenia, only one of the characteristic symptoms is required if delusions are considered bizarre
however, this creates problems for reliability of diagnosis
in one study, 150 senior psychiatrists in the US were asked to differentiate between bizarre and non bizarre delusions
they produced inter rater reliability correlations of only around 0.49, forcing the researchers to conclude that even this central diagnostic requirement lacks sufficient reliability for it to be a reliable method of distinguishing between schizophrenic and non-schizophrenic patients
RELIABILITY EVALUATION
cultural differences in the diagnosis of schizophrenia
7
Barnes (2004) has established cultural and particularly racial differences in the diagnosis of schizophrenia
the prognosis for members of ethnic minority groups may actually be more positive than for majority group members
the ethnic culture hypothesis predicts that ethnic minority groups experience less distress associated with mental disorders because of the protective characteristics and social structures that exist in most ethic minority cultures
Brekke and Barrio (1997) found evidence to support this hypothesis in a study of 184 individuals diagnosed with schizophrenia or a schizophrenia spectrum disorder
this sample was drawn from two non-white minority groups (African Americans and Latinos) and a majority group (white Americans)
they found that non-minority group members were consistently more symptomatic of schizophrenia than members of the two ethnic minority groups
this supports the ethnic culture hypothesis and the idea that there are cultural differences in the diagnosis of schizophrenia
