Reliability and Validity of Diagnosis Flashcards
Reliability
consistency over time and between individuals
SCHIZ: Consistency and of a measuring tool (eg. The DSM) or other tests used in diagnosis. A test/tool must be valid for it to be reliable.
Validity
correctness
- The diagnostic system is assessing what it claims to be assessing
- To have validity it must have high reliability
The extent to which a diagnosis is accurate and meaningful.
This means it must represent something real and be distinct from other disorders.
The DSM must enable a valid diagnosis to be made, by ensuring it
measures what it claims to measure (symptoms of a disorder,
schizophrenia) which is different to other mental health conditions and enables psychiatrists to recognise symptoms & provide treatment.
Diagnostic systems
if the assumptions are sound (valid) then it
should be easy to diagnose a mental disorder
Test-retest reliability?
Doctors must be able to reach the same conclusions about a patient at two different points in time. The DSM and other tests must enable this to occur, so a patient isn’t ‘labelled’ with a diagnosis which then changes or is re-diagnosed as another condition later on.
Inter-rater reliability?
The extent to which different assessors agree on their assessments.
Doctors must reach the same conclusions about a patient’s diagnosis. The DSM and other tools must enable inter-rater reliability. Eg. Doctors should be able to consult the DSM about a patient, independently from each other and achieve the same diagnosis.
Cheniaux et al (2009)
Cheniaux et al (2009) had two psychiatrists independently
diagnose 100 patients using the DSM criteria and the ICD
criteria. One doctor diagnosed 26 patients with Schizophrenia
according to the DSM and 44 according to the ICD. The other
doctor diagnosed 13 according to DSM and 24 according to
ICD
Cultural Differences in Diagnosis
Research suggests there is a significant variation between countries when it comes to diagnosing schizophrenia ie culture has an influence on the diagnostic process. Copeland (1971) gave 134 US and 194 British psychiatrists a description of a patient. 69% of the US psychiatrists diagnosed schizophrenia but only 2% of the British ones did.
Luhrmann et al. (2015) showed the differences in ‘hearing voices’ across cultures. He interviewed 60 people with schizophrenia who heard voices; 20 each from Ghana, India and the US. Many of those from Africa and India reported positive experiences of their voices ie playfulness, however none of the Americans did (most Americans reported the voices as hateful and/or violent).
Gender Bias
Occurs when accuracy of a diagnosis is dependent on the gender of an individual.
This may occur due to gender-biased diagnostic criteria, or clinicians basing their judgements on stereotypical beliefs held about gender.
Broverman et al. (1970) found the clinicians in the US equated mentally healthy ‘adult’ behaviour with mentally healthy ‘male’ behaviour therefore women tended to be seen as less mentally healthy.
Symptom Overlap
Many of the symptoms of schizophrenia are also found in other
disorders such as depression and bipolar disorder.
Read (2004) states that most people with schizophrenia have
sufficient symptoms of other disorders that they could also
receive at least one other diagnosis.
Co-morbidity
Refers to the extent that two or more conditions co-occur.
Schizophrenia often occurs alongside substance abuse, anxiety and depression.
If conditions occur together a lot of the time, then this calls into question the validity of their diagnosis and classification (may actually be a single condition)
Buckley et al. (2009) estimated that co-morbid depression occurs in 50% of patients, and 47% also have a lifetime diagnosis of co-morbid substance abuse.
Swets et al. (2014) carried out a meta-analysis and found that at least 12% of patients with schizophrenia also fulfilled the diagnostic criteria for OCD and about 25% displayed significant OCD obsessive-compulsive symptoms.
ROSENHAN (1973) ON BEING SANE IN INSANE PLACES - Study 1
This was a covert participant observation with eight participants consisting of five men and three women (including Rosenhan himself). Their task was to follow the same instructions and present themselves in 12 psychiatric hospitals in the USA claim to hear a single voice saying ‘thud’
All participants were admitted to various psychiatric wards and all but one were diagnosed with schizophrenia. The last one was diagnosed with manic depression (now Bipolar depression).
All pseudo-patients behaved normally while they were hospitalised because they were told that they would only get out if the staff perceived them to be well enough.
The pseudo-patients took notes when they were hospitalised but this was interpreted as a symptom of their illness by the staff. It took between 7 to 52 days before the participants were released. They came out with a diagnosis of Schizophrenia in remission.
ROSENHAN (1973) ON BEING SANE IN INSANE PLACES - Study 2
A follow-up study was done later where the staff at a specific psychiatric hospital were told that impostors would
present themselves at the hospital and that they should try to rate each patient whether he or she was an impostor. Of the 193 patients, 41 were clearly identified as impostors by at least one member of the staff, 23 were suspected to be impostors by one psychiatrist, and 19 were suspected by one psychiatrist and one staff member. There were no impostors.
Evaluation
This controversial study was conducted nearly 40 years ago but it had an enormous impact in psychiatry. The development of diagnostic manuals has increased reliability and validity of diagnosis although the diagnostic tools are not without flaws. Yet whether the study has much to suggest about current reliability and validity of a diagnosis must be considered.
The method used raises ethical issues (the staff were not told about the research) but it was justified since the results provided evidence of problems in diagnosis which could benefit others. There were serious ethical issues in the follow-up study since the staff thought that impostors would present, but they were real patients and may not have had the treatment that they needed.
Issues in Diagnosis Summary
Reliability
Validity
Co-morbidity
Symptom overlap
A03
Inter-rater reliability is poor in diagnosis
Validity is poor in diagnosis (over diagnosed in ICD, under in DSM)
High co-morbidity with severe depression – are they one disorder?
Gender bias in diagnosis
Cultural bias in diagnosis
