Reliability and Validity of Diagnosis Flashcards
what are the key terms of diagnosis?
Diagnostic criteria are a set of signs, symptoms, and tests developed for use in routine clinical care to guide the care of individual patients.The diagnosis typically guides treatment.
Classification criteria are standardised definitions grouping health issues into categories based on similar properties. These categories may be grouped by type of disease, body system or anatomy. Classification criteria have no treatment implications for patients.
what are the pros of diagnosis?
Useful for those suffering to get appropriate therapy once diagnosed and aids recovery.
what are the cons to diagnosis?
STIGMA related to the label ‘SZ’- could make the condition worse & put restrictions on someone’s life
(due to labelling /SFP)
Discrimination - leading to poor health care, education, housing, employment.
Biased/wrong diagnosis will lead to wrong treatment /prognosis.
This is why it is really important to be sure the diagnosis made has both Reliability & Validity
what is validity?
Validity= accuracy of diagnosis eg. is the diagnosis correct
what is reliability?
Reliability= consistency of diagnosis eg. 2 or more practitioners make the same diagnosis of the same person
what is inter-rater reliability and a study related?
Clinicians make identical diagnosis of the SAME patient….
BECK et al 1963 in a review of 153 patients who had been diagnosed by multiple doctors found only 54% concordance rate between the doctors assessments.
This suggests there is LOW inter-rater reliability in the diagnosis of SZ.
This suggests many people have been diagnosed incorrectly, potentially having inappropriate treatment.
Low rates of clinicians agreement could be influenced by:
own background, background of patient, training, interpretation
what is another study related to inter-rater reliability?
Copeland (1970) gave a description of a patient to 134 US and 194 British psychiatrists and found that 69% of the US psychiatrists diagnosed the patient with schizophrenia whereas only 2% of British psychiatrists gave the same diagnosis. This calls into question the reliability of the diagnosis of schizophrenia as American clinicians were far more likely to diagnose SZ than their UK counterparts.
what is test-re-test reliability?
Test- retest reliability (external reliability) - a clinician should show external reliability by making the SAME diagnosis of the patient on separate occasions based on the SAME information.
What are the positives evaluations of reliability?
- However, a positive way to improve reliability for diagnosing schizophrenia has been identified in a tool used in diagnosis.
Farmer (1988) found that standardised interview techniques known as Present state examination ( PSE) increases reliability of diagnosing SCH as it focuses on frequency & severity of symptoms.
This is a mental status test that assesses a patient’s current psychiatric condition. It contains 140 items, each scored on a 3-point or 4-point scale. The PSE can be learned by American-trained clinicians with interrater reliability comparable to that of British and European clinicians
This implies that using the right standardised tools like PSE leads to reassurance for patients that the outcome and treatments they receive are based on diagnosis that would be consistent amongst practitioners. - Another positive is that the DSM has also now improved the reliability of diagnosis through revising the criteria.
The DSM- V has stripped the criteria out that was difficult to differentiate for example deciding between bizarre (an impossible) delusion and non-bizarre (possible) delusions.
For example Mojtabi & Nicholson (1995) found that 50 psychiatrists failed to differentiate between bizarre and non bizarre delusions, so due to reliability being low, patients may receive the wrong diagnosis (i.e bizarre - believing the intelligence agencies have put a microchip inside you to track your movements and non-bizarre being you are being followed by intelligence services and under surveillance).
This supports that notion that the DSM has helped to increase the reliability of diagnosis by adapting and changing certain specific criteria to make categorising more consistent.
what are the negatives evaluations of reliability?
- A research study that highlights problems with inter-rater reliability in the diagnosis of schizophrenia, was conducted by Copeland (1970) and illustrates how the culture of the clinician can damage INTER-RATER reliability.
E: Copeland found that when the same description of a patient was given to British psychiatrists and US Psychiatrists , they found 69% of US psychiatrists diagnosed the patient with SCH compared to just 2% of British Psychiatrists, showing US clinicians are far more likely to diagnose SCH than their UK counterparts.
L: This implies that even with standard criteria inter- rater reliability cannot be guaranteed due to the possible influence of cultural bias that could impact diagnosis. - Another research study by Read (2004) reported concerning data that test- retest analysis is as low as 37% for schizophrenia, especially when we consider the potential false positives and false negatives this could create.
Which means patients without the disorder may be diagnosed as having SZ (false Positive) and patients with the disorder may be diagnosed as not having SZ (false negative).
This means people who need treatment may not get it or get the wrong treatment due to the unreliable diagnosis.
what is comorbidity?? - validity
Comorbidity refers to the occurrence of two illnesses or conditions together.
E.g., a person has both schizophrenia and another disorder.
Schizophrenia and depression = 50%
Schizophrenia and PTSD = 29%
Schizophrenia and OCD = 23%
If conditions occur together this calls into question the validity of their classification as many people have complex problems and multiple problems rather single problems or problems one at a time.
SZ is commonly diagnosed with other conditions - Patients with SZ e.g issues with substance abuse, PTSD or OCD.
Mental health tends to be a complex area not a simple one so VALIDITY of diagnosis can be challenged.
what is symptom overlap?? - validity
Symptoms of SZ are not PATHOGNOMONIC (no symptoms of SZ are exclusive to the SZ alone).
Bipolar disorder, OCD & Autism have overlapping symptoms with SCH e.g social withdrawal is found in bipolar AND SZ
Read et al (2011) estimated 13 % of population hear voices but only 1% are diagnosed with SCH so this questions validity of diagnosis
hared symptoms between disorders could lead to an unreliable / invalid diagnosis - the diagnosis may not be correct.
E.g.,
Delusions - shared by schizophrenia and bipolar
Hallucinations - shared by schizophrenia and PTSD
Attention difficulties - shared by schizophrenia and ADHD
Avolition - shared by schizophrenia and depression
Etc. (there is a very long list of shared symptoms!)
Symptoms of SZ are Heterogeneous, what does that mean?
meaning they can present in very different ways for patients.
This can make accurate diagnosis difficult even with the use of diagnostic manuals such as DSM- V (USA) and ICD-11 (UK).
what are the positive evaluations of validity?
- A positive to improve validity is the use of classification systems such as the DSM where patients have to meet more than one criteria to be diagnosed with SZ.
For example, one criteria for SZ is the characteristic symptoms and another criteria is social/occupational malfunction which have to both be met as part of the diagnosis AND symptoms need to be present for a certain time frame.
This suggests that having diagnostic criteria that covers a variety of categories, helps to make accurate diagnoses for an area of medicine where the symptoms are not always physical.
what are the negative evaluations of validity?
- There is research evidence to support the claims that diagnosis can be lacking in reliability and therefore validity.
Rosenhan (1973) investigated the reliability & validity of staff diagnosis in psychiatric hospitals. He had Pseudo-patients, with no mental illness, pose as mentally ill to see if staff in hospitals could accurately diagnose them.
Rosenhan had 8 ‘sane’ people complain of hearing unclear voices saying ‘thud, hollow, empty’. All 8 were admitted to hospital & all but one was diagnosed with SZ, the other with manic-depressive psychosis. Upon admission, all pseudo-patients stopped showing any ‘symptoms’ and took part in ward activities. The average length of stay was 19 days. On release, the pseudo-patients were given the diagnosis of SZ ‘in remission’. - In a second follow up study one hospital was told that one or more pseudo-patients would try to be admitted & hospital staff were asked to rate the patients on the likelihood of them being a pseudo-patient. 44% were judged by at least one member of staff to be a pseudo patient- in fact NO pseudo patients were ever sent so real patients with mental illness symptoms were turned away!
This implied that psychiatrists could not reliably tell the difference between an insane & sane person, calling into question the reliability of a SZ diagnosis. ‘Normal’ behaviour was misinterpreted as ‘abnormal’ to support their idea that the pseudopatients had a mental illness which suggests the validity of psychiatric diagnoses using the DSM-2 at the time, was flawed. - There is research evidence to support the idea that comorbidity reduces the validity of diagnosis as a clinician could make an inaccurate diagnosis due to the additional disorder.
Buckley et al (2009) found that around 50% of patients with a diagnosis of schizophrenia also have a diagnosis of depression (50%) or substance abuse (47%). Post-traumatic stress occurred in 29% of cases and OCD in 23%, showing that schizophrenia commonly occurs alongside other mental illnesses and the disorders are comorbid.
This implies that it is hard to judge which part of the disorder is just SCH and which belongs to another disorder meaning the wrong focus of treatment may be provided to the patient. - There is research evidence to support the idea that the overlapping symptoms can decrease the validity of diagnosis as a clinician may identify symptoms which are common with other disorders.
Konstantareas and Hewitt (2001) investigated the symptoms of autistic patients and patients of schizophrenia. They found that when comparing 14 autistic patients and 14 with SZ (all were male) they found not all of the SZ patients had autism symptoms but 50% of autistic patients had symptoms of SZ ( negative symptoms).
This implies that.. symptom overlap makes it difficult to come to a valid diagnosis and therefore leads to treatments that are not directly targeting the SZ symptoms.
