lecture 5- convergent and discriminant validity Flashcards
convergent validity
- If a measure has convergent validity then it correlates with (i.e., varies with, is associated with) other measures of the same construct or with related constructs
- So, for example, we would want a self-report depression scale to correlate
highly with other self-report depression scales - We would also want it to correlate with ratings of depression obtained by other methods
- For example, we would want it to correlate highly with clinicians’ ratings
of depression - So, if we gave the scale to a sample of patients and also had clinicians
interview them, we could look at the relationship between scores from the
two methods…
convergent validity cont.
- Most psychological tests have decent convergent validity (they wouldn’t
be used otherwise) - For example, most verbal memory scales correlate highly with each other
- Similarly, most depression scales correlate highly with each other, and with clinician ratings of depression
- However, high convergent validity is not always found
- Sometimes construct variance (the thing we want to measure) is swamped by method variance (the way we have measured it)
- A dramatic example is provided by Cole’s (1987) analysis of patient and child
ratings of the child’s level of depression…
cole, 1987
- Can be seen that there is good agreement across different scales when the children rated their own level of depression
- There is also good agreement across the same scales when the mothers rated their childs’ level of depression
- However, there is very poor agreement between the two different methods of assessment (the correlation is only 0.15)
what is discriminant validity?
- Discriminant validity is the flip side of convergent validity: if a measure has good discriminant validity then it does not correlate with (is not associated with) measures it should be unrelated to, or weakly related to
- Take the example of a test / exam of (say) geographical knowledge
- Suppose the test correlates highly with writing speed: then it has poor discriminant validity
- The test is supposed to be measuring geographical knowledge, not the ability to write fast
- To be clear: a high correlation with an unrelated measure indicates low
or poor discriminant validity
discriminant validity cont.2
- As another example, Depression scales correlate very highly (-0.9) with measures of social desirability (Langevin & Stanger, 1979)
- Social desirability scales are supposed to measure the extent to which people attempt to present themselves to others in an overly positive light
- Example item: “There have been occasions when I took advantage of someone”
- Depression scales tend to correlate highly with social desirability (high depression is associated with low social desirability scores – i.e., they are negatively correlated)
discriminant validity cont.3
- Therefore there is a problem with the discriminant validity of these scales
- But what is the locus of the problem? Is the problem with depression scales, or with social desirability scales?
- The problem is probably largely with the social desirability scales
- Social desirability scales are supposed to measure the extent to which people portray themselves to others in an overly positive light
- But it appears they may also be measuring our ability to portray ourselves to
ourselves in an overly positive light
discriminant validity contd.4
- In other words the non-depressed wear rose-tinted spectacles – they protect their self-esteem / mood by seeing themselves in an overly positive light
- In contrast, depressed people do not (depressive realism) and indeed devalue themselves
- Thus, social desirability scales appear to have poor discriminant validity
Can Self-Report Scales Differentiate Between Anxiety and Depression?
- Anxiety and depression scales generally have good convergent validity (i.e., anxiety scales correlate highly with each other, and with clinician’s ratings; same goes for depression scales)
- (The earlier example of mother and child ratings is an exception to this rule)
However, how well do they perform in terms of discriminant validity?
-Not well at all!
Can Self-Report Scales Differentiate Between Anxiety and Depression? cont2
- It is not at all uncommon for an anxiety scale to correlate more highly with a
depression scale than it does with another anxiety scale (same goes for depression
scales) - Formal way of saying that: between-construct correlations often exceed within-
construct correlations - Informal way: You think you are measuring anxiety but in fact looks like you may be
measuring depression and vice-versa
Can Self-Report Scales Differentiate Between Anxiety and Depression? cont 3
Why should that happen?
- Note: it is not just self-report scales that have this problem: also happens with
clinician’s ratings
Is the problem with the scales or with the constructs?
- There certainly is a good deal of genuine overlap between anxiety and depression so
we would expect a correlation between scales
–However, it should still be possible to differentiate them
Clark & Watson (1991,1995 etc) in a series of influential papers introduced tripartite
theory
This theory proposes that anxiety and depression share a common feature: negative
affectivity
what does negative affectivity (NA) refer to?
Negative affectivity (NA) refers to unpleasant engagement i.e., feeling “distressed”,
“scared”, “upset”, “irritable”
what do Clark and Watson argue?
- Clark & Watson argue that most anxiety and depression scales predominantly measure negative affectivity, rather than anything specific to anxiety and depression
- It is argued this is why anxiety and depression scales have poor discriminant validity
- Tripartite theory proposes that there are specific components to anxiety and
depression that allow them to be differentiated…
low positive affect
- The proposed specific component for depression is low positive affect : that is the
absence of feeling such as “enthusiasm”, “interest”, “alertness” - Low positive affect is very similar to the psychiatric concept of anhedonia (“loss-of-
pleasure”) - It is suggested that someone who is suffering from anxiety does not necessarily have low positive affect: they can still enjoy some things
- In contrast, the severely depressed person gets no pleasure (and has little or no
interest) in anything - The proposed specific component for anxiety is physiological hyperarousal
according to tripartite theory
- both anxious and depressed individuals will be high on negative affectivity
- Depressed people will (in addition to high NA) have low positive affect
- Anxious individuals will (in addition to high NA) be high on physiological hyperarousal
Note: “anxious arousal” is used as a synonym for physiological hyperarousal
the MASQ
- Tripartite theory suggests that, if self-report depression scales targeted (low) positive affect, and anxiety scales targeted anxious arousal, we would get round the problem of very poor discriminant validity
- Clark & Watson have come up with a candidate scale: the Mood and Anxiety
Symptom Questionnaire (MASQ) - Has an Anhedonic Depression scale: this should measure the specific component
of depression
Also has an Anxious Arousal scale: this should measure the specific component of
anxiety
-
How should we evaluate whether the MASQ is preferable to existing scales (such as
HADS, Beck, etc)?
Would it be enough just to show that the two subscales had a low correlation? (i.e.,
that they had good discriminant validity)
- No: two scales that are simply very unreliable will have a low correlation
- No: the anxiety and depression scales may have poor convergent validity (e.g., the
MASQ Anxiety scale may show little relationship with other measures of anxiety)
-Data collected here from a large (N =1800+) sample of the general adult population
(Crawford et al, in prep; De Witt, 2011)
evaluation of the MASQ
The foregoing slide shows (in order of entry):
That the MASQ is at least as reliable as existing scales
- The MASQ anxiety and depression scales have decent convergent validity
- The conventional scales demonstrate the usual discriminant validity problem
For example, the HADS Anxiety scale correlates more highly with HADS
Depression than it does with MASQ Anxiety
evaluation of the MASQ cont
- In contrast, the MASQ scales have very good discriminant validity
- Conclusion: The MASQ has better psychometric properties than
existing scales - That is, it appears that the MASQ does target the specific components of anxiety and depression
- That is, it gets round the poor discriminant validity seen with existing
scales
conclusion
➢Psychometric scales are a valuable tool for
assessing and diagnosing psychological constructs
➢We have to be confident that they are both
reliable and valid tools – otherwise why use them?
➢Reliability is the consistency of a measure and can be evaluated using Cronbach’s alpha and test- retest methods
➢Validity is whether a test measures what it’s
supposed to measure and can be established using a range of methods: Face, Content, Ecological, Construct, Convergent and Discriminant
➢Practitioners need a good understanding of the reliability and validity of tests
