Chapter 16: Faking Flashcards
Response Sets
define it
Cronbach (1946) “defined a response set as any tendency that might cause a person to consistently give different responses to test items than he/she would have given if the same content was presented in a
different form.”
What we don’t know is what portion of the variance is “error” and what portion is reliable variance for “stylistic” personality differences
They require us to assess them independently
Response Sets
1. Tendency to gamble
could include guessing, not answering, to select a neutral response alternative, to be cautious
* Can increase reliability because have greater individual
differences
* Reduces validity
Response Sets
2. Definition of judgment categories
How do subjects define response categories?
Most tests require the subject to respond using given response categories, such as the Likert response scale. But different subjects give different meanings to the response options, e.g., the mean- ing assigned to such response categories as “frequently”: Does that mean every day? Six times a day? Once a week?
Response Sets
3. Inclusiveness
When the subject can make as many responses as he or she likes, some individuals make more responses than others. This occurs not only on essay exams, where one per- son’s answer may be substantially longer, but also on tests such as the Rorschach, where one per- son may see many more percepts on an inkblot, or the Adjective Check List, where one person may endorse substantially more items as self- descriptive.
Response Sets
4. Bias or acquiescence
person is more likely to endorse “true” or “yes” responses to dichotomous items
* False items are more valid
* True items are less valid
Response Sets
5. Speed versus accuracy
Where speed of response is an important element, the respondent can answer carefully, sacrificing speed, or can answer rapidly, sacrificing accuracy.
Response Sets
6. Response sets on essay tests
there are multiple response sets because it depends on characteristics of the person (e.g., how organized, fluent, detail-oriented)
Response Sets
how to deal with biases
They are seen as potential threats to reliability and validity.
The following are some of the ways that have been suggested to deal with these biases:
1. Have one or more measures of response bias incorporated in the self-report measure (e.g., MMPI, CPI)
2. Compare (typically correlate) the results of a self-report measure with a measure of bias
3. Determine how susceptible a scale is to faking; ask people to complete using different directions (e.g., fake good, fake bad, standard) and see how the responses differ
Faking
define
“Deliberate systematic distortion of the responses given to test items because the respondent wishes to create a particular impression”
May be made up of two components:
* Emphasis on socially desirable characteristics
* Denial of negative characteristics
Also referred to as “impression management” and “response bias”
Faking
Rogers (1984) described patterns of responding:
- Honest responding – sincere attempt to be accurate
- Irrelevant responding – response not relevant to item content (e.g., answer randomly)
- Defensiveness – conscious denial or minimization
- Malingering – conscious fabrication or exaggeration (e.g., to obtain an external incentive such as monetary compensation, avoiding punishment for criminal behavior, or avoiding military duty)
Incidence of Faking
It is difficult to determine how frequently faking occurs because most of the time, we don’t find out about it
There is disagreement on how frequently it does occur
Seems to depend on the sample and the circumstances of the test administration
Research has shown that people can fake when they are instructed to do so
Legal Issues
How often does malingering occur in criminal populations?
What is the criteria?
the malingering of psychosis is of special concern as a defendant may be found legally insane or incompetent to stand trial based on such a diagnosis
Another issue is that, traditionally, clinicians have worked for the client; if testing is to be administered it is for the joint effort of clinician and client to help the client. However, in forensic situations, the clinician often assumes a neutral role that may be perceived as adversarial by the defendant – i.e., the client is being tested to deter- mine insanity, not necessarily because the client is to be helped, but because such an evaluation is mandated by the court or the legal proceedings.
Lack of Insight
Three issues of concern –
1. Motivation to distort the results in a particular way, such as faking mental illness or attempting to look more positive than one really is
2. Random responding in a conscious effort to sabotage the testing situation
3. Inaccurate reporting of one’s abilities, beliefs, etc. through lack of insight
Various issues
- Content vs Style – what a person says or does versus how a person acts. We can distinguish, at least theoretically, between what a person says or does (content) and how a person acts (style)
- Set vs Style – responding in a way to create an image versus a tendency to select a response category
distinguished between “set,” which refers to a conscious or unconscious desire to respond in such a way as to create a certain image (e.g., to fake good) vs. “style,” which refers to a tendency to select some response category a disproportionate amount of the time, independent of the item content - Is response set error? Is this a meaningful dimension to assess
Some writers think that response sets in fact represent meaningful dimensions of behavior to be assessed (e.g., Cronbach, 1946; 1950), while others think that response sets need to be corrected for or eliminated from test - Faking is more than faking – can provide information that is useful in itself
In many instances, scales to assess faking also can yield valuable information in their own right. For example, on the CPI the three validity scales can also be useful in interpreting the individual’s personality structure and dynamics. As an example, in high-school males (but not females) random answering on the CPI is related to a lower probability of going on to college, a lower GPA, and a greater likelihood of being perceived as delinquent
Faking good and faking bad
Faking good
* Composed of two components: “self-deceptive enhancement” and “impression management”
* More difficult to detect
* More about virtue and honesty
* Want to show have good adjustment and mental health
Faking bad
* Over-endorse symptoms
* Endorse specific symptoms that they think make them look mentally ill
* Appear poorly adjusted or mentally ill
Ways to distort personality tests
- Deliberate faking
- Idealized presentation of oneself as opposed to realistic presentation
- Inaccurate presentation because of lack of insight
Some psychometric issues
Scale-development strategies
1. One group – one instruction
- Calculate the endorsement rate for an item when given to a normal sample with standard instructions
- Find items with low endorsements
- High scorers on those items are making unusual claims
Some psychometric issues
Scale-development strategies
2. One group – two instructions
- A sample takes a measure using standard instructions and then again using faking instructions (e.g., generic or specific)
- Items are retained if they show a significant response shift
Some psychometric issues
Scale-development strategies
3. Two group – two instructions
- Psychiatric patients take the measure using standard
instructions - A normal sample takes the measure and is instructed to fake their answers as if they were a psychiatric patient
- Find items with differential endorsement
Some psychometric issues
Faking psychopathology (type I and II)
- Type I items
Have predictive validity, but low or no face validity
May be endorsed by the psychiatric group, but not by
malingerers
Example: “subtle” MMPI items - Type 2 items
Have face validity, but no predictive validity
Endorsed by malingerers, but not by patients
Example: “visual hallucinations”
Some psychometric issues
Suppressor Variable
There are different techniques and methods which have been used to detect faking, but none seem to change the validity
Research can develop generic or scales which are specific to the content area
Scales used as a correction
Suppressor variable – removes the variance that is assumed to be irrelevant between a predictor and a criterion
Usually the suppressor is associated with the predictor, and not with the criterion
Research has found that it may not help
Some psychometric issues
Stylistic scales
Oblong trapezoids – items assess beliefs about things that can’t be true
How useful are stylistic scales, that is, personality scales that attempt to mea- sure one’s personality style, such as impulsivity?
Stylistic scales
* Research using the CPI didn’t differ in overall validity
* Those scales were the least valid and were not useful predictors
* Using as corrections didn’t increase validity
* Might be used as a moderator, but didn’t work for the CPI
Techniques to Discourage Faking
Intentional distortion
1. Instructions or warnings that distortion can be detected and/or punishment will follow.
2. Use of forced-choice items that are equated on social desirability.
3. The use of subtle vs. obvious items.
4. Use of validity scales.
Disguised titles – simply change the title of the measure to something generic
Filler items – add extra items to the measure to disguise the purpose (not scored)
Forced-choice format
* Lowers reliability – if it lowers too much, then validity will suffer
* May increase construct validity
Developing faking scales
* Approaches
Empirical – differential endorsement
Rational – items selected based on content
Cross-cultural perspective
* Some cultures may be more likely to endorse extreme responses or more positive responses (e.g., Hispanics)
Symptom validity testing
* Present repeated two-alternative, forced-choice, discrimination problems
* If a person has no knowledge, they should answer at a chance level
* If a person tries to answer incorrectly, then their score would be below chance
Lin, Y. (2021). Using reliabilities reliably
Using reliabilities reliably – A deeper look at the reliability of forced-choice assessment scores.
The main findings
The study confirmed that the different estimation methods do not converge in values, even when they are applied to the exact same assessment. Therefore, there
are three things that researchers and practitioners need to keep in mind when working with forced-choice reliabilities:
1.When reporting reliabilities, it is essential to specify the estimation method used.
2.When interpreting reliability estimates, it is important to consider the assumptions and limitations of the estimation method used.
3.When comparing the reliability of scores from different forced-choice assessments, the reliability estimation method should be kept constant.
