Personality Test Construction

Question 1

What are the three general approaches to personality questionnaire construction?

Answer 1

1) Factor analytic (i.e., inductive)
2) Empirical (i.e., external)
3) Rational (i.e., deductive)

Question 2

What’s the general procedure for a factor analytic approach?

Answer 2

1) Begin with a diverse pool of items (don’t know which items yet you are going to measure)
2) Obtain responses to these items and factor-analyze them
3) Items should load highly on one (and only one) factor then retained for the final questionnaire
4) If the initial items are good and the sample is large, then this strategy can produce some psychometrically sound scales

Question 3

What are some of the major issues related to the factor analytic approach?

Answer 3

• Won’t be able to measure more than a few traits (less than 10) because FA won’t give you a large number of meaningful dimensions
• It’s important to make your initial item pool fairly comprehensive of the domain of your interest. If not, you might get a biased set of personality scales

Question 4

What’s a useful strategy when employing the factor analytic approach?

Answer 4

• Can use FA to find the main dimensions of interest, and use some other strategy to measure these dimensions
• This is how HEXACO was developed

Question 5

What’s the general procedure when using the empirical approach?

Answer 5

1) Begin with a large and diverse pool of items
2) Administer them to a large sample of people who are known to differ on some criterion
3) Then select the items that discriminate best between groups (or show highest r, if criterion is continuous)

Question 6

What are some of the major issues regarding the empirical approach?

Answer 6

• Items sometimes lack face validity (could be a strength of weakness) and the items can sometimes be too subtle
• Generalizability of the validity of the scale can be an issue
• Some items might have shown high r simply because of luck (especially when N is small)
• The selected items not be so valid in a different sample
• The selected items might not be so valid regarding a different trait-relevant criterion

Question 7

What is the general procedure for the rational approach?

Answer 7

1) You write items specifically to measure the traits of interest. In writing these items, you try to maximize content validity
2) Administer these items to a large group of respondents and select items that show high item-total correlations. May also want to include reverse-keyed items
Assumption: Content-valid items will have high and generalized criterion validity

Question 8

What are some of the major issues regarding the rational approach?

Answer 8

• The items are transparent (very specific to trait, may make it easier to fake tests)

Question 9

What’s a major benefit of the rational approach?

Answer 9

• Can be less time-consuming and efficient

Question 10

What does the term item subtlety refer to?

Answer 10

• The degree to which respondents are unaware of what specific traits the items are measuring

Question 11

What has item validity been correlated with?

Answer 11

• It has been positively correlated with the face validity index and negatively correlated with item subtlety

Question 12

Out of all three test construction approaches, which is considered the better one?

Answer 12

• The rational approach has been considered the better one due to its efficiency and the fact that its high face validity is correlated to item validity

Question 13

What are things to keep in mind when writing personality questionnaire items?

Answer 13

• Relevant content - follow trait definition
• Comprehensive content - reflect different aspects of a trait
• Clarity of meaning - should only mean one thing
• Simplicity of language
• Cultural generality
• Sex/age generality
• Balanced scale - to avoid acquiescence bias, meaning reverse-keyed items should be present
• Try not to make items too socially (un)desirable
• Mean response (p-value) - should want to try to get a binomial distribution

Question 14

What should be kept in mind when selecting items to form a scale?

Answer 14

• Avoid extreme means as these are associated with a smaller SD
• Variance of each item (or p-value if item is binary) should be maximized as this increases reliability
• Want high item-total correlations
• Other considerations: Correlation with other scales, content breadth, and balanced scales