Scale Development Flashcards
Psychological constructs
- Psychologists are interested in relationships between psychological experiences
- In previous research methods modules, you have learned how to model these kinds of relationships e.g., general linear models
- But we cannot test the relationship between A and B directly. Instead, we look at the relationship between measures of A and B, and assume this corresponds to the relationship between constructs A and B.
- Measures are proxies for constructs that we cannot directly assess.
Psychological constructs and measures
- for our claims to be true, measures need to correspond to their constructs
Constructs - concrete to abstract
- Psychological constructs tend to be more abstract.
- This makes it trickier to create a clear correspondence between a construct and its measurement.
- Concrete constructs:
- Connected to relatively unambiguous, observable events.
- Usually observed directly, using physical instruments.
- Examples - age, height, weight
- Abstract constructs:
- Observed indirectly through self-report measures, by observing behaviour, or by using physiological measures, brain imaging, etc.
- Examples - memory, attention, anxiety, identity, happiness
Psychological constructs = abstract
· “[…] psychological constructs like depression, personality traits, attitudes, or cognitive abilities often do not permit direct observation. We can directly observe the height of a person next to us on the bus, but we often have limited insight into their psychological processes, states, and attributes.”
- (Flake & Fried, 2020, p. 3)
Latent and observed variables
· Constructs are called latent variables (i.e., they are not directly observed)
· Measures intended to capture the latent variables are called observed variables.
· When constructs correspond to measures, the latent variable is assumed to cause the scale scores.
· In other words, it is assumed a person answers the scale items the way they do because of an underlying psychological quality.
· Examples - participants will highly agree with items on the Rosenburg self-esteem scale because they have high self-esteem.
Psychological measures
· Psychological constructs are measures in many ways:
1. Behavioural observation
2. Cognitive tasks
3. Psychological measures
4. Brain scans
5. Self-report scales
Disadvantages of self-report scales
- Dependent on participants memory
- Responses can be affected by primacy/recency effects
2. Influenced by self-presentation concerns - People are motivated to present themselves in a positive light to others (and to themselves)
3. Influenced by participants response styles
- For instance, some participants may agree with scale items regardless of their content; they are said to have an acquiescent response style
- Responses can be affected by primacy/recency effects
Advantages of self-report scales
- Information richness
- No one has access to as much information about the participant as the participant themselves
2. Motivation to report - Participants are more fascinated with themselves than anyone else; people are usually pleased to talk about themselves (vs others) and may put in more time and effort
3. Ease of administration - Inexpensive, quick, and easy to set up, code, and analyse
- No one has access to as much information about the participant as the participant themselves
Scale development
- phases of investigating a new construct using scales.
1. substantive phase - construct conceptualisation and literature review
2. structural phase - item analysis
- determining dimensionality
- reliability
3. external phase - convergent and discriminant validity
Substantive phase of scale development
- Here, you make key decisions about your target construct and what your scale will look like (e.g., what kinds of items you will use, what response scales)
- Main steps within the substantive phase:
a. Construct conceptualisation and literature review - key activities: defining the boundaries of your construct
b. Generating items - key activities: making measurement format decisions, brainstorming items, reviewing items
- Main steps within the substantive phase:
Construct conceptualisation and literature review
· You can guide your thinking and reading using some key questions.
· General questions:
- What experiences are about the construct?
- What experiences are outside the construct?—How is the construct different from related constructs?
· Some theoretical considerations:
- Type of psychological experience
- Construct specificity
construct specificity
· At what level of specificity do you want to measure your construct?
· Contrast global meaning in life to meaningful work:
- Global meaning in life: “My life as a whole has meaning”
- Meaningful work: “I have found a meaningful career.”
· These are separate constructs—e.g., someone might feel that they have a meaningful life, while not having a meaningful career
· Specificity-matching principle: variables that have matched levels of specificity are more strongly related (see Swann et al., 2007)
- All other things being equal, constructs about “life as a whole” will be more strongly related to other constructs about “life as a whole” than to constructs that are more narrowly focused (e.g., about the workplace)
Generating items
· Key activities: Making measurement format decisions, brainstorming items, reviewing items (quality-control)
· In practice, the three activities will often overlap: generating items is an iterative process–as part of reviewing your items, you might find the need to generate new items or that you need to tweak the measurement format
Measurement format decisions 1 - direct vs indirect items
· Direct: Participants are directly invited to reflect on the construct of interest. The name of the construct is used in the item wording
- Example: “I have high self-esteem” (Single-Item Self-Esteem scale; Robins et al., 2001)
· Indirect: Participants are invited to reflect on experiences that are about the construct of interest without naming it. The construct of interest is obscured in the item wording
- E.g., A narcissism scale might ask about participants’ leadership (“People see me as a natural leader”) or their attractiveness (“I have been compared to famous people”)… but the scale is not about these things
· It is harder to get participants to precisely reflect on the construct of interest using indirect items, but it may help to avoid confounds (e.g., socially desirable responding)
An interesting case - positive and reverse-worded items
· Consider two items from the Self-Liking Scale (Tafarodi & Swann, 1995)
1. I like myself
2. I tend to devalue myself
· Participants high on self-liking would agree with the first item but disagree with the second.
· Answers that express the reverse of the studied construct (i.e., in this case, self-disliking) are called reverse-worded items.
- Can be used to detect biases in response patterns such as acquiescent response styles (if participants agree with both these items) or careless responding (other inconsistent patterns of responding such as disagreeing with both items)
