Selecting the best Measure Flashcards
What is one of the difficult things about measurement in psychology?
We need operational definitions
What are some observable measures we can choose to look at?
- Verbal response
- Nonverbal response
- Physiological response
- Overt actions
What are the 2 elements that are contained in every measurement?
“True” score-hypothetical concept
Error
“True” score+ error=observed score
What are 3 sources of measurement error?
Experimenter, participant, observer/scorer
What are 2 sources of experimenter error?
Random Error: Time of day, temperature, noise.
Bias: Experimenter characteristics, experimenter expectancies.
What do we mean by experimenter characteristics?
When a particular aspect of the experimenter affects how participants respond-can be physical characteristics (age, gender, ethnicity) OR personality (friendliness, hostility, anxiety).
How do we control for experimenter characteristics?
Use standardized methods- train experimenters to follow set standards when administering procedures, standardize aspects of experimenter as much as possible (appearance, attitude, etc)
Replication!
What do we mean by experimenter expectancies?
When the expectations of the experimenter affect how the participant behaves. Not limited to humans!
What are 2 examples of experimenter expectancies?
- The Rosenthal effect (educational “Bloomers”
- Maze bright versus maze dull rats.
How do we control for experimenter expectancies?
- Standardization (instructions scripted, recorded in advance, or presented via computer).
- Objectivity (make coding schemas as objective as possible, automated recording equipment)
- Single-blind research
What are some participant errors?
Random: Carelessness, distraction
Bias: Demand Characteristics, good participant effect, response bias
What are demand characteristics?
Features of an experiment that seem to inadvertently cause participants to act in a certain way.
What is the good participant effect?
Tendency for participants to behave as they perceive the researcher wants them to behave.
What is it called when demand characteristics and the good participant effect work together?
Pact of Ignorance (Orne 1968).
How do we control for demand characteristics?
Conduct double-blind research
Deception
What is response bias?
When the context affects the way a participant responds. (yes sayers vs nay sayers). Social desirability is also an issue.
How do we control for response bias?
Include both “agree” and “disagree” items.
Randomize question presentation
Careful review of questions/setting
Pilot testing
What are some types of observer error?
Random: carelessness, distraction
Observer/scorer bias: Confirmatory bias (see what we want to see).
How do we control for observer error?
Eliminate human observer-use mechanical measure instead
Limit observer subjectivity-focus on observable behaviour, use standardized coding schema
Make observer “blind”-unaware of experimental condition
What is construct validity?
The extent to which your manipulation or measure actually represents the claimed construct (ex: does your measure of extraversion actually measure obnoxiousness?)
What are some criteria needed for construct validity?
1) Reliability
2) Content validity
3) Convergent validity
4) Discriminant or divergent validity
What is reliability?
The repeatability or consistency of the research
What is test-retest reliability?
Comparable scores on retest. Relationship between scores at time 1 and time 2. The difference between the observed value and 1 equals the proportion of random error in scores.
What is inter-rater reliability?
Comparable scores between observers. Calculated the same way as test-retest, and then percentage agreement.
What is internal consistency?
The extent to which responses to items that propose to measure the same construct are similar. Variability across items may be due to random error or more than one construct being assessed.
How do we test internal consistency?
Average inter-item correlation
Split-half correlation
Cronbach’s Alpha
How do we improve internal consistency?
Add items/questions-random error balances out. Create better items/questions (reduces potential variability in interpretation)
What is content validity?
The extent to which a measure covers all aspects of a construct. Ex) measuring love measures commitment, sexual attraction, liking.
How do we make sure something measures all aspects of a construct?
Theory, definitions, experts. Make sure to measure all component dimensions, have a large enough set of measures for each dimension.
What is convergent validity?
The extent to which a measure correlates with other indicators of the same construct. ex) People scoring high on your measure should also score high on other measures of the same construct.
How do we asses convergent validity?
Similar measures, known comparison groups, other indicators of construct.
What is discriminant validity?
The extent to which your measure is distinguishable from related constructs. People scoring high on your measure should not be scoring as high on measures of similar constructs.
What is discriminant validity?
The extent to which your measure is distinguishable from other constructs (unrelated constructs). People scoring high on your measure should not also score high on measures of the “wrong” construct.
What else do you need in a research study.
Best measure. Best fit for the research context. Additional issues-is the scale appropriate to the context? Is the measure sensitive enough?
What is sensitivity?
Ability of measure to detect effects. Is the measure strong enough for what you want to study? Does your measure minimize the influence of error?
How do we achieve sensitivity in measurement?
Use measure with score variance (avoid restriction of range)
Avoid all or nothing measures (ask how much instead(
Add scale points to rating scale
Pilot test measure.
