Chapter 6 Flashcards
Surveys and Observations
The Butterfly Ballot
- Confusion over Palm Beach Count ballot
- The central problem: Poorly designed questionnaire measure which changed course of US history (Bush won over Al Gore)
- Similar challenge for Psychology Research: assess beliefs/moods/intentions/etc., make unobservable into “countable,” assign “values” to psychological states
Measuring Psychological Constructs
- Converting feelings, attiudes, beliefs, and thoughts into numbers
- tallied, averages, divided, correlated; reported verbally; or observed indirectly (neuroscience)
Construct Validity of Surveys and Polls
- Many people may have been asked to participate in surveys/polls, method of questioning people
- Online surveys, in person, written interviews
What is construct validity?
The degree to which a test or measurement tool accurately captures the theoretical concept it is intended to measure
Choosing Question Formats
Open-ended questions
- People can respond in any way they like and can say anything
- Use their own words
- Unpredictable responses
- An example of a self-report question format
- Ex: As you know, this study is primarily concerned with quality of life. What does quality of life mean to you?
Choosing Question Formats
Structured Self-Report Questionnaire
- Converts ideas to numbers
- Self report inventories and behavioral ratin scales
- Characteristics: Standard questions, everyone sees the same test, forced-choice format, highly reliable
What are the strengths of structured self-report?
- Specific administration rules
- Standardized methods for scoring
- Strict guidelines for interpretation
- Strict rules for reporting of results
Fixed Alternative Question aka Fixed Choice
- respondent selects answer from a set of specified responses
Question Formats: Forced Choice
Likert Scales
Respondent indicates extent to whcih they agree or disagree with the statement (strongly disagree, disagree, somewhat disagree, etc)
Question Formats
Semantic differential format
- When there is an anchor at both ends
- Nothing in middle, left to interpretation
Wording Questions Well
Question Wording Matters- Delaware Voters ex.
- Looked at support for voter ID law
- Randomly assigned one of 3 versions of a question
- Leading questions: wording leads people to answer a particular way
Wording Questions Well
Double-barreled Questions
- How much do you love vacation and final exams?- these questions have poor construct validity
- Should instead ask one thing at a time, don’t ask participants to evaluate 2 different things with one response
- Break complex questions into 2-3 simpler questions
Wording Questions Well
Keeping it simple
- Avoiding complexity: one thought or concept at a time
- Avoid negations: “Not” can cause confusion, especially double negatives
- Use similar words instead (not standing–> sitting)
Easter egg: bad one= i do not-should not-have not
REMEMBER
need to watch ted talk on are we in control of our decisions by dan ariely
Wording Questions Well
Question Order
- The order of questions can also affect responses
- One of the best ways to control for this is to randomize question order and create several versions of your survey
Wording Questions Well
Use informal Language
- Consider your intended audience: age/generation, reading skill/level of education, don’t patronize/oversimplify
- No psychology jargon: Hallucinations–>hearing voices, anorexic–>body image, aspirations–>wishes/dreams
Wording Questions Well
Forced Choice and Variance
- Choosing yes or no: one of two responses is the only option
- Problems: may not provide enough information, may not have enough variance, liking apples vs oranges
Wording Questions Well
Variance is Desirable
- Don’t want questions that are answered the same way by everyone. Ex. Do you find puppies cute?
- Recall the Goal: to study differences/changes. People’s answers to questions must vary. Hard if everyone gets 100% or 0% on the test
Wording Questions Well
Variance is desirable- restriction of range
- Restriction of Range: people’s score measure little variation/ few differences
- floor effects: everyone responds at a low level on your measure of DV
- ceiling effects: everyone responds at a high level on your measure of DV
Why is restriction of range bad? What is the solution?
- Restriction of range is not desirable because it is hard to predict anything based on scores that measure a non-varied DV
- Could be your construct (ex puppies) or your measure
- Solution is to pilot your DV measure
Wording Questions Well
Response Bias
- Solutions to the problem:
- reverse-worded items
- neutral language in questions
- use “normalizing” questions (questions designed to make a participant feel like their experiences are common or typical)
- stressing anonymity
Wording Questions Well
Response Bias
- Social desirability bias: answering the way you think is appropriate/expected/desirable, to loaded/sensitive topics
- Response set bias (non differentiation), always giving middle-of-the-road answer (fence sitting), always agreeing (acquiescnce/yea-sayer), alsways disagreeing (naysayer)
Wording Questions Well
How to prevent response bias?
- Guarantee anonymity: participant answers never linked to their actual identity
- Researchers sees only the groups’ answers or see only participants’ secret code
Wording Questions Well
How many questions is enough?
- Sometimes: need only ask one questions
- Often-times: need several times to capture construct
- More is better for reliability, idiosyncratic answers tend to average out