Chapter 6 Flashcards

Surveys and Observations

1
Q

The Butterfly Ballot

A
  • Confusion over Palm Beach Count ballot
  • The central problem: Poorly designed questionnaire measure which changed course of US history (Bush won over Al Gore)
  • Similar challenge for Psychology Research: assess beliefs/moods/intentions/etc., make unobservable into “countable,” assign “values” to psychological states
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Measuring Psychological Constructs

A
  • Converting feelings, attiudes, beliefs, and thoughts into numbers
  • tallied, averages, divided, correlated; reported verbally; or observed indirectly (neuroscience)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Construct Validity of Surveys and Polls

A
  • Many people may have been asked to participate in surveys/polls, method of questioning people
  • Online surveys, in person, written interviews
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is construct validity?

A

The degree to which a test or measurement tool accurately captures the theoretical concept it is intended to measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Choosing Question Formats

Open-ended questions

A
  • People can respond in any way they like and can say anything
  • Use their own words
  • Unpredictable responses
  • An example of a self-report question format
  • Ex: As you know, this study is primarily concerned with quality of life. What does quality of life mean to you?
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Choosing Question Formats

Structured Self-Report Questionnaire

A
  • Converts ideas to numbers
  • Self report inventories and behavioral ratin scales
  • Characteristics: Standard questions, everyone sees the same test, forced-choice format, highly reliable
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What are the strengths of structured self-report?

A
  • Specific administration rules
  • Standardized methods for scoring
  • Strict guidelines for interpretation
  • Strict rules for reporting of results
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Fixed Alternative Question aka Fixed Choice

A
  • respondent selects answer from a set of specified responses
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Question Formats: Forced Choice

Likert Scales

A

Respondent indicates extent to whcih they agree or disagree with the statement (strongly disagree, disagree, somewhat disagree, etc)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Question Formats

Semantic differential format

A
  • When there is an anchor at both ends
  • Nothing in middle, left to interpretation
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Wording Questions Well

Question Wording Matters- Delaware Voters ex.

A
  • Looked at support for voter ID law
  • Randomly assigned one of 3 versions of a question
  • Leading questions: wording leads people to answer a particular way
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Wording Questions Well

Double-barreled Questions

A
  • How much do you love vacation and final exams?- these questions have poor construct validity
  • Should instead ask one thing at a time, don’t ask participants to evaluate 2 different things with one response
  • Break complex questions into 2-3 simpler questions
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Wording Questions Well

Keeping it simple

A
  • Avoiding complexity: one thought or concept at a time
  • Avoid negations: “Not” can cause confusion, especially double negatives
  • Use similar words instead (not standing–> sitting)

Easter egg: bad one= i do not-should not-have not

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

REMEMBER

A

need to watch ted talk on are we in control of our decisions by dan ariely

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Wording Questions Well

Question Order

A
  • The order of questions can also affect responses
  • One of the best ways to control for this is to randomize question order and create several versions of your survey
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Wording Questions Well

Use informal Language

A
  • Consider your intended audience: age/generation, reading skill/level of education, don’t patronize/oversimplify
  • No psychology jargon: Hallucinations–>hearing voices, anorexic–>body image, aspirations–>wishes/dreams
17
Q

Wording Questions Well

Forced Choice and Variance

A
  • Choosing yes or no: one of two responses is the only option
  • Problems: may not provide enough information, may not have enough variance, liking apples vs oranges
18
Q

Wording Questions Well

Variance is Desirable

A
  • Don’t want questions that are answered the same way by everyone. Ex. Do you find puppies cute?
  • Recall the Goal: to study differences/changes. People’s answers to questions must vary. Hard if everyone gets 100% or 0% on the test
19
Q

Wording Questions Well

Variance is desirable- restriction of range

A
  • Restriction of Range: people’s score measure little variation/ few differences
  • floor effects: everyone responds at a low level on your measure of DV
  • ceiling effects: everyone responds at a high level on your measure of DV
20
Q

Why is restriction of range bad? What is the solution?

A
  • Restriction of range is not desirable because it is hard to predict anything based on scores that measure a non-varied DV
  • Could be your construct (ex puppies) or your measure
  • Solution is to pilot your DV measure
21
Q

Wording Questions Well

Response Bias

A
  • Solutions to the problem:
  • reverse-worded items
  • neutral language in questions
  • use “normalizing” questions (questions designed to make a participant feel like their experiences are common or typical)
  • stressing anonymity
22
Q

Wording Questions Well

Response Bias

A
  • Social desirability bias: answering the way you think is appropriate/expected/desirable, to loaded/sensitive topics
  • Response set bias (non differentiation), always giving middle-of-the-road answer (fence sitting), always agreeing (acquiescnce/yea-sayer), alsways disagreeing (naysayer)
23
Q

Wording Questions Well

How to prevent response bias?

A
  • Guarantee anonymity: participant answers never linked to their actual identity
  • Researchers sees only the groups’ answers or see only participants’ secret code
24
Q

Wording Questions Well

How many questions is enough?

A
  • Sometimes: need only ask one questions
  • Often-times: need several times to capture construct
  • More is better for reliability, idiosyncratic answers tend to average out
25
# Wording Questions Well Sensitive to diversity
* Not taking typical for granted: your representative sample is heterogeneous; culture/gender/sexual orientation * Language can't alienate sub-groups: mother --> primary caregiver, spouse --> significant other
26
# Wording Questions Well Sensitive to personal topics
* Your wife is on the roof: when dealing w/ sensitive topics tread lightly (wording) and use questions that progress in intensity * Easy/innocuous first, and then difficult/sensitive later to get more honest responses
27
Observer bias
* Observations can often be better than self-report * Although it's still work to make you observations reliable and valid * Observer bias- when expectation influence interpretations of behaviors
28
Observer bias example
* participants (psychotherapists) watched the same video of a person answering questions about work experiences * Group 1 was told this man was a patient, and his behaviors were rated as defensive ad frightened of his aggressive impulses * Group 2 was told the man was a job applicant, rated him as attractive, candid, and innovative
29
Observer effects examples
* Students w/ maze bright and maze dull rats- rats labeled as bright performed significantly better simply because the researchers expected then to * Clever hans: when someone or an animal sense what another person wants them to do without being given any deliberate signals. Named after clever hans, a horse that appeared to perform intellectual tasks like arithmetic and reading but was just responding to subtle cues from his trainer
30
How to prevent observer bias?
* Clear codebooks: instructions on how to rate behavior * Using multiple observers: for interrater reliability * Masked (blind design): researchers are unaware of purpose of study and/or who is in which group
31
Reducing reactivity
* People change behavior when they know they are being watched * Blend in: unobtrusive observations, researcher is less noticeable * Wait it out: let participants "get used" to the researcher, works in weight loss studies * Measure behavioral results: measure traces of behavior/indicators of behaviors