Lecture 2: Survey Methods

Question 0

Why surveys?

Answer 0

• Limits of behavioral observation
• Allows gathering large amounts of information
• Online surveys and global research
• weaknesses: self-report, biases (social desirability), memory errors and limits of insight

Question 1

What is ‘descriptive research’?

Answer 1

• Description of individual variables
• Describe how things are, rather than explaining why they are like that
• Observational & survey research
• Survey then perform descriptive statistics

Question 2

Why collect demographic data?

Answer 2

• Is your sample representative?

- confounds? E.g gender, ESL

Question 3

Describe open-ended questions & advantages/disadvantages

Answer 3

Respondents formulate their own answers
Advantages
- allows any scope of response, no direction given
Disadvantages
- difficult to code/interpret for researcher, compare responses
- difficult for respondent to articulate
- more taxing for respondents, harder to motivate

Question 4

Describe close ended questions & advantages/disadvantages

Answer 4

T/f
Advantages
- easier to complete, compare, less ambiguous, quantification
Disadvantages
- restricted response, may not capture answer

Question 5

Rating scale

Answer 5

• close ended response
• Numerical value on predetermined scale with semantic anchors
• likert scales - can choose even numbers to avoid ‘neutral’ option, need to justify

Question 6

Scale construction

Answer 6

• most researchers use established scales

- pre-testing & establishing psychometric properties (reliability & validity) determines a good scale

Question 7

Problems with scales

Answer 7

• Double negative
• Avoid double-barreled items
• leading questions
• avoid prestige bias

Question 8

Sources of bias in scale response

Answer 8

• Social desirability -> embed lie scales

- semantic anchors should allow equal +ve/-ve response values

Question 9

Response set
Acquiescence response set
Solutions

Answer 9

• RS: tendency to enter same response e.g neutral
• ARS: tendency to agree irrespective of content of the question
• too many questions may set up response set
• solutions: reverse scored items, forced choice items

Question 10

What is validity?

Answer 10

Do they really assess what they are intended to assess?

Question 11

Construct validity

Answer 11

Does the scale correlate with the construct?

Question 12

Convergent validity

Answer 12

E.g. Two tests of (same construct) verbal IQ should correlate

Question 13

Divergent validity

Answer 13

Two tests of different constructs shouldn’t correlate

Question 14

Concurrent validity

Answer 14

New and old test should correlate positively but not perfectly

Question 15

Criterion/predictive validity

Answer 15

How well does the test correlate with the criterion? How well does it predict what it is supposed to predict?

Question 16

Reliability

Answer 16

• Consistency of assessment
• test-retest reliability
• internal consistency reliability: Cronbach’s alpha
  0. 7 adequate, 0.8 good, 0.9 excellent