Week 9-Survey Methods Flashcards
Explain what Surveys are
■Systematic collection of info.
■Predetermined questions.
■Verbal or written.
■Questionnaires and structured
interviews.
■Survey = process.
■Questionnaire/interview = tool.
What ways can you administer surveys?
-mail
-internet
-phone
-group
-structured interview
-personally
Evaluate Mail surveys
+Convenient.
– Response rate/bias (Dillman, 2000).
Evaluate internet surveys (email/website)
+Efficient and cheap.
+Convenient for participants.
+Large and diverse sample (including niche groups).
–Representativeness (less of an issue now).
Evaluate phone surveys
+Some questions easier to ask.
+Large and diverse sample.
–Selection bias.
–Interviewer bias.
Evaluate group surveys
+Captive audience.
+Large amount of data quickly.
–Privacy/anonymity.
–Pressure.
Evaluate structured interviews
+Same questions, same order.
–Interviewer bias/Social context
Evaluate Personal (FtF)
+Convenient/Large sample.
+Good response rate.
–Representativeness/Demand
Characteristics/ Questionnaire fatigue.
What do Psychometric Tests measure?
■Ability/Aptitude: E.g. Numerical/verbal reasoning.
■Personal Qualities: E.g. Personality/attitudes.
–Employment suitability.
What’s the history of survey methods?
■China from 2000 BC.
■Proficiency tests.
■605-1905 Chinese civil service exams, testing fundamental skills.
■Influenced development of exam/testing systems both in the East and in the West.
Explain the history of Robert Yerkes (WW1) - Army Alpha and Beta tests (1917)
–Systematic method of evaluating intellectual and emotional functioning.
–Verbal/numerical ability, ability to follow directions etc.
–Capability of serving, job classification, leadership
potential.
–Beta test – non-verbal (for illiterate or foreign soldiers).
What’s the flaw with some of the historical tests?
■Flawed
–Measure acculturation and
education.
–Longer in the US - better marks.
–Better education – better marks.
–Cultural and educational bias.
What’s the Woodworth Personal Data Sheet?
–A test in World War 1 testing emotional stability - susceptibility to shell shock.
–First personality test which was highly influential in the development of later personality tests.
What Modern-day tests are used?
■Stanford-Binet IQ test.
■Children with learning difficulties.
■ Used today for:
–Clinical and neurological assessment.
–Educational placement.
■First successful intelligence test.
How is a questionnaire designed?
■Topic e.g., Beliefs/attitudes/Facts/ Behaviour.
■Type.
■Draft.
■Re-examine/revise (ideally with
experts).
■Pre-test/pilot study.
■Edit and detail procedures for use.
What questions should be avoided in a questionnaire?
–Double-barrelled questions
■E.g. How satisfied are you with your essay marks and feedback?
–Loaded questions
■Bad: Do you think our hardworking and caring doctors and nurses should be paid more?
■Good: E.g. Do you think the NHS budget should be increased?
–Negative wording
■Bad: Do you think that students shouldn’t have to pay tuition
fees?
■Good: Do you think that students should pay tuition fees?
Define open-ended/partially open-ended questions
■Open-ended:
–Qualitative, descriptive response.
■Partially Open-ended:
–Multiple choice plus Other
________
Evaluate open-ended questionnaires
+Detailed answers.
+/-Quick to design, long time to
analyse.
–Subjective interpretation.
+Participant led (unanticipated data).
Give examples of close-ended questionnaires
–True/False.
–Multiple Choice.
–Likert Scale.
What are some of the problems with close-ended questionnaires?
–Guessing (T/F = 50/100).
–Unsubtle.
+/-Complex to design, but quick to
mark.
–Theory-led.
Explain the Questionnaire Fallacy
The mistaken belief that a
questionnaire provides a true picture of what people
do and think.
How do you create a Likert scale?
–Choose a specific topic.
–Formulate statements.
–Pilot test.
True or false: high-frequency scales should be used for sub-specific groups
True
What’s a Semantic Differential Scale?
Connotative meaning - Bi-polar
adjectives e.g., strong and weak and you rate the level you feel closest with.
What are examples of Questionnaire Bias?
■Order Effects/Priming.
■Demand Characteristics.
■Social Desirability.
■Acquiescence.
■Extreme and Neutral Responses.
■Cultural.
Define Order Effects
■Question order can affect responses.
■Detailed to General.
What’s Priming and how can it be overcome aswell as order effects?
Thinking about the answer to one question whilst answering the subsequent question.
Overcome by:
–Counter-balance questions.
–2 versions:
One order then another, compare results.
–Randomise every question.
Explain Social Desirability
■P’s try to make themselves look as ‘good’ as possible e.g., Eysenck Personality Inventory
■Increase the positive or decrease
the negative.
■Can be accentuated by the type of questionnaire/setting E.g., Drug use.
■Emphasise anonymity.
What’s Acquiescence?
■Yea-saying.
■Nay-saying.
■Balance of negative and positive
questions.
When are Extreme and Neutral Responses used?
■Mostly found in questionnaires using Likert Scales.
–Neutral - not interested? Bored or fatigued?
–Extreme – Yea/nay-saying? Topic?
–Check raw data.
Why does Cultural Bias arise?
■Early IQ tests culturally biased.
■Language that could be misunderstood.
■Various interpretations of words.
■Social attitudes/desirability differs e.g., Homosexuality.
■Complex or ambiguous academic
terminology.
Explain Psychometric Tests
■Avoids response bias.
■Implicit psychological processes.
■Unconscious aspects of human nature.
■Implicit vs explicit.
■Explicit = “Do you drink alcohol?”
■Implicit = “I prefer drinking with lots people” etc.
What assumptions are there in Attitude Questionnaires?
–Attitudes can be expressed using verbal statements.
–Statements will have the same meaning for all participants.
–Attitudes expressed as verbal statements can be quantified.
What are the problems with attitude questionnaires?
–Consistency with behaviour.
–Social desirability.
–Ambivalence.
–Normative response bias.
–Lie Scale (helps check honesty).
E.g. ‘I never say things I later
regret.’ OR ‘I always tell the truth.’
What are Word Association Tests?
■First introduced by Francis Galton (who also founded psychometric psychology)
■Developed by Jung as a clinical diagnostic tool (but not just used in a clinical setting)
■Jung claimed that by using WAT he could identify:
–Abnormal patterns of response.
–Psychological complexes.
–Intellectual and emotional deficiencies.
What are the advantages and problems of WATs?
A:
–Quick - easy to administer.
–Predict prospective drug use. (Ames et al., 2007)
P:
–Colloquialism.
–Standardised procedures (across cultures)?
–Implicit?
What are Implicit Tests?
■Implicit cognitive tasks.
■Infer attitude and beliefs from performance on different tasks.
■Often use reaction times.
■E.g. IAT (Implicit Association Test).
–Automatic association between concepts.
–Greenwald, McGhee, Sherry and Schwartz (1998). Age, gender, race etc.
What are IATs?
■Computer- based.
■Categorise target concepts with an attribute/picture as quickly as possible.
■Faster pairing = stronger association.
What are the criticisms of implicit tests like the IAT?
–Cultural views vs beliefs and
attitudes.
–Ecological validity - Attitude vs behaviour?
What’s a key element of Psychometric Tests and what types are there?
Reliability:
–External – consistent across time and setting.
–Internal – all items measure the same thing.
–Test-retest reliability.
–Split-half reliability.
What’s Classical Test Analysis?
■Observed score (X), true score (T), random error score (E).
■X = T + E.
■Random error (changes from person to person, time to time):
–Failing to read questions properly.
–Answering in a socially acceptable way.
–Good/bad mood.
–Tired or distracted.
What’s Systematic error + characteristics of test?
Predictable across people,settings, etc.
■Characteristic of test.
–E.g. Happiness: How often do you… cinema, theatre, shopping, holiday, visit friends,
exciting experiences?
–Influence of participants’ wealth.
–Problem with validity.
Define validity + its types
The extent to which your measure is measuring what it is supposed to.
–Content validity: Covers all behaviours/aspects.
–Construct validity: Measures theoretical construct.
–Criterion-oriented validity: Correlates with established measures.
Explain Standardisation
–Standardised instructions and
procedures.
■Established population norms:
–Age – children, teens, adults etc.
–IQ : UK average.
■Must relate to established tests and theories.
■Already validated, widely used tests.
■Social desirability.
Questionnaires - Give Key Terms & Distinctions
■Knowledge Based:
–Ability, aptitude, achievement.
–Intelligence tests, and clinical
assessment instruments.
■Person Based:
–Personality, mood, attitude.
–Differences, categorisation.
Give 2 types of Response references
■Normative Reference Testing:
–Scores are compared to the norm e.g., sample mean.
■Criterion Reference Testing:
–Scores are compared to a pre-
determined criteria.
Explain Normative reference testing
–Compare to the norm.
–Mean or Median splits.
E.g., Number of units of alcohol per week.
Above the mean/median = heavy
drinkers.
Below = light drinkers.
Easy but arbitrary.
Explain Criterion Reference Testing and how is it restrictive?
–Compare to pre-determined criteria.
–At risk? Normal/pathological?
E.g. AUDIT - 11+ = hazardous
drinking.
Beck Depression Inventory (BDI) 9+ = depressed.
■Restrictive:
–Individual differences?
–Non-clinical samples (student
drinkers)?
Explain Criterion Reference Testing and how is it restrictive?
–Compare to pre-determined criteria.
–At risk? Normal/pathological?
E.g. AUDIT - 11+ = hazardous
drinking.
Beck Depression Inventory (BDI) 9+ = depressed.
■Restrictive:
–Individual differences?
–Non-clinical samples (student
drinkers)?
