Survey Methods Flashcards
Descriptive Survey
- Descriptive <> Describe %
- Researcher seeks to determine what percentage of the population has a certain characteristic, holds a certain opinion, or engages in a particular behavior.
- Do you approve of how president Bush is handling the war in Iraq?
- Ex: What percent of the population…?
Analytic Survey
•Analytic <> Associated Variables
•Researcher seeks to determine relevant variables and how they might be related.
•What factors are involved in the experience of depression.
Ex: What are the factors associated with …?
Two Types of Surveys
- Descriptive
2. Analytic
Five Types of Survey Questions
- Yes-No Questions
- Forced Alternative Questions
- Multiple Choice Questions
- Likert-Type Scales
- Open-Ended Questions
Yes-No Questions
Have only Yes or No answers
Ex: The thought of death seldom enters my mind? Yes or No
Forced Alternative Questions
Only get two options at the polar extremes from each other and must choose one or the other.
Ex a: There are institutions in our society that have considerable control over me.
Ex b: Little in this world controls me. I usually do what I decide to do.
Locust of Control
Internal Locust: Whether or not we have control over things
External Locust: Whether things have control over me (Luck, outside influences).
Terror Management Theory
We are all aware on some level that we are going to die.
If you make one aware that you are dying you will become more racist.
Fear of dying -> in group focus. Don’t like people outside our group.
Multiple Choice Questions
Compared to the average student:
A. I give much more effort
B. I give an average amount of effort
C. I give less effort
Likert-Type Scales
Usually 5, 7 points. A point value scale for characteristics.
Ex: I enjoy social gatherings to be with people:
1. Not at all Characteristic
2. …
3. …
4. …
5. Very Characteristic
Open-Ended Questions
Not Yes or No questions
Ex 1: How would you summarize your chief problems in your own words?
Ex 2: Describe what you did today?
Mistakes to Avoid When Writing Survey Questions
- Avoid Leading Questions
- Avoid Social Desirability Bias
- Avoid Double-Barreled Questions
- Avoid Long Questions
- Avoid Negation
- Avoid Big Words
- Avoid Words That May Be Misinterpreted
Avoid Leading Questions
A question that leads participant to the answer you want.
Ex: You disapprove of the biased, horrible way that television news covers the abortion issue, don’t you?
Avoid Social Desirability Bias
Socially correct answers.
Ex: Do you donate money to worthwhile causes?
Avoid Double-Barreled Questions
Asking two questions at once; two factors in same question.
Ex: How much do you agree with the following statement:
Colleges need to spend more time on students emotional AND physical development.
Avoid Long Questions
No more than 10 words.
Avoid Negation
Negative sentence format.
Ex 1: Do you NOT LIKE it when students DON’T STUDY?
Ex 2: Do you LIKE it when students STUDY?
Avoid Big Words
Keep questions at a fourth grade reading level.
Avoid Words That May Be Misinterpreted
Ex: Junk food: could have lots of different meanings to lots of different people.
Pretesting
Bring in, Look at Questions
Bring in participants, give them the questions, and ask to critique them
On target audience.
Order of Questions
1. Put personal questions last So people will answer. So you don't change how people answer. (stereotype threat) 2. Put demographic questions last 3. Keep similar questions together. 4. Be aware if response sets.
Response Sets
When survey participants do not answer questions truthfully.
1a: Acquiescence
1b: Nay-Saying
2a: Social Desirability
2b: Faking Bad
3a: Extremity
3b: Cop-Out
4a: Inconsistent
4b: Fabrication
Acquiescence
Tendency to answer Yes or True
• Always Yes/True
Nay-Saying
Tendency to answer No or False
•Always No/False
•Avoid by using Reverse Coding, changing wording on questions.
Social Desirability
Tendency to present oneself in a socially acceptable light.
•Socially Acceptable
Faking Bad
Tendency to present oneself in a socially unacceptable light.
•Socially Unacceptable
•”Betty Badass”
•Avoid by using: Lie Scale
Extremity
Tendency to select the extreme alternatives.
•Always Great/Aweful
Cop-Out
Tendency to select neutral answers.
•Always OK/Maybe
Inconsistent
Tendency to give careless answers, not read thoroughly.
•Don’t care
Fabrication
Tendency to make up answers to questions even when they are unaware of the issues involved.
•Make up answers
Controlling for Response Sets
- Lie Scales
2. Negative Wording?Reverse Coding
Lie Scales
Questions that are used to detect response sets and other dishonest responses.
•Used to avoid Faking Bad and social desirability
Negative wording/Reverse Coding
The good answer is changed from all yes to now no answers.
•Used to avoid Nay-saying, Acquiescence, and Inconsistent responding.
Formats of Surveys
- Mail Survey
- Personal Interview
- Telephone Survey
Mail Surveys
Pros: •Lots of people Cons: •Don't know who filed it out •Low return rate •Potential biased interpretation
Ex: Women and Love Survey
1/2-million surveys went out and 12,000 came back = >10%.
•Must Have 67% Response Rate >50% responses - inadequate 50% responses - adequate 60% responses - good 70%< responses - very good
Types of Personal Interviews
- Structured
2. Semi-Structured
Structured Personal Interview
Standard list of questions in a standard order.
Semi-Structured Personal Interview
Core standard of questions without a strict order.
Pros: High response rates Cons: •Expensive •Interviewer Bias •Social Desirability Bias
Telephone Interview
Pros: •Reduced bias •Not face-to-face Cons: •Expensive •Interviewer Bias •Social Desirability Bias •Lack of Rapport (easy to hang up)
Tests and Inventories
Assess a specific attribute, ability, or characteristic possessed by the individual being tested.
Reliability
Consistency in measurement over time.
The extent to which the test is consistent in its evaluation of the same individual over repeated administrations.
How Reliability is Assessed
- Test-Retest
- Parallel or Alternate Forms
- Internal-Consistency
a: Split-Half
b: Coefficient (Cronbachs) Alpha
Test-Retest
1 group of people
1 test
2 time periods
•The HIGHER the reliability, the HIGHER the correlation coefficient.
•Higher correlational coefficient higher the reliability.
Ex: height over time
Parallel or Alternative Forms
1 group of people
2 different tests that are parallel/equivalent
2 time periods
Ex: Midterm
Form A
Form B
Cons: Risk loosing participants at 2nd time period.
Internal-Consisitency
1 group of people
1 test
1 time period
Correlate parts of the tests with each other.
Internal-Consistency:
Split-Half
Divide test in halves and correlate the two halves in some way. High correlation.
Ex: Questions 1-20, correlate with Questions 21-40
or Evens and Odds
Internal-Consistency:
Coefficient (Cronbachs) Alpha
Average correlation between every item and every other item on the test.
Ex: 40 item midterm
right = 1, wrong = 0
correlate 1-1, 1-2, 1-3, 1-4…
get one right you should get them all right.
Best test questions are where the top 25% of class get them right and the bottom 25% get them wrong.
Validity
Are you measuring what you say you are measuring?
Reliability and validity are NOT the same thing.
Types of Valitidy
(Like a Pyramid you have to build from the bottom up)
- Content Validity (bottom)
- Criterion Related Validity (middle)
- Construct Validity
Content Validity (BOTTOM)
Does your measure include the universe of all of the relevant content of phenomenon that you want to measure
•Content that is on your survey, basics, items.
Types of Content Validity
Must establish these two to get Content Validity:
- Face Validity
- Sampling Validity
Content Validity
Face Validity
Does the measure look like its measuring what it’s supposed to?
Questions should be relevant to your subject.
Content Validity
Sampling Validity
Get expert opinions on whether the whole range of the construct is captured by your measure. What is important and not to subject.
Criterion Related Validity (MIDDLE)
How well does the test estimate present or future performance.
Types of Criterion Related Validity
- Concurrent Validity
2. Predictive Validity
Criterion Related Validity
Concurrent Validity
Concurrent: At the same time.
Do scores on the test correlate with other measures of the construct.
Do multiple sources of information agree?
Criterion Related Validity
Predictive Validity
Does your instrument predict future performance for the construct accurately?
Construct Validity (TOP)
How well does the instrument measure the theoretical construct it was designed to reflect?
Types of Construct Validity
- Convergent Validity
2. Divergent Validity
Construct Validity
Convergent Validity
Converge: Come (funneled) together.
Concurrent and predictive come together. The extent to which a measure correlates to all existing measures related to the same construct.
Construct Validity
Divergent/discriminant Validity
Divergent: To separate.
The extent to which a measure does not correlate with measures of a different construct.
Can be Reliable but not be Valid
Can’t be Valid without being Reliable
Types of Tests and Inventories
- Achievement Tests
- Aptitude Tests
- Personality Tests
- Objective Tests
Achievement Test
Evaluates an individual’s level of mastery or competency.
•Level of Mastery
Ex: Midterm Exam
Aptitude Tests
Assess an individual ability or skill in a particular situation or job.
•Potential ability or skill
Ex: ASVAB, IQ, LSAT
Personality Tests
Measures a specific aspect of an individual’s personality.
Personality Tests
Objective Tests
Someone’s personality is assessed by the answers they give on standardized questions.
Ex: NEO-PI Personality Inventory
The Big Five
Uncorrelated, can have every single one in a section and low in another section.
•People don’t change. They are who they are, you are who you are.
NEOAC = OCEAN
The Big Five: N
Neuroticism
tendency to experience unpleasant emotions. More likely to be anxious, angry, depressed, self-conscious. Challenging type of people.
Ex: ask how their day was at different times and their is still a long chaotic list of negatives.
Neurotism |——| Emotional Stability
The Big Five: E
Extroversion
Tendency to seek stimulation and to enjoy the company of others. If you are pumped up by the more people you are around you are extrovert, if you thrive on being alone you are introvert.
Ex: Facebook: pictures and friends have lots.
The Big Five: A
Agreeableness
Tendency to be compassionate toward others.
The Big Five: C
Conscientiousness
Perfectionists
The Big Five: D
Openness to Experience
Tendency to enjoy new things.
Personality Inventories
Projective Tests: Tell me what you see?
- Rorschach Inkblots
- Thematic Apperception Test
- Implicit Personality Test
