Chapter 9: Test Construction and Sampling probability

Question 1

What are the steps for questionnaire design?

Answer 1

1. Determine the purpose of the questionnaire
2. Determine the type of questions
3. Write the Items
4. Determine how the data will be analyzed
5. Determine the method of administration

Question 2

A widely used method of gathering scientific info
determine the pop. thoughts about an issue, effect of an even on behaviour

opportunity to examine correlations among the participants responses to look for possible patterns of cause and effect

Answer 2

Survey

Question 3

What is a major function of surveys?

Answer 3

Dispel myths

Question 4

What is in involved in the first step of questionnaire design (determining the purpose of the questionnaire)?

Answer 4

what do you expect to accomplish with this survey?
anticipate question of interpretation that may arise when data is received- based on population
questions that are unbiased towards genders, background

Question 5

What is in involved in the second step of questionnaire design (Determine the types of questions)

Answer 5

determining which types of questions to employ in the survey-there are two basic categories: open and closed-ended

Question 6

What are open ended questions?

Answer 6

respondents answer in their own words

Question 7

What are closed-ended questions?

Answer 7

one that limits the respondents to certain alternatives determined in advance by the questionnaire’s designers

Question 8

What are the advantages of open ended questions?

Answer 8

permits the respondents to answer more completely and reveal reasoning behind answers

more likely that questionnaire will discover something unanticipated by survey designers

Question 9

What are the disadvantages of open ended questions?

Answer 9

harder to code (since answers are in narrative form)

necessary to categorize responses in a way to summarize data after survey is complete (breaks the rule since you don’t know in advance how you will analyze your data)

more effort is required from respondents

more difficult for less articulate respondents

Question 10

When are open ended questions employed in surveys?

Answer 10

useful for smaller and preliminary studies (manageable)
by prelim studies: you can determine the range of likely answers, which permits you to standardize the alternatives into a closed-ended format for large administration

Question 11

What are the advantages of closed-ended questions?

Answer 11

easier to code and analyze

easier to less articulate respondents

Question 12

What are the disadvantages of closed-ended questions?

Answer 12

issues may be too complex to reduce to a small set of alternatives

respondents may not agree with any of the items or agree with more than one= simplistic results

suggest an alternative not thought up by respondent

errors: misinterpret the question/ clerical error in coding the data

Question 13

How do you reduce errors in closed-ended question surveys?

Answer 13

many questionnaires required that each response is recorded in two places so the responses can be tested for consistency

Question 14

When are closed ended questions used for surveys?

Answer 14

large studies since standardization is easier

Question 15

What are the advantages of using a combo of both types of questions in surveys?

Answer 15

respondents are offered the opportunity to expand on their answers to a closed-ended question
data is coded and analyzed easily but gives insight to reasoning by respondent

Question 16

What are the basic principles of item writing?

Answer 16

address a single issue per item

avoid bias

make alternatives clear

beware of the social desirability tendency

beware of acquiescence

determine the format of the item

determine the sequence of the items

Question 17

address a single issue per item

Answer 17

items should be unambiguous
each item should address a single issue and in a clear, unbiased fashion
do not combine an opinion and a reason
ex. college students should receive grades because this prepares them for the work world

fix: college students should receive grades
another item: preparing students for the work force

Question 18

avoid bias

Answer 18

importance of phrasing in an unbiased manner-reword

bias can also stem from context of survey-set clear frame of evaluation and reporting period

Question 19

make alternatives clear

Answer 19

closed-ended question such that the options are distinct and cover all possibilities

Question 20

What is mutually exclusive?

Answer 20

categories defined so that membership in one excludes the other (can’t belong to more than one category of the same type at one time)
ex. Catholic and Jewish

Question 21

What is exhaustive?

Answer 21

categories are defined so that all possible cases will fall into one of them
ex. graduate student, undergraduate and non degree student, other
other should be used with care- if other is popular answer-hard to interpret data

Question 22

Social Desirability Tendency

Answer 22

a characteristic of certain responses that causes people to choose that response even if it does not represent their true opinion

Question 23

how do you avoid social desirability?

Answer 23

word questions so that each alternative appears equally socially desirable
balance people’s natural tendency by including a another question that presents the issue is another way
ex. women should have the right to decide whether to continue a pregnancy
ex. abortion should be restricted by law

Question 24

What is a verification key and why is it used?

Answer 24

collection of items on a questionnaire designed to detect dishonest answers (and tendency to be influenced by SD) and avoid social desirability

Question 25

What is the lie scale and which scale uses this as verification key?

Answer 25

MMPI (Minnesota Multiphasic Personality Inventory)
stolen anything-most people have
hard to make yourself look better than you really are without scoring high on the lie scale
high score on the lie scale= tendency to give dishonest answers

Question 26

What is acquiescence?

Answer 26

tendency to respond in the same manner (agree with the item), no matter the content of the question
participants agree with both a statement and its opposite

Question 27

When does the problem of acquiescence arise?

Answer 27

during binary closed-ended questionnaires-highly susceptible to acquiescence bias

Question 28

Format of the item: Different types

Answer 28

binary closed ended: T/F, agree/disagree
MC
Ratings: visual analogue scale
likert scale

Question 29

Visual analogue scale:

Answer 29

question that asks for a response by marking a line between the min and the max value for the statement
a single line labeled at either end -

Question 30

Likert scale

Answer 30

question that asks for a rating of the extent of agreement or disagreement with a statement: rating scale
7 point/9 point scale

Question 31

When is it not appropriate to use the VAS or Likert?

Answer 31

creating questionnaires that concern sensation (ex. pain)
less valid results than Labeled Magnitude Scale (this takes into account past experiences-comparison of sensations between respondents)

Question 32

Branching items

Answer 32

a set of questions that enable he respondent to move through a survey in different ways and more efficiently and appropriately depending upon previous answers
ex. are you single- yes (skip to question 25)
no (skip to question 2)

Question 33

Sequence of items

Answer 33

answers of some questions may be biased if they were to follow others -introduce then detail

Question 34

Determining how the data will be analyzed

Answer 34

how will survey be scored/ interpreted
should be done in advanced (may not be the case for open ended questions)
if you think different categories will respond differently must include question to permit the classification of respondents on those dimensions

Question 35

What are the methods (4) of administrating a survey

Answer 35

face-face
telephone
internet
written response

Question 36

What are the types of sampling (4)

Answer 36

haphazard samples
purposive samples
convenience samples
probability samples

Question 37

Haphazard Sample

Answer 37

population subgroup for who selection the research uses hit/miss methods
on the street interviews-almost worthless

Question 38

purposive sample

Answer 38

a nonrandom sample that is chosen for some characteristic that it possesses
similar to a population ex. presidents of leading colleges

problem: error in judgement by researcher in choosing sample
this selected sample may not extend to entire population

Question 39

convenience sample

Answer 39

non random sample that is chosen for practical reasons
ex. students enrolled in psych intro courses
similar to purposeful but more convenient (location)

Question 40

probability samples

Answer 40

research knows the prob that any given individuals will appear in the sample
permits researcher to apply various statistics

Question 41

what are the types of probability samples?

Answer 41

systematic samples
simple random samples
stratified random samples
cluster samples

Question 42

random sample

Answer 42

sample in which every member of the population has an equal and independent chance of being selected

Question 43

Sampling Frame

Answer 43

population as it is defined for the purposes of selecting subjects for a study -may be different from actual population
ex. names on roster after an official date

Question 44

element

Answer 44

individual member of a sampling frame

Question 45

systematic samples

Answer 45

probability sample that is not randomly selected
ex. taking every nth individuals (non random if list has structure)
list in alphabetical order (essentially random if taken nth except in clusters)

Question 46

simple random samples

Answer 46

group chosen from an entire population such that every member of the pop has an equal and independent chance of being selected in a single sample (random number table)

Question 47

stratified random sample

Answer 47

random sample in which two or subsamples are represented according to some predetermine proportion generally in the same proportion as they exist in the pop

proportions matches the real proportions
separate random sample for each subgroup (males and females)

can oversample by including a greater frequency than it is represented in the pop

Question 48

cluster sample

Answer 48

group selected by using clusters/groupings from a larger pop
random is not feasible (too much to number)
can stratify the clusters

more similar than random but larger pop to offset

Question 49

More sophisticated form of cluster sampling

Answer 49

multistage sampling

Question 50

multistage sampling

Answer 50

form of cluster sampling in which clusters are further broken down by taking samples from each cluster

each stage introduces another source of sampling error

Question 51

randomized response method

Answer 51

survey technique that encourages honesty by introducing a random variable that makes it impossible to identify whether an answer it true of a particulate individual (coin toss for liar-no for heads and tails, truth teller- yes for head, no for tails)