Research and design quiz #8 Flashcards
what is the first (1st) rule in questionnaire construction?
Rule #1 – Start with a list of all the items one is
interested in knowing about the group, concept,
or phenomenon
* List what you want to know
* List what you need to know
* Think about each item
* Think about the information you would need
for each item
* Be sure to identify what pertinent
independent, dependent, and control
variables should be included in the
questionnaire
what is the second (2nd) rule in questionnaire construction?
Rule #2 – Be prepared to establish validity and reliability
* Validity the questionnaire is measuring what it claims to measure
* Face – the questions are believed to be measuring some
concept or construct
* Content – whether the items or questions are representative of
possible items
* Criterion – whether the scores relate to some external
standard, such as scores on a similar instrument
* Construct – whether the items/questions measure what they
intend to measure
* Reliability over time, the questionnaire yields consistent results
* Stability – respondents provide same answers to same
questions on a second testing
* Consistency – questions are strongly related and measure the
same concept
* Reliability can be tested in different ways
* Test-retest distribute the questionnaire/instrument
twice/multiple times to same population to obtain
potentially consistent results (preferred method)
* Split-half divide the questionnaire into halves or
sections and administer to same or similar groups to
obtain potentially consistent results
* Multiple forms use various versions of the same
instrument to determine whether responses/data produce
consistent results
Rule
#2
what is the third (3rd) rule in questionnaire construction?
Rule #3 – the wording in the questionnaire must
be appropriate for the target audience
* DO NOT ask college level questions of 6th
grade elementary students, for example
* DO NOT use words or phrases that the
respondents are not likely familiar with
* DO write questions that are clearly worded
* DO write questions that are concise
* DO write questions that the target audience
will understand
* Failure to comply with this rule can have
profound effects on the utility of the data you
collect
What is the fourth (4th) rule in questionnaire construction?
Rule #4 – be sure that it is clearly identifiable who
should answer questions or the survey generally
* Provide a statement advising who should
complete the survey or specific questions
* You do not want, for example, a child answering
survey questions that the parent or homeowner
is intended to answer
* It might be wise to provide a specific question
which asks the desired respondent to continue
answering questions, and if they are not the
targeted respondent, to discontinue answering
and exit
* this could be achieved in digital surveys by
having the survey automatically stop after
someone provides a particular response to
a question
* Failure to ensure this rule is followed can
compromise survey reliability
what is the fifth (5th) rule in questionnaire construction?
Rule #5 – avoid asking any questions that are biased,
leading, threatening, or double-barreled in nature
* Questions or statements that confuse respondents
can result in ambiguous responses
* Questions that guide the respondent can generate
false responses
* The structure of questions can potentially influence
responses to later questions in the survey, so be
cognizant of where on the survey things are, how they
are ordered, and how they are asked
* Some bad survey questions examples include
* Does it not feel great to get high?
* How often do you get high and do you enjoy it?
* Do you cheat on exams and if you do, do you
know you are only cheating yourself?
* REMEMBER! Garbage in, garbage out. If you choose to
make a survey or questionnaire, avoid erroneous
questions like those above
What is the sixth (6th) rule in questionnaire construction?
Rule #6 – before construction, a decision must be made whether to
use open- or closed-ended questions, or a combination
* Open-ended questions allows respondent to provide a free
response
* “how do you feel about the police in your neighborhood?”
* Open-ended questions make data analyses more difficult, are
time consuming, require subjective coding practices (if you
are trying to quantify), and may produce responses that are
of little value to your research objective
* However, they still provide considerably more depth than
other types of questions, which can be very valuable and
insightful
* Closed-ended questions restricts respondent answers to a
dichotomy or a few categories
* “the police provide positive support in my neighborhood”
Yes, No
* “How likely is it that you will call the police if you see a crime
being committed in your neighborhood” 1 = very unlikely 4 =
very likely
* Closed-ended questions can provide data that are more
readily analyzable
What is the seventh (7th) rule in questionnaire construction?
Rule #7 – consider that the respondents may not
have all the general information needed to
complete the questionnaire
* Do not make assumptions about what
respondents have or do not have
* Do not make assumptions about what
respondents have or have not experienced
* To avoid issues where respondents cannot
answer the questions you are asking, simply
provide an “escape” response option in the
choices:
* “other”
* “unknown”
* “no prior knowledge”
What is the eighth (8th) rule in questionnaire construction?
Rule #8 – whenever possible, pretest the
questionnaire before it is officially used
* It is good research practice to employ this
rule whenever you can
* After completing the first draft of the
questionnaire, test it out on a practice
sample or group
* Doing this pretest can help identify potential
errors, response pitfalls, bad questions,
flawed language, and poor design
What is the ninth (9th) rule in questionnaire construction?
Rule #9 – set up questions so that the responses
are easily recognizable whether it is self-
administered or an interview
* Avoid questions that leave respondents
confused about how they should/can
respond
* Give crystal clear instructions
* Establish recognizable ways to respond
* Keep your survey/questionnaire format
simple
* Make the survey legible (avoid small print)
What is the tenth (10th) rule in questionnaire construction
Rule #10 – the questionnaire should be organized
in a concise manner that keeps the interest of the
respondent, encouraging them to complete the
entire questionnaire
* Survey fatigue (the struggle is real bro)
* You are likely to encounter serious attrition
problems (and therefore missing data) if
your survey . . .
* has too many questions
* Is disorganized
* Is formatted in a confused manner
* Is not clear, concise, and easy-to-read
* As a general recommendation, try to keep your
survey at two pages
* Of course, there are exceptions to this and
depends on what the research is attempting
to accomplish
Scales are important for three primary reasons
Allow collapsing of several variables into a single variable
producing a representative value (aggregation tool)
- Offers measures that are quantifiable and precise,
allowing for statistical analyses - Increases measurement reliability
Principle of __________ holds that items making up
the scale need to represent one dimension befitting a
continuum that is supposed to be reflective of only one
concept
- Ex: if you are measuring job satisfaction, the scale should
not also be capable of measuring job stress
Unidimensionality
DEFINITION: ______ ________ are a means
of assisting in making the necessary and
proper connections between phenomena
and numbers for analytical purposes
Scaling procedures
Broadly, there are two types of scaling
procedures
* ________ scales – designed to measure what the
researcher believes it is measuring and is based
on face validity and professional judgement
* This type of scale does provide a viable starting point
for exploratory research, but it is limited
- _________ scales – designed to measure
respondent perceptions, beliefs, values, or
experiences using scaled questions
Arbitrary
Attitudinal
Three types of attitudinal scales:
T_____ne,
Lik__t,
Gu____an
Three types,
Thurstone,
Likert,
Guttman
