Chapter 9 - Constructing & Administering Flashcards
Surveys
- Focus on group outcomes.
- Results are presented at the group level
- Surveys are scored by the percentage of respondents who selected each answer.
Psychological Tests
- Focus on individual outcomes.
- The results are reported at the individual outcome.
- Reported in terms of overall derived scores or scaled scores.
- Sample the behaviours thought to measure an attribute or thought to predict an outcome.
Survey Research Firms
- Companies that specialize in the construction and administration of surveys and analysis of data.
Survey Researchers
- People who design and conduct surveys and analyze their results.
What are good characteristics of surveys?
- Specific and measurable objects
- Straightforward questions
- Pretested
- Good sample
- Appropriate analysis
- Accurate reporting of results
- Reliability and Validity
Experimental research techniques vs descriptive research techniques
- Experimental: Help determine cause and effect.
- Descriptive: Help describe a situation or phenomenon.
What research technique are surveys typically used for?
- Descriptive Research Techniques
What are Helmstadter’s Six Methods for Acquiring Knowledge?
- Tenacity: Based on something we already believe.
- Intuition: Without any reasoning or inferring.
- Authority: From a highly respected source.
- Rationalism: Through reasoning.
- Empiricism: Through personal experience.
- Scientific Method: By using the scientific method.
What are the five steps associated with the scientific method while constructing a survey?
- Identifying - Presurvey issues
- Designing - Construct the survey
- Conducting - Administer the survey
- Analyzing - Analyze data
- Communicating - Communicate the findings
What’s the first step in preparing a survey?
- Planning the survey objectives
- The purpose of the survey and what the survey will measure.
What is the second step in preparing a survey?
- Establish operational definitions
- Specific behaviours that represent the purpose
What is the third step in preparing a survey?
- Constructing a plan
- Includes a list of all the phases and steps necessary, a cost estimate, survey’s development and administration, analysis of data, a timeline
What is involved in the second phase of creating a survey?
- The construction of a survey
- Involves writing survey questions, preparing the survey instrument, and pretesting the survey
Self-Administered Surveys
- Those that individuals complete themselves.
Mail Surveys
- Mailed to respondents with instructions for completing and returning them.
Individually Administered Surveys
- Administered by a facilitator in person for respondents to complete in the presence of the facilitator.
Personal Interviews
- Surveys that involve direct contact between the survey researcher and the respondents in person or by phone.
- Can include face-to-face surveys or telephone surveys.
Face-to-Face Surveys
- The interviewer asks a series of questions in respondents’ homes, a public place, or the researcher’s office.
Telephone Surveys
- Interviewer calls respondents and asks them questions over the phone.
Structured Record Reviews
- Forms that guide data collection from existing records.
Structured Observations
- Forms that guide an observer in collecting behavioural information.
What are the stages people go through when answering survey questions?
- Comprehension
- Retrieval
- Judgment
- Response Communication
Qualities of good survey questions
- Purposeful and straightforward
- Unambiguous
- Correct syntax
- Appropriate response options
- Appropriate categorical alternatives
- Written at a comfortable reading level
-
Double-Barreled Questions
- A question that is actually asking two or more questions in one.
What happens after we write our survey questions?
- We prepare the survey instrument
- Create the survey so people want to complete it.
What should the front page of a survey include?
- State the purpose of the survey
- Explain who is sponsoring or conducting the survey
- Explain why it is important for respondents to complete the survey to the best of their ability
- Explain how to complete the form
- Assure confidentiality
- Thank them for their cooperation
What should a survey include?
- Appeal and instructions
- Headings and subheadings
- Transitions
- Response instructions
- Bold typeface
- Justification of response spaces
- Shading/White spaces
- Good printing/Font
Nonsampling Measurement Errors
- Errors associated with the design and administration of the survey
What does pretesting allow us to do?
- Identify sources of error
- Examine the effectiveness of the questions/survey
- Examine the effect of alternate versions of a question
- Assess the final version of a survey for understanding, time, and ease
- Obtain data and make format changes to make analysis more efficient
Focus Group
- Bring together people who are similar to the target respondents in order to discuss issues related to the survey.
Field Test
- An administration of the survey or test to a large representative group of individuals to identify problems with administration, item interpretation, and so on.
Split-Sample Tests
- This method involves field-testing two or more versions of a question, sets of questions, or surveys.
- Its objective is to find which version gives the most accurate results.
Item Nonresponse Rate
- How often an item or question is not answered.
What is the third step in administering a survey?
- Actually administering the survey.
Probability Sampling
- A type of sampling that uses statistics to ensure that a sample is representative of a population.
Simple Random Sampling
- Every member of a population has an equal chance of being chosen as a member of the sample.
Systematic Sampling
- A variation of simple random sampling.
- Every -nth (e.g., every fifth) person in a population is chosen as a member of the sample.
Stratified Random Sampling
- Population is divided into subgroups or strata.
- A random sample is selected from each stratum.
- The strata should be based on some evidence that they are related to the issue or problem the survey addresses.
Cluster Sampling
- Used when it is not feasible to list all of the individuals who belong to a particular population and is a method often used with surveys that have a large target population.
Nonprobability Sampling
- A type of sampling in which not everyone has an equal chance of being selected from the population.
- Often used because they are convenient and less expensive than probability sampling.
Convenience Sampling
- The survey researcher uses any available group of participants to represent the population.
The Homogeneity of the Population
- How similar the people in your population are to one another.
What is the last step of the third phase of creating a survey?
- Distributing the survey.
What is the fourth phase of survey development?
- Involves coding and entering the survey data into computer software for analysis.
Database
- A matrix in the form of a spreadsheet that shows the responses given by each participant and for each question in the survey.
Response Rate
- The number of individuals who responded to the survey divided by the total number of individuals to whom the survey was sent.
Univariate Analyses
- Computation of statistics that summarize individual question responses.
Bivariate Analyses
- Provide information on two variables or groups.
Multivariate Analyses
- Provide information on three or more variables or groups.
What are the steps to survey development?
- Preparing the survey
- Pre-testing the survey
- Administering the survey
- Coding, entering, and analyzing data
What are the three sets of instructions?
- For the person administering the test
- For the person taking the test
- For the person scoring/interpreting the results
Testing Environment
- The circumstances under which the test is administered that affect how one may respond.
