L6 - Survey design and sampling Flashcards
Quantitative research method definition
Research techniques seeking for quantify data and typically applying some form of statistical analysis.
Difference between quantitative and qualitative methods:
Quantitative methods are descriptive research methods that answer the questions of “what” is happening for a particular phenomenon.
Meanwhile, qualitative explanatory research answers the questions of “why” and “how”.
Quantitative research methods include
Experiments, Questionnaires, and Surveys. (Technique: Observation, research, physical measurements, questionnaires, etc.)
Questionnaire
Has fixed set of answers and is most commonly administered as part of surveys.
7 steps before conducting survey
1) Develop question topics
2) Select question and response formats
3) Select wording
4) Determine sequence
5) Design layout and appearance
6) Pilot test
7) Undertake survey
How to motivate answering when designing a questionnaire
Simple, easy to understand questions to motivate complete, honest and accurate answers.
Misleading scale formats
Double-barrelled questions, leading questions, ambiguous questions, complex questions, double negative questions.
Double-barrelled question
Include two or more different attributes or issues in the same question, but responses allow only a single issue.
(Two or more questions are combined into one)
Leading question
Introduce bias and influence the way respondent answer a question.
» Clue to what respondents should answer.
Loaded question
Suggest a socially desirable answer or involve an emotionally charged issue.
Ambiguous question
A situation in which possible responses can be interpreted in many ways.
» Respondent find it difficult to remember very specific information of the question.
Complex question
A situation in which question is worded that respondent isn’t sure how they are supposed to respond.
» Solution: Defining the issue through the 5 Ws (who, what, when, where, why)
Double negative question
2 negatively signed statements that lead towards a positive direction.
» Contain two negative thoughts in the same question.
Optimal number of response in rating scales
- Rating scale with few response categories yield the least reliable score and perform the worst.
- 10, 9, 7 are the most preferred.
- Different scales may be best suited to different purposes: 2, 3, 4 can be quick, easy to use and reduce demotivation.
Two tasks in measurement process
(1) construct selection/development and (2) scale measurement.
Construct definition (Hair et al., 2017)
A hypothetical variable made up of a set of component responses or behaviour that are thought to be related.
Ex of marketing construct: Customer satisfaction, Brand loyalty, Service quality, Advertising recall.
Construct development definition (Hair et al., 2017)
Construct development: the process in which researchers identify characteristics that define concept being studied by the researcher.
Scale measurement definition
The process of assigning a set of scale descriptors to represent the range of possible responses to a question about an object or construct.
Evaluating the measurement scales based on two criteria:
Reliability: the extent to which the scale can reproduce the same or similar measurement results in repeated trials. It is a measure of consistency.
Validity: the extent to which the scale measures what is supposed to measure.
Three basic scale levels:
Nominal, Ordinal and Interval/Ratio scale
Nominal variable
Variables have no logical order or value.
> Equivalence
Ordinal variable
Variables have a logical order to the value labels.
> Equivalence & Order
Ratio variable
Actual value is measured directly.
> Equivalence, Order & Equal intervals.
Criteria for scale development
1) Understanding of the questions: to consider the intellectual capacity and language ability of respondents
2) Discriminatory power of scale descriptors: the scale’s ability to differentiate between the scale responses. The more scale points, the greater discriminatory power, the greater the variability in the data.
3) Types: balanced / unbalanced scale; forced / non-forced choice scales; negatively worded statements; desired measures of central tendency and dispersion.
4) Adapting established scales.
Types of scales to measure attitudes and behaviour
Likert scale, Semantic differential scale, and Behavioural intention scale
Likert scale
Agree / disagree with a series of mental or behavioural belief statements about a given objects. It is often expanded to the format of 7-point scale. It is best used for self-administered surveys, personal interviews or online surveys.
Semantic differential scale
Ask about the attitude regarding the bipolar adjectives. It is used to develop and compare profiles of different brands, or indicate how an ideal product would rate.
Behavioural intention scale
Ask the likelihood that people will demonstrate some type of predictable behaviour intent toward purchasing an object in a future time frame.
Comparative and non-comparative scales
1) Comparative rating scale: when the objective is to have a respondent express her attitudes, feelings, behaviour about an object or its attributes.
2) Non-comparative rating scale: (…) without making reference to other object or its attributes.
Dichotomous scale
Scales that provide only 2 possible outcomes (ex: Yes/No)
Open-ended questions
Allow respondents to provide a non-numerical answer and instead express themselves through words. It represents the qualitative element of surveys and qualitative analysis.
Pilot study / test
A preliminary small scale study conducted before the main data collection stage.
It is done to test the first draft of the questionnaire in order to spot potential issues like long or difficult to understand questions, and spelling mistakes.
Sampling
Selection of small number of elements from a larger defined target group to conclude about a larger group.
> It is often used when it is impossible or unreasonable to conduct a census (every member of target population).
Role of sampling
- Less time-consuming and costly than conducting a census.
- Influence the type of research design, survey instrument and actual questionnaire.
Factors underlying sampling theory
The central limit theorem (CLT) - the sampling distribution derived from a simple random sample will be approximately normally distributed.
Tools used to assess the quality of samples
Sampling error and Non-sampling error
Sampling error definition
any bias type that is attributable to mistakes in selection process for prospective sampling units or in determining the sample size. It can be reduced by increasing the sample size.
Non-sampling error definition
Regardless of whether a sample or census is used, it occurs at any stage of the research process. For example, inappropriate question/scale measurements or poorly designed questionnaire.
Differences between probability and non-probability sampling
1) Probability:
- Specific selection rules to ensure unbiased selection and proper sample representation of the population => Can judge the data reliability and validity.
- The observed difference can be partially attributed to the existence of sampling error. The results can be generalized to the target population with a specified margin of error.
2) Non-probability: Selection is based on intuitive judgement or researcher knowledge. Sample error is unknown.
Probability sampling include 3 types:
Simple random sampling; Systematic random sampling; and Stratified random sampling
Non-probability sampling include 4 types:
Convenience sampling; Judgment sampling; Quota sampling; and Snowball sampling
Simple random sampling
A known and equal chance of being selected.
Systematic random sampling
Known and equal chance of being selected, but the target population is ordered in some way, and selected systematically > Frequently used
Stratified random sampling
Separate the target population into different homogenous groups (strata), then select samples from each stratum.
Convenience sampling
Draw samples based on convenience (people who are the most available or most easily selected).
Judgment sampling
Select samples based on an experienced individual’s belief that they will meet the requirements.
Quota sampling
Select samples based on prespecified quotas regarding demographics, attitudes, behaviour, etc.
Snowball sampling
Respondents are chosen and help researcher identify additional people to be included in the study.
Factors to consider when determining sample size
- Probability sample sizes: variance, level of confidence, desired error.
- Non-probability sample sizes: specific characteristics of sample like age, income, education.
- Other sample size determination approaches
Questionnaire can be failed due to:
- Incomplete with missing values
- Pattern of responses indicate participant did not understand or follow the instructions
- Received after pre-established cut-off date
- Answered by unqualified person.
Verifying the questionnaire by several methods
- Discarding unsatisfactory participants
- Assigning missing values
- Returning to the field
Pros and Cons of simple random sampling
- Pros: (1) Easily understood. (2) Can be generalized to the defined target population with a prespecified m.e. (3) Unbiased estimates of population’s characteristics.
- Cons: Difficulty in obtaining a complete and accurate listing of the target population elements.
Pros and Cons of Systematic random sampling
- Pros: Less costly and easier than SRS due to availability of lists and shorter time to draw a sample.
- Cons: (1) of hidden patterns in the list that create bias. (2) The number of sampling units must be known
Pros and Cons of Stratified random sampling
- Pros: (1) Assure the representativeness of the sample. (2) Study each stratum and compare between strata. (3) Estimate population with greater precision and less error.
- Cons: Difficult to determine the basis for stratifying (target population’s characteristics of interests), unavailable required information to select criteria as factors to stratify.
Pros and Cons of Convenience random sampling
- Pros: Able to interview many respondents in a relatively short time. > Early stages of research
- Cons: (1) Unreliable constructs if used to study a broader target population (2) Ungeneralized data.
Pros and Cons of Judgement sampling
- Pros: Correct judgment will generate better than convenience sampling.
- Cons: (1) Ungeneralized data. (2) Data should be interpreted cautiously.
Pros and Cons of Quota sampling
- Pros: (1) Contain specific subgroups in the proportions desired by researchers. (2) Reduce selection bias by the field workers.
- Cons: (1) Ungeneralized data. (2) The success depends on subjective decisions made by researchers.
Pros and Cons of Snowball sampling
- Pros: (1) Able to identify small, hard-to-reach, uniquely defined target populations. (2) Most useful in qualitative research.
- Cons: 1) Ungeneralized data. (2) Allow bias in the study
Pros of Survey method
(1) Accommodate large sample sizes so results can be generalized
(2) Produce precise enough estimates to identify even small differences
(3) Easy to administer and record answers to structured questions
(4) Facilitate advanced statistical analysis
(5) Concepts and relationships not directly measurable can be studied
Cons of Survey method
(1) Difficult to develop questions that accurately measure respondent attitudes and behavior.
(2) Difficult to obtain In-depth data.
(3) Low response rates can be a problem.
Two methods of stratified random sampling
- Proportionately stratified sampling: the sample size from each stratum is dependent on that stratum’s size relative to the defined target population.
- Disproportionately stratified sampling: the sample size selected from each stratum is independent of that stratum’s proportion of the total defined target population.
