Survey Design

Question 1

operationalization

Answer 1

turning abstract concepts into measurable observations; for ex. - the concept of social anxiety can’t be directly measured but it can be operationalized through self-rating scores on a social anxiety scale

Question 2

construct

Answer 2

the abstract idea, underlying theme or subject matter that someone wishes to measure using survey questions; for ex. - skill, attribute, or ability based on one or more established theories; exist in the human brain and are not directly observable

Question 3

conceptual definition

Answer 3

tells you what the concept means and what your constructs are by explaining how they are related to other constructs

Question 4

categorical variable vs. continuous variable

Answer 4

categorical - variables that contain a finite number of categories or distinct groups

continuous - variables that are numeric that have an infinite number of values between any two values (for ex. - the length of a part or the date and time a payment is received)

Question 5

scoring open responses

Answer 5

create categories and subcategories to provide richer detail; code to make quantitative

Question 6

coding system/plan

Answer 6

assigning data to categories; allows you to turn qualitative data into numerical (quantitative) data; used to group responses together

Question 7

Kappa (inter-rater reliability)

Answer 7

aka Cohen’s Kappa coefficient; used to measure inter-rater reliability for qualitative (categorical) items; Kappa of 1 indicates perfect agreement, kappa of 0 indicates agreement equivalent to chance; limitation - kappa is affected by the prevalence of the finding under observation

Question 8

Cronbach’s alpha

Answer 8

most common measure of internal consistency (reliability); when you have multiple Likert questions in a survey/questionnaire that form a scale and you wish to determine if the scale is reliable; a measure of internal consistency (how closely related a set of items are as a group); measure of “scale reliability”; a “high” alpha value for alpha does not imply that the measure is unidimensional; rule of thumb: .70 and above is good (dependent/reliable), .80 and above is better, and .90 and above is best

Question 9

populations of interest

Answer 9

the population/group from which a researcher tries to draw conclusions; subset of the general population that the surveyor wants to know more about; many research studies require specific groups of interest to make decisions based on their findings

Question 10

value of a survey

Answer 10

tool for collecting info that can describe the characteristics of a large population; high representativeness - provide a high level of general capability in representing a large population; little to no observer subjectivity; great for gathering qualitative feedback; more direct than interpreting usage data; lower cost than other methods; wide range of possibilities and quick/easy to create

Question 11

drawbacks of a survey

Answer 11

respondents may not feel encouraged to provide accurate, honest answers; respondents may not feel comfortable providing answers that present themselves in an unfavorable manner; survey fatigue could lead to response biases, wrong questioning or wording can lead to inaccurate data; could have unreliable respondents