Measurment and survey research: ch5&7

Question 1

Level of Measurement

Answer 1

The relationship between numerical values on a measure. There are different types of levels of measurement (nominal, ordinal, interval, ratio) that determine how you can treat the measure when analyzing it. For instance,
it makes sense to compute an average of an interval or ratio variable but does not for a nominal or ordinal one.

Question 2

Nominal Level of Measurement Measuring

Answer 2

A variable by assigning a number arbitrarily in order to name it numerically so that it might be distinguished from other objects. The jersey numbers in most sports are measured at a nominal level.

Question 3

Ordinal Level of Measurement

Answer 3

Measuring a variable using rankings. class rank is a variable measured at an ordinal level.

Question 4

Interval level of Measurement

Answer 4

Measuring a variable on a scale where the distance between numbers is interpretable. For instance, temperature in Fahrenheit or celsius is measured on an interval level.

Question 5

Ratio Level of Measurement

Answer 5

Measuring a variable on a scale where the distance between numbers is interpretable and there is an absolute zero value. For example, weight is a ratio measurement.

Question 6

A theory that maintains that an observed score is the sum of two components: true ability (or the true level) of the respondent; and random error.

Answer 6

True Score Theory

Question 7

Random Error

Answer 7

A component or part of the value
of a measure that varies entirely by chance. Random error adds noise to a measure and obscures the true value.

Question 8

systematic error

Answer 8

A component of an observed score that consistently affects the responses in the distribution.

Question 9

Triangulate

Answer 9

combining multiple independent measures to get at a more accurate estimate of a variable.

Question 10

inter-rater or inter-observer Reliability

Answer 10

The degree of agreement or correlation between the ratings or coding’s of two independent raters or observers of the same phenomenon.

Question 11

Test-retest reliability

Answer 11

The correlation between scores on the same test or measure at two successive time points.

Question 12

Parallel-forms reliability

Answer 12

The correlation between two versions of the same test or measure that were constructed in the same way, usually by randomly selecting items from a common test question pool.

Question 13

internal consistency reliability

Answer 13

A correlation that assesses the degree to which items on the same multi-item instrument are interrelated. The most common forms of internal consistency reliability are the average inter-item correlation, the average item-total correlation, the split half correlation and cronbach’s Alpha.

Question 14

Cohen’s Kappa

Answer 14

A statistical estimate of inter-rater agreement or reliability that is more robust than percent agreement because it adjusts for the probability that some agreement is due to random chance.

Question 15

Average inter-item correlation

Answer 15

An estimate of internal consistency reliability that uses the average of the correlations of all pairs of items.

Question 16

Average item-total correlation

Answer 16

An estimate of internal consistency reliability where you first create a total score across all items and then compute the correlation of each item with the total. The average inter-item correlation is the average of those individual item-total correlations.

Question 17

Split half reliability

Answer 17

An estimate of internal consistency reliability that uses the correlation between the total score of two randomly selected halves of the same multi-item test or measure.

Question 18

Cronbach’s Alpha

Answer 18

one specific method of estimating the internal consistency reliability of a measure. Although not calculated in this manner, cronbach’s Alpha can be thought of as analogous to the average of all possible split-half correlation.

Question 19

Translation validity

Answer 19

A type of construct validity related to how well you translated the idea of your measure into its operationalization.

Question 20

criterion-related validity

Answer 20

The validation of a measure based on
its relationship to another independent measure as predicted by your theory of how the measures should behave.

Question 21

Face validity

Answer 21

A validity that checks that “on its face” the operationalization seems like a good translation of the construct.

Question 22

Content validity

Answer 22

A check of the operationalization against the relevant content domain for the construct.

Question 23

concurrent validity

Answer 23

An operationalization’s ability to distinguish between groups that it should theoretically be able to distinguish between.

Question 24

Convergent validity

Answer 24

The degree to which the operationalization is similar to (converges on) other operationalizations to which it should be theoretically similar.

Question 25

Discriminant validity

Answer 25

The degree to which the operationalization is not similar to (or diverges from) other operationalizations that it theoretically should not be similar to.

Question 26

threats to construct validity

Answer 26

Any factor that causes you to make an incorrect conclusion about whether your operationalized variables (e.g., your program or outcome) reflect well the constructs they are intended to represent.

Question 27

mono-operation bias

Answer 27

A threat to construct validity that occurs when you rely on only a single implementation of your independent variable, cause, program, or treatment in your study.

Question 28

mono-method bias

Answer 28

A threat to construct validity that occurs because you use only a single method of measurement.

Question 29

hypothesis guessing

Answer 29

A threat to construct validity and a source of bias in which participants in a study guess the purpose of the study and adjust their responses based on that.

Question 30

dichotomous response format

Answer 30

A question response format that allows the respondent to choose between only two possible responses.

Question 31

nominal response format

Answer 31

A response format that has a number beside each choice where the number has no meaning except as a placeholder for that response.

Question 32

Ordinal response format

Answer 32

A response format in which respondents are asked to rank the possible answers in order of preference.

Question 33

interval level response format

Answer 33

A response measured using numbers spaced at equal intervals where the size of the interval between potential response values is meaningful. An example would be a 1-to-5 response scale.

Question 34

Likert type response scale

Answer 34

A response format where responses are gathered using numbers spaced at equal intervals.

Question 35

filter or contingency question

Answer 35

A question you ask the respondents to determine whether they are qualified or experienced enough to answer a subsequent one.

Question 36

double barreled question

Answer 36

A question in a survey that asks about two issues but only allows the respondent a single answer. For instance, the question “What do you think of proposed changes in benefits and hours in your workplace?” asks simultaneously about two issues but treats it as though they are one.

Question 37

response bracket

Answer 37

A question response format that includes groups of answers, such as between 30 and 40 years old, or between $50,000 and $f 00,000 annual income.

Question 38

response format

Answer 38

The format you use to collect the answer from the respondent.

Question 39

structured response format

Answer 39

Provides a specific format for the respondent to choose their answer. For example, a checkbox question lists all of the possible responses.

Question 40

multi-option or multiple response variable

Answer 40

A question format in which the respondent can pick multiple variables from a list.

Question 41

single option variable

Answer 41

A question response list from which the respondent can check only one response.

Question 42

unstructured response format

Answer 42

a response format that is not predetermined and that allows the respondent or interviewer to determine how to respond. An open-ended question is a type of unstructured response format.