Week 3

Question 1

What are the four major levels of measurement?

Answer 1

Nominal; ordinal; interval; ratio

Question 2

What are the two main indicators of the quality of measurement?

Answer 2

Reliability and validity

Question 3

Define ‘level of measurement’

Answer 3

Level of measurement describes the relationship between numerical values on a measure.

Question 4

Describe nominal level of measurement:

Answer 4

Measuring a variable by assigning a number arbitrarily in order to name it numerically so that is might be distringuished from other objects

Question 5

Explain ordinal level or measurement

Answer 5

Measuring a variable using ranking

Question 6

Explain interval level of measurement

Answer 6

Measuring a variable on a scale where the distance between numbers is interpretable

Question 7

Explain ration level of measurement

Answer 7

Measuring a variable on a scale where the distance between numbers is interpretable and there is an absolute zero value

Question 8

Why is level of measurement important?

Answer 8

1. It helps you decide how to interpret the data from the variable
2. It helps you decide what statistical analysis is appropriate on the values that were assigned.

Question 9

There are two criteria for evaluating the quality of measurement. Name both and explain them.

Answer 9

Reliability: the consistency of measurement
Validity: The accuracy with which a theoretical construct is translated into an actual measure.

Question 10

How can you infer the degree of reliability?

Answer 10

Does the observation provide the same results each time?

Question 11

Explain true score theory:

Answer 11

True score theory maintains that every observable score is the sum of two components: the true ability of the respondent on that measure; and random error

Question 12

What’s a ‘true score’?

Answer 12

Essentially the score that a person would have received if the score were pretty accurate

Question 13

Why is true score theory important?

Answer 13

1. It is a simple yet powerful model for measurement
2. It is the foundation of reliability theory
3. It can be used in computer simulations as the basis for generating observed scored with certain known properties.

Question 14

What if some errors are not random, but systematic.

Answer 14

One way to deal with this is to revise the simple true score model by dividing the error component into two subcomponents, random error and systematic error

Question 15

What is ‘random error’?

Answer 15

Random error is a component or part of the value of a measure that varies entirely by chance.

Question 16

What is ‘systematic error’?

Answer 16

Systematic error is a component of an observed score that consistently affects the response in the distribution.

Question 17

What’s the difference between random error and systematic error?

Answer 17

Unlike random error, systematic errors tend to be either positive or negative consistently; because of this, systematic error is sometimes considered to be bias in measurement

Question 18

How can you reduce measurement errors?

Answer 18

1. Pilot test your instruments and get feedback from respondents
2. Train the interviewers or observers thoroughly
3. Double-check the data for your study thoroughly
4. Use statistical procedures to adjust for measurement error
5. Use multiple measures of the same construct

Question 19

Name and explain the four types of reliability

Answer 19

1. Inter-rater or inter-observer reliability is used to assess the degree to which different raters/ observers give consistent estimates of the same phenomenon
2. Test-retest reliability is used to assess the consistency of an observation from one time to another
   3.Parallel-forms reliability is used to assess the consistency of the results of two tests constructed in the same way from the same content domain
   4.Internal consistency reliability is used to assess the consistency of results across items within a test.

Question 20

Explain Cohen’s Kappa

Answer 20

Cohen’s Kappa, a statistical estimate of inter-rater reliability that is more robust than percent agreement because it adjusts for the probability that some agreement is due to random chance, was introduced to avoid the problem

Question 21

Explain Cronbach’s Alpha

Answer 21

Cronbach’s Alpha takes all possible split halves into account. So Cronbach’s Alpha is mathematically equivalent to the average of all possible split-half etimates

Question 22

There are 4 different internal consistency measures. Name them and explain them shortly

Answer 22

• Average inter-item correlation uses all of the items on your instrument that are designed to measure the same construct.
• The average item-total correlation involves computing a total score across the set of items on a measure and treating that total score as though it were another item, thereby obtaining all of the item-to-total score correlations.
• In split half reliability, you randomly divide into two sets all items that measure the same construct. It’s an estimate of internal consistency reliability that uses the correlation between the total score of two randomly selected halves of the same multi-item test or measure.
• Cronbach’s Alpha

Question 23

Define Construct validity:

Answer 23

Overarching category of validity that contributes to the quality of measurement, with all of the other measurement validity labels falling beneath it.

Construct validity is an assessment of how well your actual programs or measures reflect your ideas or theories.

Construct validity is all about representation, and it can be viewes as a truth in labeling issues.

Question 24

There are various validity types, name them.

Answer 24

• Construct validity
• Translation validity
• Face validity
• Content validity
• Criterion-related validity
• Predictive validity
• Concurrent validity
• Convergent validity and discriminant validity

Question 25

Shortly explain construct validity

Answer 25

The approximate truth of the conclusion or inference that your operationalization accurately reflects its construct

Question 26

Shortly explain translation validity

Answer 26

Focuses on whether the operationalization is a good translation of the constrcut

Question 27

Shortly explain face validity

Answer 27

Face validity is a validity that checks that on its face the operationalization seems like a good translation of the construct.

Question 28

Explain content validity

Answer 28

In content validity, you check the operationalization against relevant content domain for the construct.

Question 29

Explain the criterion-related validty

Answer 29

Examines whether the operationalization or the implementation of the construct performs the way it should according to some criterion

Question 30

Explain predictive validity

Answer 30

Based on the idea that your measure is able to predict what is theoretically should be able to predict

Question 31

Explain concurrent validity

Answer 31

About an operationalization’s ability to distinguish between groups that it should theorec