Scale Reliability

Question 1

What is Reliability?

Answer 1

• Degree to which the measurement is free from measurement error
• The extent to which the differences in respondents’ test scores are a function of their true differences, as opposed to measurement error.

Question 2

How is Reliability determined?

Answer 2

It is on a continuum where we measure the degree to which scores from a measure are or are not reliable

Question 3

What is Measurement Error

Answer 3

Random noise that creates inconsistency between
observed and true scores
- There is no perfectly reliable measure

Question 4

What does it mean for a test to have “strong Reliability?”

Answer 4

Strong signal and/or little noise (less error)
ex. Severe or constant pain, large drops in BP, Severe tremor

Question 5

How does Reliability deal with proportion of variance?

Answer 5

Tied to differences among people
- A heterogenous sample increases reliability
- Coefficient of reliability is a 0-1 score

Question 6

What does a coefficient of 1 for reliability mean?

Answer 6

True score variance is equal to observed score variance
- No measurement error of observed scores (this is not possible)

Question 7

What does a coefficient of 0 for reliability mean?

Answer 7

Means there is no variance (everyone has the same true score)
- Occurs when respondents do not differ

Question 8

Proportion of Variance

Answer 8

Reliability is the proportion of observed score variance that is attributable to true score variance
- Reliability is the lack of error variance

Question 9

Correlations in regards to Reliability

Answer 9

• Reliability is the correlation between observed scores and true scores
• Reliability is the lack of correlation between observed scores and error scores

Question 10

Types of Reliability

Answer 10

• Internal Consistency: Different set of items from same PRO
• Test- Restest: Over time
• Interrater: By different people on the same occasion
• Intra- rater: By the same persons on different occasions

Question 11

Assumption of Test-Retest Reliability

Answer 11

True scores remain stable across the test-retest interval

Question 12

Internal Consistency Reliability

Answer 12

• Requires only one test to be completed at one point in time
• Estimates reliability of multiple-item test
• Most widely used method for estimating reliability

Question 13

What is the most widely used method of estimating Internal consistency reliability?

Answer 13

Cronbach’s Alpha

Question 14

What are two facts that affect internal consistency reliability?

Answer 14

• How much observed differences on one part of an assignment are consistent with observed differences on other parts of the test
• Test Length (long tests will produce more reliable scores than a short one)

Question 15

How are confidence intervals effected by poor reliability?

Answer 15

They are less precise and wider compared

Question 16

What is Variability?

Answer 16

Variability of observed scores is larger than variability among true scores
- Noise induced by error