Lecture 3 - Measurement error and uncertainty

Question 1

TRUE or FLASE

Measurement error is basically synonymous with reliability.

Answer 1

TRUE

Question 2

TRUE or FLASE

Measurement error refers to the systematic fluctuations in test scores that follow a Gaussian distribution

Answer 2

FALSE

It’s about RANDOM fluctuations, which means they would “average out” over the long term if you could take enough measurements

Question 3

What is the most optimal form of reliability coefficient?

Answer 3

Test-retest coefficients

Question 4

TRUE or FALSE

Cronbach’s alpha is a single administration reliability coefficient

Answer 4

TRUE

Which is one reason it is not favoured, relative to test-retest coefficients

Question 5

TRUE or FALSE

Most statistical methods assume a level of measurement error

Answer 5

FALSE

Most assume zero measurement error

Question 6

What TWO factors does the predicted true score consider to improve the accuracy of the observed score?

Answer 6

1. the measurement error of the instrument
2. the scale mean (of a relevant population)

Question 7

What are the two forms of measurement error we covered this week?

Answer 7

1. Standard error of measurement (SEm)
2. Standard error of estimation (SEe)

Question 8

Name three sources of measurement error

Answer 8

Here are 5:

1. Test/item construction
2. Test taker variables
3. Examiner-related variables
4. Testing environment
5. Scoring/interpretation

Question 9

What is the equation at the heart of Classical Test Theory?

Answer 9

X = T + E

Where X = the observed score, T = the true score, and E = error

Question 10

TRUE or FALSE

Cronbach’s alpha is an example of a internal consistency measure

Answer 10

TRUE

Question 11

What is the Spearman-Brown formulation used for in the context of reliability?

Answer 11

For estimating the reliability change that occurs when changing the number of items within a measurement tool

Question 12

Does Cronbach’s alpha measure internal consistency?

Answer 12

Yes (I think)

But I’m not really sure what this is about

Question 13

What is the best alternative to Cronbach’s alpha?

Answer 13

The greatest lower bound (glb)

Question 14

TRUE or FALSE

Where possible, we should use the Standard Error of Measurement (SEm), not the Standard Error or Estimation (SEe)

Answer 14

FALSE

Although both give similar results at high levels of reliability, as you move towards lower reliability the SEe is more accurate

Question 15

What is the equation for calculating the predicted true score (PTS)

Answer 15

PTS = (Rxx * Observed score) + (Scale mean * (1 - Rxx))

Note: the scale mean should come from the best matching population

Question 16

What is the equation for calculating 95% confidence intervals

Answer 16

PTS +/- 1.96 * SEe

Question 17

TRUE or FALSE

Measurement error always attenuates the correlation coefficient between two measured constructs

Answer 17

TRUE

Question 18

What is the equation for calculating standard error of MEASUREMENT?

Answer 18

SEm = sigma * SQRT of (1-Rxx)

Where sigma = the SD of scores on the test

Question 19

What is the equation for calculating standard error of ERROR?

Answer 19

SEe = sigma * (SQRT of Rxx * (1-Rxx))

Where sigma = the SD of scores on the test