Week 2 Reliability

Question 1

Random measurement error:

Answer 1

Reflects an error that is inconsistent and due to random or chance factors that are not consistently or systematically part of the test – these random errors may affect each administration of the test (e.g. different test takers or testing occasions) differently and inconsistently.

Question 2

Systematic error:

Answer 2

Refers to a consistent error that may be inherent in the nature of the test, in that there may be a characteristic of the test that affects all test takers. In other words, the error is due to the test itself rather than chance.

Question 3

Classical test theory (CTT):

Answer 3

• Focus is on the total score
• Correlation is used to estimate proportion of error in the score
• In CTT, when we are thinking about measurement error, we focus on the total score for the test. We use correlation to estimate the proportion of error in the score

Question 4

Item response (latent trait) theory (IRT):

Answer 4

• Focus is on the items
• A logistic (non-linear) function is used to estimate the degree of precision in the information provided by each item
• In IRT, the focus is on the precision of a test score and logistic modelling is used to estimate that level of precision provided by each item.

Question 5

Reliability:

Answer 5

Think about being a reliable person. This is what you want the test to be. Reliable, consistent, dependable, predictable, trustworthy.
Reliability can be defined as the proportion of true variance to the total variance in a set of scores, with the extent of common variance being estimated by the correlation coefficient. Remember, a correlation coefficient can estimate the percentage of shared variance.