Reliability Flashcards
What does reliability refer to in CTT?
Precision or consistency of measurement.
An indicator of how well a sample of items or tests represent the universe of possible items or tests of a construct.
The ratio of true score to observed score variance.
can think of reliability as: squared correlation between the observed and true score.
As a ration, reliability will be bound between 0 and 1 (observed score variance will always be higher than true score variance)
How do you increase reliability?
Reduce error variance.
Reliability tends to increase with more items.
What is reliability not?
It is not the same as an indicator of reliability e.g. Chrobach’s alpha- these are just methods to estimate reliability.
:( these methods asses the reliability of scoring on occasions but not the tests themselves- context is important (reliable in some but not in others)
It is separate from validity.
Test-retest reliability
Test consistency.
Same test on the same participant on 2 or more occasions.
A correlation coefficient between test scores at 2 time points is an estimate of reliability.
Other factors should be held constant and should be in similar testing conditions
Split-half reliability
Conceptually similar to test-retest, but for data from a single session.
Calculate a correlation between 2 scores on 2 randomly selected halves of the data.
Provides and index of if the 2 halves of a scale are measuring as similar contruct.
Most widely used coefficient to assess internal consistency
Chronbach’s alpha
Higher values reflect higher internal consistency.
> .7 generally considered good
> .9 could indicate redundancy of items (fine if using an existing scale, but consider removing items in developing a new scale)
Issue with including poor items
:( Poor measures means you underestimate the true relationship between variables, you might be limited in observing a significant correlation.
