CH5 - RELIABILITY Flashcards
what is reliability?
consistency in measurement
this term refers to a statistic statistic that quantifies reliability, ranging from 0 (not at all reliable) to 1 (perfectly reliable)
reliability coefficient
what is measurement error?
measurement error refers to the inherent uncertainty associated with any measurement, even after care has been taken to minimize preventable mistakes
does the true score necessarily reflect the truth?
no, for example a person’s score on a depression questionnaire would differ from their true score on another measurement since depression questionnaires emphasize different aspects of depression
how do we formulize the concept of the observed score?
X (observed score)
T (true score)
E (measurement error)
X = T+E
what does it mean when people’s
observed scores are mostly determined by their true scores?
the test is reliable
what does it mean when people’s
observed scores are mostly determined by measurement error?
the test is unreliable
what statistic is useful in describing sources of test score variability?
variance (σ2)—the standard deviation squared.
what is true variance?
variance from true differences
what is error variance?
variance from irrelevant, random sources
how do we formulize total observed variance?
σ 2 = σ 2t + σ 2e
what can reliability also refer to?
the proportion of the total variance attributed to true variance.
what is the difference between random errors and systematic errors?
random errors cancel each other out while systematic errors do not because systematic errors influence test scores in a consistent direction
what is bias?
bias refers to the degree to which systematic error influences the measurement.
how is test construction considered as a source of error variance?
the content sampled in the tests affect a test taker’s score
what are the sources of error variance? (there are 4)
- test construction
- test administration
- scoring
- interpretation
what are other sources of error?
- sampling error
- methodological error
- nonsystematic error (forgetting or misunderstanding instructions regarding reporting)
how do we estimate the reliability of a measuring instrument?
we use the same instrument to measure the same thing at two points in time. (test-retest method)
what is test-rest reliability?
an estimate of reliability using the test-retest method
when is the test-retest measure appropriate?
the test-retest measure is appropriate when evaluating the reliability of a test that purports to measure something thatis relatively stable over time, such as a personality trait
what is coefficient of stability?
the estimate of test-retest reliability when the interval between testing is greater than six months
this term refers to the degree of the relationship between various forms of a test can be evaluated by means of an alternate-forms or parallel-forms coefficient of reliability
coefficient of equivalence
what are parallel forms of a test?
parallel forms of a test exist when the means and variances of observed test scores are equal
what is parallel forms reliability?
parallel forms reliability refers to an estimate of the extent to which item sampling and other errors have affected test scores on versions of the same test when, for each form of the test, the means and variances of observed test scores are equal.
what are alternate forms of a test?
alternate forms are simply different versions of a test that have been constructed so as to be parallel
this term refers to an estimate of the extent to which these different forms of the same test have been affected by item sampling error, or other error
alternate forms reliability
how do you estimate alternate forms reliability?
calculate the correlation between scores from a representative sample of individuals who have taken both tests
this term refers to the evaluation of the internal consistency of the test items
internal consistency estimate of reliability
what is split-half reliability?
a method/estimate of obtaining internal consistency estimate of reliability by by correlating two pairs of scores obtained from equivalent halves of a single test administered once
how do u compute the coefficient of split-half reliability?
Step 1. Divide the test into equivalent halves.
Step 2. Calculate a Pearson r between scores on the two halves of the test.
Step 3. Adjust the half-test reliability using the Spearman–Brown formula (discussed shortly).
how do you split a test?
- randomly assign items to one or the other half of the test.
- assign odd-numbered items to one half of the test and even-numbered items to the other half
what is the primary objective in splitting a test for the purpose of obtaining a split-half reliability estimate?
it is to create what can be called as mini-parallel forms, with each half equal to each other, in format and statistical aspects
