Classical Analysis of observed test scores Flashcards
What does Varr (Ej) denote?
Within person variance
What does measurement precision consist of?
Information and reliability
Who does information apply to?
Information applies to the test score of a single person, it is the within-person aspect of measurement precision
Who does reliability apply to?
Reliability applies to a population of persons. It is the between-person aspect of measurement precision
What is meant by the true score?
The score with a perfect measurement instrument
What is meant by measurement error?
The distortion of the true score because the measurement instrument is not perfect, they are unsystematic influences
Give a mathematical equation for the test score of a person
Sj=Tj+Ej for person j
Test score= true score + error
What is the expected value of the measurement error over infinite trials?
0
What is test taker j’s standard error of measurement?
The square root of the within-person variance
What is the information on test taker j’s true score?
The inverse of the within-person variance
What formula does the information on test taker j’s true score use?
𝐼𝑛𝑓(τ𝑗) = 1/𝑉𝑎𝑟 (𝐸 ) 𝑟𝑗
What is meant by a parallel test?
A different test with exactly the same properties as the first test
What is reliability in regards to parallellel tests?
Reliability is the correlation between two parallel test scores.
How can reliability be tested using one test?
Reliability can be calculated by splitting the test into two halves and treating them both as parallel tests. The reliability of each part equals the correlation between the two parts.
How is reliability calculated using two halves? (formula)
𝑅𝑒̂𝑙(𝑆) = 𝑠1𝑠2/1+𝑟 𝑠1𝑠2
It is two times the correlation between part one and part two divided by one plus the correlation between part one and part two.
What is chronbach’s alpha used to calculate?
The lower-bound of the reliability of the full test
What formula does chronbach’s alpha use?
It is the number of items divided by number of items minus one times (one minus the sum of item variances divided by the test variance).
What does this formula reveal about the effect of the number of items
The reliability depends on the number of items. A large test is usually more reliable than a shorter test.
Does the correlation between test scores equal the correlation between constructs?
The correlation between the test scores is not equal to the correlation between the constructs, as the test score is the true score plus measurement error. The correlation between the test score is smaller than the correlation between true scores.
What formula corrects for this?
The correction for attenuation formula:
𝐶𝑜𝑟𝑝 (𝑇𝑎 , 𝑇𝑏 ) = 𝐶𝑜𝑟𝑝 (𝑆𝑎 ,𝑆𝑏 ) /
√𝑅𝑒𝑙(𝑆𝐴 )(𝑅𝑒𝑙(𝑆𝐵 )
It is the correlation between scores of test A and the scores of test B divided by the square root of the reliability of test A times the reliability of test B.
What can be done regarding reliability after using a number of parallel tests containing a number of items ?
The reliability of a test after adding a number of parallel tests containing a number of items can be calculated using the Spearman-Brown formula
Give the Spearman-Brown formula
𝑅𝑒𝑙{𝑆(𝐾)} = 𝐾𝑅𝑒𝑙(𝑆)/
1 + (𝐾 − 1)𝑅𝑒𝑙(𝑆)
It is the number of parallel parts times the reliability of one parallel part divided by one plus (the number of parallel parts minus one) times the reliability of one parallel part.
How can the number of items needed for a particular reliability be calculated?
𝐾 = {1 − 𝑅𝑒𝑙(𝑆)}𝑅𝑒𝑙{𝑆(𝐾)} /[1 − 𝑅𝑒𝑙{𝑆(𝐾)}]𝑅𝑒𝑙(𝑆)
It is one minus the reliability of the test times the desired reliability divided by one minus the desired reliability times the reliability of the test.
Rel(S) denotes the reliability of the test. Rel{S(K)} denotes the desired reliability. K denotes the number of parts that should be added to the test.
What is the relationship between the reliability of the test and the standard error of measurement?
The higher the reliability the smaller the standard error of measurement. The lower the reliability the larger the standard error of measurement.
How can the standard error of measurement be calculated?
𝑆𝑒̂𝑚 = √𝑉𝑎̂𝑟 (𝑆){1 − 𝑅𝑒̂𝑙(𝑆)}
It is the square root of the variance of the test times one minus the reliability of the test.
How else can the standard error of measurement be calculated?
using the within-person variance:
√ ∑𝑁𝑠 𝑗=1 (𝑆1𝑗−𝑆2𝑗)2 / 𝑁 𝑠
It is the square root of the sum of the score of test taker j on test 1 minus the score of test taker j on test 2 squared divided by the number of test-takers.
