Classical Analysis of observed test scores

Question 1

What does Varr (Ej) denote?

Answer 1

Within person variance

Question 2

What does measurement precision consist of?

Answer 2

Information and reliability

Question 3

Who does information apply to?

Answer 3

Information applies to the test score of a single person, it is the within-person aspect of measurement precision

Question 4

Who does reliability apply to?

Answer 4

Reliability applies to a population of persons. It is the between-person aspect of measurement precision

Question 5

What is meant by the true score?

Answer 5

The score with a perfect measurement instrument

Question 6

What is meant by measurement error?

Answer 6

The distortion of the true score because the measurement instrument is not perfect, they are unsystematic influences

Question 7

Give a mathematical equation for the test score of a person

Answer 7

Sj=Tj+Ej for person j

Test score= true score + error

Question 8

What is the expected value of the measurement error over infinite trials?

Answer 8

0

Question 9

What is test taker j’s standard error of measurement?

Answer 9

The square root of the within-person variance

Question 10

What is the information on test taker j’s true score?

Answer 10

The inverse of the within-person variance

Question 11

What formula does the information on test taker j’s true score use?

Answer 11

𝐼𝑛𝑓(τ𝑗) = 1/𝑉𝑎𝑟 (𝐸 ) 𝑟𝑗

Question 12

What is meant by a parallel test?

Answer 12

A different test with exactly the same properties as the first test

Question 13

What is reliability in regards to parallellel tests?

Answer 13

Reliability is the correlation between two parallel test scores.

Question 14

How can reliability be tested using one test?

Answer 14

Reliability can be calculated by splitting the test into two halves and treating them both as parallel tests. The reliability of each part equals the correlation between the two parts.

Question 15

How is reliability calculated using two halves? (formula)

Answer 15

𝑅𝑒̂𝑙(𝑆) = 𝑠1𝑠2/1+𝑟 𝑠1𝑠2

It is two times the correlation between part one and part two divided by one plus the correlation between part one and part two.

Question 16

What is chronbach’s alpha used to calculate?

Answer 16

The lower-bound of the reliability of the full test

Question 17

What formula does chronbach’s alpha use?

Answer 17

It is the number of items divided by number of items minus one times (one minus the sum of item variances divided by the test variance).

Question 18

What does this formula reveal about the effect of the number of items

Answer 18

The reliability depends on the number of items. A large test is usually more reliable than a shorter test.

Question 19

Does the correlation between test scores equal the correlation between constructs?

Answer 19

The correlation between the test scores is not equal to the correlation between the constructs, as the test score is the true score plus measurement error. The correlation between the test score is smaller than the correlation between true scores.

Question 20

What formula corrects for this?

Answer 20

The correction for attenuation formula:
𝐶𝑜𝑟𝑝 (𝑇𝑎 , 𝑇𝑏 ) = 𝐶𝑜𝑟𝑝 (𝑆𝑎 ,𝑆𝑏 ) /
√𝑅𝑒𝑙(𝑆𝐴 )(𝑅𝑒𝑙(𝑆𝐵 )
It is the correlation between scores of test A and the scores of test B divided by the square root of the reliability of test A times the reliability of test B.

Question 21

What can be done regarding reliability after using a number of parallel tests containing a number of items ?

Answer 21

The reliability of a test after adding a number of parallel tests containing a number of items can be calculated using the Spearman-Brown formula

Question 22

Give the Spearman-Brown formula

Answer 22

𝑅𝑒𝑙{𝑆(𝐾)} = 𝐾𝑅𝑒𝑙(𝑆)/
1 + (𝐾 − 1)𝑅𝑒𝑙(𝑆)

It is the number of parallel parts times the reliability of one parallel part divided by one plus (the number of parallel parts minus one) times the reliability of one parallel part.

Question 23

How can the number of items needed for a particular reliability be calculated?

Answer 23

𝐾 = {1 − 𝑅𝑒𝑙(𝑆)}𝑅𝑒𝑙{𝑆(𝐾)} /[1 − 𝑅𝑒𝑙{𝑆(𝐾)}]𝑅𝑒𝑙(𝑆)

It is one minus the reliability of the test times the desired reliability divided by one minus the desired reliability times the reliability of the test.
Rel(S) denotes the reliability of the test. Rel{S(K)} denotes the desired reliability. K denotes the number of parts that should be added to the test.

Question 24

What is the relationship between the reliability of the test and the standard error of measurement?

Answer 24

The higher the reliability the smaller the standard error of measurement. The lower the reliability the larger the standard error of measurement.

Question 25

How can the standard error of measurement be calculated?

Answer 25

𝑆𝑒̂𝑚 = √𝑉𝑎̂𝑟 (𝑆){1 − 𝑅𝑒̂𝑙(𝑆)}

It is the square root of the variance of the test times one minus the reliability of the test.

Question 26

How else can the standard error of measurement be calculated?

Answer 26

using the within-person variance:

√ ∑𝑁𝑠 𝑗=1 (𝑆1𝑗−𝑆2𝑗)2 / 𝑁 𝑠

It is the square root of the sum of the score of test taker j on test 1 minus the score of test taker j on test 2 squared divided by the number of test-takers.