Reliability

Question 1

Ratio between true score variance on a test and the total variance

Answer 1

Reliability coefficient

Question 2

Chance features of the individual or situation

Answer 2

Random measurement of error

Question 3

Tool used to estimate or infer the extent to which an observed score deviates from a true score

Answer 3

Standard error of estimate (SEE)

Question 4

Correlates performance on 2 interval scale measures and uses that correlation to indicate the extent of true score differences

Answer 4

Reliability analysis

Question 5

Sources of error variance

Answer 5

• test construction (item or content sampling)
• test administration (testtaker and examiner-related variables)
• test scoring and interpretation (computer or subjective scoring)

Question 6

Appropriate when evaluating reliability of a test that purports to measure something that is relatively stable over time (e.g. personality trait)

Answer 6

Test-retest method (aka time sampling reliability)

Question 7

When interval between testing is greater than 6 months

Answer 7

Coefficient of stability

Question 8

An effect where the first testing session influences results of second session

Answer 8

Carryover effect

Solution: prolong time (as little as 4 days to avoid error)

Question 9

Type of carryover effect when second session of test is higher than the first test

Answer 9

Practice effect

Question 10

Each form of test, the means and variances of observed test scores are equal; scores obtained correlate equally with other measures

Answer 10

Parallel forms

Question 11

Different versions of a test that have been constructed so as to be parallel; COEFFICIENT OF EQUIVALENCE

Answer 11

alternate forms (item sampling reliability)

Question 12

Assess consistency of results across items of a test

Answer 12

internal consistency

Question 13

Correlate 2 pairs obtained from equivalent halves of a single test administered once

Answer 13

Split-half reliability

Question 14

Useful when it is undesirable to assess reliability with 2 tests because of factors such as time, expense

Answer 14

Split-half method

Question 15

Correction formula which allows you to estimate what the correlation between 2 halves would have been if each half had been the length of the whole test; specific application to estimate reliability of the test that is lengthened or shortened by any number of items

Answer 15

Spearman brown formula

Question 16

Also used to determine the number of items needed to attain a desired level of reliability

Answer 16

Spearman brown formula

Question 17

Degree of correlation among all items in a scale; useful in assessing homogeneity of test; calculated from a single administration of a single form of test

Answer 17

Inter-item consistency

Question 18

An estimate of reliability that determine the inter-item consistency of dichotomous items (right or wrong); used for items that have varying diffilculty; 0.5 reasonable reliability

Answer 18

KR 20 (for group intelligence and ability tests)

Question 19

Used when all items about the same difficulty

Answer 19

kr 21

Question 20

Estimates the internal consistency of test in which the items are non-dichotomous or there is no right or wrong answer (e.g. Likert scale)

Answer 20

Coefficient alpha/Cronbach’s alpha

Question 21

Thought as the mean of all possible split-half correlations, test-retest; used when 2 halves of the test have unequal variances; 0.7 reasonable reliability

Answer 21

Coefficient alpha/Cronbach’s alpha (measure only 1 hidden variable or dimension)

Question 22

The degree of agreement or consistency between 2 or more scorers with regard to a particular measure

Answer 22

Inter-scorer reliability (for individual intelligence tests and projective tests)

Question 23

If test is heterogeneous, what reliability estimate should you use?

Answer 23

test-retest method

Question 24

In measuring dynamic traits, what reliability estimate is appropriate?

Answer 24

Internal consistency

Question 25

In measuring static (unchanging) traits, what reliability estimate is appropriate?

Answer 25

test-retest, alternate/parallel forms

Question 26

The type of internal consistency which is considered as the average of all split-half correlations

Answer 26

Average inter-item correlations