Reliability

Question 1

an index of reliability, a proportion that indicates the ratio between the true score variance on a test and the total variance

Answer 1

reliability coefficient

Question 2

the prerequisite of validity

Answer 2

high reliability

Question 3

reliability increases with [ ]

Answer 3

test length

Question 4

standard deviation squared. is useful because it can be broken into components

Answer 4

variance

Question 5

variance from true differences

Answer 5

true variance

Question 6

variance from irrelevant, random sources

Answer 6

error variance

Question 7

refers to the proportion of the total variance attributed to true variance

Answer 7

reliability

Question 8

sources of variance

Answer 8

test construction
administration
scoring
interpretation

Question 9

variance in test construction

Answer 9

item sampling or content sampling

Question 10

variance in test administration

Answer 10

test environment
testtaker variables
examiner-related variables

Question 11

test scoring and interpretation

Answer 11

scorers and scoring system

Question 12

an estimate of reliability obtained by correlating pairs of scores from the same people on two different administrations of the same test

Answer 12

test-retest reliability

Question 13

how stable is the construct or measure

Answer 13

coefficient of stability

Question 14

If the duration of test-retest is too short, there is a tendency for [ ]

Answer 14

carryover effect/practice effect

Question 15

test-retest is not applicable for [ ]

Answer 15

states

Question 16

how to measure test-retest reliability

Answer 16

pearson r or spearman rho

Question 17

The consistency of test results between two different – but equivalent – forms of a test.

Answer 17

parallel forms and alternate-forms reliability

Question 18

for each form of the test, the means and the variances of observed test scores are equal.

Answer 18

parallel forms

Question 19

are simply different versions of a test
that have been constructed so as to be parallel.

Answer 19

alternate forms

Question 20

coefficient for parallel and alternate forms

Answer 20

coefficient of equivalence

Question 21

the advantage of having another form

Answer 21

eliminates carryover/practice effects

Question 22

how to measure parallel and alternate forms reliability

Answer 22

pearson r or spearman rho

Question 23

Defines measurement error strictly in terms of consistency or inconsistency in the content of the test.

Answer 23

internal consistency reliability

Question 24

obtained by correlating two pairs of scores obtained from equivalent halves of a single test administered once.

Answer 24

split-half reliability estimate

Question 25

three steps in split-half reliability estimate

Answer 25

Step 1. Divide the test into equivalent halves.
Step 2. Calculate a Pearson r between scores on the two halves of the test.
Step 3. Adjust the half-test reliability using the Spearman-Brown formula.

Question 26

allows a test developer or user to estimate internal consistency reliability from a correlation of two halves of a test.

Answer 26

spearman-brown formula

Question 27

-Used with ratio or interval data.
-Mean of all possible split-half correlations
-Preferred statistic for obtaining an estimate
of internal consistency reliability.
-Typically ranges in value from 0 to 1

Answer 27

cronbach’s coefficient alpha

Question 28

used for test with dichotomous items, primarily those
items that can be scored right or wrong (such as multiple-
choice items). useful in terms of evaluating the internal
consistency of highly homogenous items

Answer 28

kuder-richardson formula

Question 29

is used for items that have varying difficulty. For example, some items might be very easy, others more challenging. it should only be used if there is a correct answer for each
question

Answer 29

kr-20

Question 30

it’s used for a test where the items are all about the same difficulty.

Answer 30

kr-21

Question 31

refers to the degree of correlation among all the
items on a scale.

Answer 31

inter-item consistency

Question 32

Ideally, the average inter-item correlation for a set of items should be between [ ] and [ ], suggesting that while the items are reasonably homogenous, they do contain sufficiently unique variance so as to not be isomorphic with each other.

Answer 32

.20 and .40

Question 33

The degree of agreement or consistency between
two or more scorers (or judges or raters) with regard
to a particular measure.

Answer 33

inter-scorer reliability

Question 34

how to measure inter-scorer reliability

Answer 34

pearson r or spearman rho

Question 35

A reliability coefficient of .80 indicates that 20% of the variability in test scores are due to [ ].

Answer 35

measurement error

Question 36

Coefficient of inter-rater reliability provides information about error as a result of [ ]

Answer 36

test-scoring

Question 37

Coefficient of stability provides information on error as a result of

Answer 37

length in the time of administration

Question 38

Coefficient of equivalence provides information on error as a result of [ ]

Answer 38

instrument (items) itself

Question 39

high degree of internal consistency

Answer 39

test homogeneity

Question 40

low degree of internal consistency

Answer 40

tests heterogeneity

Question 41

a characteristic where the best estimate of reliability would be obtained from a measure of internal consistency.

Answer 41

dynamic characteristic

Question 42

a characteristic where the test-retest or the alternate-forms method would be appropriate.

Answer 42

static characteristic

Question 43

If the variance of either variable in a correlational analysis is
[ ] by the sampling procedure used, then the resulting correlation coefficient tends to be lower.

Answer 43

restricted

Question 44

If the variance of either variable in a correlational analysis is [ ] by the sampling procedure, then the resulting
correlation coefficient tends to be higher.

Answer 44

inflated (increased)

Question 45

all the items are of the same degree of difficulty. There is time limit within which the test taker is required to answer all the items.

Answer 45

speed test

Question 46

assesses the underlying ability of the individuals by allowing
them sufficient time; no time limit.

Answer 46

power test

Question 47

designed to provide an indication of where a test-taker stands with respect to some variable or criterion, such as an educational or a vocational objective.

Answer 47

criterion-referenced test