Reliability - Doc Joyce Discussion

Question 1

what is reliability?

Answer 1

the degree to which a test is consistent is measuring its supposed variable

Question 2

what are the four types of reliability estimates?

Answer 2

1. test-retest reliability
2. parallel-forms and alternate-forms reliability
3. split-half reliability
4. internal consistency reliability

Question 3

for test-retest reliability, what is its source of error, procedure, reliability coefficient/index, and when is it used?

Answer 3

source of error: time

procedure: correlating the same test at different times

reliability coefficient/index: coefficient of stability

used for time-dependent variables

Question 4

for parallel forms and alternate-forms reliability, what is its source of error, procedure, reliability coefficient/index, and when is it appropriate

Answer 4

source of error: item form/structure/arrangement

procedure: correlate the scores with two tests, with the same everything including test takers

reliability coefficient/index: coefficient of equivalence

appropriate for form or arrangement dependent variables

Question 4

for split-half reliability, what is its source of error, procedure, reliability coefficient/index, and when is it appropriate

Answer 4

source of error: item heterogeneity

procedure: divide the test items, correlate, and correct with spearman-brown

reliability coefficient/index: coefficient of item homogeneity

appropriate for one-time administration and all types of variables

Question 4

for internal consistency reliability, what is its source of error, procedure, reliability coefficient/index, and when is it appropriate

Answer 4

source of error: item heterogeneity

procedure: Kuder-Richardson 20/21 forms

reliability coefficient/index: coefficient of item homogeneity

appropriate for test that are dichotomous or binary response

Question 4

what are the estimates used in internal consistency reliability?

Answer 4

1. KR20
2. KR21
3. Cronbach’s Alpha

Question 5

when is KR20 appropriate?

Answer 5

when it is a true dichotomous test (true or false)

Question 6

when is KR21 appropriate?

Answer 6

when it is an artificial dichotomy (multiple choice, binary scoring)

Question 7

when is Cronbach’s alpha appropriate?

Answer 7

when it is a true multiple choice test (Likert Scale)

Question 8

what are the estimates categorized as external reliability estimates? and why are they called external?

Answer 8

test-retest
parallel-forms and alternate forms reliability

called external because they assess the consistency of a measure across different individuals or administrations

Question 9

what are the estimates categorized as internal reliability estimates?

Answer 9

split-half
internal consistency