4 Reliability MC

Question 1

1 According to classical test score theory, what happens to the true score variance as error
in a measure increases?
A it increases
B it decreases
C it remains constant
D classical test score theory makes no statement on this point

Answer 1

B it decreases

Question 2

2 According to classical test score theory, a test score is made up of
A true score variance and nonsystematic variance
B observed score variance and true score variance
C observed score variance and error variance
D observed score variance and systematic variance

Answer 2

A true score variance and nonsystematic variance

Question 3

3 The wording of several items on a psychological test makes it more likely that test
takers will endorse the ‘Yes’ rather than the ‘No’ option. This is best described as
A systematic variance in the test
B unsystematic variance in the test
C clever item writing
D a problem for the test taker

Answer 3

A systematic variance in the test

Question 4

4 Systematic error in a test exerts what kind of effect on test scores?
A random
B consistent
C unknowable
D inconsistent

Answer 4

B consistent

Question 5

5 Another way of talking about the reliability of a test for a particular purpose is to talk
about its
A dependability
B validity
C utility
D discriminability

Answer 5

A dependability

Question 6

6 The proportion of observed score variance attributable to random error is known as
A the reliability coefficient
B the coefficient of nondetermination
C the error coefficient
D one minus the reliability coefficient

Answer 6

D one minus the reliability coefficient

Question 7

7 Test-retest reliability is sometimes referred to as
A stability
17
B consistency
C long-term reliability
D concurrent reliability

Answer 7

A stability

Question 8

8 The domain sampling model proposes that
A items in a test are a random sample from a population of possible items
B the only items possible have been used in the test
C items have been sampled without replacement
D the majority of items have the same content

Answer 8

A items in a test are a random sample from a population of possible items

Question 9

9 The domain sampling model as originally conceived could not deal well with
A split half reliability
B internal consistency reliability
C equivalent forms reliability
D test-retest reliability

Answer 9

D test-retest reliability

Question 10

10 Which of the following procedures does not yield an estimate of the reliability of a test?
A correlating the total of all even-numbered items with the total of all odd-numbered
items
B correlating the total of items in the first half of the test with the total of items in
the second half of the test
C correlating each item with the total score on the test
D finding the average of the correlation of each item with every other item

Answer 10

C correlating each item with the total score on the test

Question 11

11 Estimating test reliability by correlating scores from two administrations of the test 6
months apart assumes
A the trait being measured changes over time
B the trait being measured is essentially episodic in character
C the trait being measured does not change over time
D there is a systematic practice effect on the test

Answer 11

C the trait being measured does not change over time

Question 12

12 The reliability of expert judgment can be estimated by
A correlating the judgments made by a panel of experts over a number of instances
of judgment making
B counting the frequency of instances in which a panel of experts disagree
C finding the proportion of instances in which a panel of experts is undecided
D averaging the number of decisions a panel of experts gets wrong

Answer 12

A correlating the judgments made by a panel of experts over a number of instances
of judgment making

Question 13

13 Inter-rater reliability
A overcomes the problems of test reliability
B is a special case of test reliability
C cannot be estimated statistically
D uses the same formula as that used for equivalent forms reliability

Answer 13

B is a special case of test reliability

Question 14

14 The concept of ‘domain sampling’ in the psychometric theory of reliability refers to
A sampling persons from the population with whom a test may be used
B sampling items from the population of possible items that could be used in a test
C sampling tests from the population of tests available to measure a construct
D sampling methods from the population that could be used to construct a test

Answer 14

B sampling items from the population of possible items that could be used in a test

Question 15

15 The standard error of measurement of a raw score
A increases directly as the reliability increases
B decreases directly as the reliability increases
C increases proportionately as the reliability increases
D decreases proportionately as the reliability increases

Answer 15

B decreases directly as the reliability increases

Question 16

16 In making judgments about the precision of a score on a test we need to know
A the reliability of the test for the purpose for which we are using it
B the standard deviation of scores on the test
C the mean and standard deviation of scores on the test
D the reliability of the test for the purposes for which we are using it and the
standard deviation of scores on the test

Answer 16

D the reliability of the test for the purposes for which we are using it and the standard deviation of scores on the test

Question 17

17 Equivalent forms of a test are usually developed
A when the test is first developed
B when the test’s reliability is first questioned
C when the test is first readministered
D when the test is being revised

Answer 17

A when the test is first developed

Question 18

18 The Spearman-Brown prophecy formula is so called because it purports to indicate
A what the reliability of the test would be if certain changes were made to it
B what the individual’s true score on the test is
C what an individual’s score on the test will be at some future time
D what the person’s true score would be if the test were lengthened

Answer 18

A what the reliability of the test would be if certain changes were made to it

Question 19

19 The Spearman-Brown prophecy formula requires
A the reliability of the current test
B the number of items in the current test
C both A and B
D neither A nor B

Answer 19

C both A and B

Question 20

20 The internal consistency of a test would be high if
A it included items that related to different aspects of the construct to be measured
B it included items that related to different constructs
C each item was drawn from a different item domain
D all the items were the same

Answer 20

D all the items were the same

Question 21

21 A high coefficient alpha indicates that
A the test has high generalisability
B scores on the test are stable
C the test has high internal consistency
D the test has only one factor

Answer 21

C the test has high internal consistency

Question 22

22 Reliability of a test
A can change if the range of scores on the test is smaller relative to the original
sample of scores
B is an unchanging property of a test
C changes from one administration of a test to another
D will differ depending on the mean score of the sample or the test

Answer 22

A can change if the range of scores on the test is smaller relative to the original
sample of scores

Question 23

23 Coefficient alpha can be calculated
A only for tests with dichotomously scored items
B only for tests with items that have three or more categories
C only for tests that use a Yes/No or True/False format
D for all objectively scored tests

Answer 23

D for all objectively scored tests

Question 24

24 Generalisability theory requires that we know
A the reliability of the test
B the standard error of the test
C how the test is to be used
D the mean score on the test

Answer 24

C how the test is to be used

Question 25

25 Expectations about what constitutes a satisfactory degree of reliability
A depend on the purpose for which the test is being used
B have been determined by consensus
C seldom depart from the agreed value of 0.9
D depend on the magnitude of the standard error of measurement

Answer 25

C seldom depart from the agreed value of 0.9

Question 26

26 In general the best reliabilities have been obtained with psychological tests in the
A cognitive domain
B personality domain
C motivation domain
D projective domain

Answer 26

A cognitive domain

Question 27

27 The correlation between scores on two variables varies
A directly with the product of their reliabilities
B directly with the square root of the product of their reliabilities
C inversely with the sum of their reliabilities
D inversely with the square root of the lower of the two reliabilities

Answer 27

A directly with the product of their reliabilities

Question 28

28 Two variables may not correlate highly
A because of the poor reliability of one or both of them
B because their standard errors of measurement are skewed in opposite directions
C because their reliabilities are unknown
D because similar items have been used in assessing both variables

Answer 28

B because their standard errors of measurement are skewed in opposite directions

Question 29

29 Reliability is
A relevant when considering the score a person obtains on a test or other assessment
device
B relevant only when psychological test results are being considered but not when
expert judgements are employed
C irrelevant for most practical decision making with psychological tests
D relevant for tests of intelligence only

Answer 29

A relevant when considering the score a person obtains on a test or other assessment
device

Question 30

30 Reliability of an assessment device can be improved within limits by
A increasing its length (e.g. using more items)
B decreasing the time taken to administer it
C supplementing it with the judgment of the assessor
D replacing it with the judgment of the assessor

Answer 30

A increasing its length (e.g. using more items)