unknown quiz questions

Question 1

can systematic error be eliminated

Answer 1

yes

Question 2

what is true about random error

Answer 2

it can never be eliminated entirely and it affects all test scores

Question 3

what is the true score

Answer 3

it is the persons score without random error, it might or might not be the same as the observed score, and it can never be measured precisely

Question 4

can reliability be computed directly

Answer 4

no, it is a theoretical property of a test

Question 5

what is true about the variance of the observed scores in very large samples

Answer 5

it is greater than the variance of the true scores

Question 6

relationship between random measurement error and true scores?

Answer 6

random measurement error is uncorrelated with true scores

Question 7

what does random measurement error do

Answer 7

make people look more different from each other than they actually are

Question 8

what does a reliability of .90 mean

Answer 8

90% of the variance in the observed scores is due to real or true differences among test takes on the trait being measured

Question 9

what is the test/retest method used to estimated the amount of error associated with…

Answer 9

conditions at the time when the test is administered

Question 10

if the test/retest interval is very short than the test’s reliability will be

Answer 10

over estimated by the test/retest method because of memory for the original responses and similar conditions affecting test and retest

Question 11

if the test/retest interval is very long than the test’s reliability will be

Answer 11

under estimated by the test / retest method because of real changes in the trait being measured

Question 12

on a test where there is substantial practice effects what will the test-retest method do

Answer 12

overestimate the amount of time sampling error

Question 13

explain regression to the mean

Answer 13

overtime with test-restest scores will gravitate toward the mean, scores that are very high will decrease on retesting and scores that are very low will increase

Question 14

if the test-retest reliability is .8 then what is the % of the variance in obtained scores due to time sampling error

Answer 14

20%

Question 15

typical sources of measurement error

Answer 15

item sampling, time sampling, internal consistency, and inter rater differences

Question 16

which of the following is the symbol for a predicted retest score

Answer 16

ZR

Question 17

what is true regarding percent agreement

Answer 17

percent agreement can overestimate interrater reliability because raters could agree by chance

Question 18

percent agreement and Kappa?

Answer 18

percent agreement s always greater than or equal to Kappa

Question 19

what does standard error of measurement allow us to estimate

Answer 19

the amount of error in an individual’s score

Question 20

what is the SEM

Answer 20

the standard deviation of the error scores

Question 21

what is the difference between the observed score and the estimated true score when the reliability is high

Answer 21

with high reliability, the difference is less

Question 22

what is the standard error of difference used for

Answer 22

to decide if two scores are significantly different from one another

Question 23

what is the purpose of investigating the validity of a test

Answer 23

to determine if the test is measuring what it specifically intends to be measuring

Question 24

does a test have to be valid in order to be reliable

Answer 24

no

Question 25

why could a test with high face validity introduce measurement error

Answer 25

because it is easier for test-takers to distort their answers

Question 26

two kinds of criterion validity

Answer 26

concurrent and predictive

Question 27

two kinds of content validity and examples

Answer 27

1) Construct-irrelevant content (including items that don’t belong)
- Test on Chapters 1-4 includes items from Chap 5

2) Construct underrepresentation (failing to include items that should be included)
- Test on Chapters 1-4 include items only on material in Chapter 3.

Question 28

what are item specifications and expert panels used to assess

Answer 28

content validity

Question 29

face vs content validity

Answer 29

face validity pertains to a test’s appearance (can test takers tell what is being assessed by looking at the test items) whereas content validity pertains to a test’s coverage (do the test items adequately cover the domain the test intends to assess)

Question 30

what can compromise content validity

Answer 30

including items that do not belong on the test and failing to include items that should be included on the test

Question 31

concurrent vs. predictive validity

Answer 31

two tests at same time see if they are related

one test now to see if it relates to one test later

Question 32

what is criterion contamination

Answer 32

if the assessment of a criterion is not independent of the test results

Question 33

what can affect the magnitude of a validity coefficeint

Answer 33

restricted range, heteroscedasticity, and a non-linear relationship between test and criterion

Question 34

if a validity study uses a criterion with low reliability the true validity of the test will be

Answer 34

underestimated