unknown quiz questions Flashcards
can systematic error be eliminated
yes
what is true about random error
it can never be eliminated entirely and it affects all test scores
what is the true score
it is the persons score without random error, it might or might not be the same as the observed score, and it can never be measured precisely
can reliability be computed directly
no, it is a theoretical property of a test
what is true about the variance of the observed scores in very large samples
it is greater than the variance of the true scores
relationship between random measurement error and true scores?
random measurement error is uncorrelated with true scores
what does random measurement error do
make people look more different from each other than they actually are
what does a reliability of .90 mean
90% of the variance in the observed scores is due to real or true differences among test takes on the trait being measured
what is the test/retest method used to estimated the amount of error associated with…
conditions at the time when the test is administered
if the test/retest interval is very short than the test’s reliability will be
over estimated by the test/retest method because of memory for the original responses and similar conditions affecting test and retest
if the test/retest interval is very long than the test’s reliability will be
under estimated by the test / retest method because of real changes in the trait being measured
on a test where there is substantial practice effects what will the test-retest method do
overestimate the amount of time sampling error
explain regression to the mean
overtime with test-restest scores will gravitate toward the mean, scores that are very high will decrease on retesting and scores that are very low will increase
if the test-retest reliability is .8 then what is the % of the variance in obtained scores due to time sampling error
20%
typical sources of measurement error
item sampling, time sampling, internal consistency, and inter rater differences
which of the following is the symbol for a predicted retest score
ZR
what is true regarding percent agreement
percent agreement can overestimate interrater reliability because raters could agree by chance
percent agreement and Kappa?
percent agreement s always greater than or equal to Kappa
what does standard error of measurement allow us to estimate
the amount of error in an individual’s score
what is the SEM
the standard deviation of the error scores
what is the difference between the observed score and the estimated true score when the reliability is high
with high reliability, the difference is less
what is the standard error of difference used for
to decide if two scores are significantly different from one another
what is the purpose of investigating the validity of a test
to determine if the test is measuring what it specifically intends to be measuring
does a test have to be valid in order to be reliable
no
why could a test with high face validity introduce measurement error
because it is easier for test-takers to distort their answers
two kinds of criterion validity
concurrent and predictive
two kinds of content validity and examples
1) Construct-irrelevant content (including items that don’t belong)
- Test on Chapters 1-4 includes items from Chap 5
2) Construct underrepresentation (failing to include items that should be included)
- Test on Chapters 1-4 include items only on material in Chapter 3.
what are item specifications and expert panels used to assess
content validity
face vs content validity
face validity pertains to a test’s appearance (can test takers tell what is being assessed by looking at the test items) whereas content validity pertains to a test’s coverage (do the test items adequately cover the domain the test intends to assess)
what can compromise content validity
including items that do not belong on the test and failing to include items that should be included on the test
concurrent vs. predictive validity
two tests at same time see if they are related
one test now to see if it relates to one test later
what is criterion contamination
if the assessment of a criterion is not independent of the test results
what can affect the magnitude of a validity coefficeint
restricted range, heteroscedasticity, and a non-linear relationship between test and criterion
if a validity study uses a criterion with low reliability the true validity of the test will be
underestimated
