Test Construction Flashcards

1
Q

Items with moderate difficulty levels are typically retained in classical test theory because

A
  • .5 = moderate
  • increases test score variability
  • helps ensure scores are normally distributed
  • provides maximum discrimination b/w examinees
  • maximizes the test’s reliability
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Item discrimination index measures

A

the extent to which a test item discriminates between examinees who obtain high versus low scores on the entire test

  • ranges from -1 to 1
  • .35 or above is acceptable
  • items with moderate difficulty most likely to differentiate
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Benefits of Item Response Theory

A
  • item characteristics (parameters) are sample invariant (same across different samples)
  • possible to equate scores from different sets of items/tests
  • easier to develop computer adaptive test
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Which theory of test construction uses an item characteristic curve?

A

item response theory

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

item characteristic curves provide information on ___

A

difficulty, discrimination, probability of guessing correctly

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

According to an item characteristic curve, an items ability to discriminate between high and low achievers is represented by the ____

A
  • slope of the curve

- the steeper the slope, the greater the discrimination

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

According to an item characteristic curve, the probability of guessing correctly is indicated by _____

A

the point at which the ICC intercepts the vertical axis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

According to an item characteristic curve, an item’s difficulty level is indicated by ____

A

the ability level at which 50% of examinees in the tryout sample provided a correct response

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

According to classical test theory, an examinee’s obtained test score (X) is composed of ____ and ____

A

their true score (T) and an error component (E)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

A reliability coefficient of .84 indicates

A

that 84% of variability in scores is due to true score differences among examinees, while the remaining 16% is due to measurement error.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Kuder-richardson Formula 20

A

a variation of coefficient alpha for when test items are scored dichotomously (right/wrong)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

internal consistency reliability is not appropriate for

A

speeded tests

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

the reliability coefficient is maximized when the range of scores is

A

unrestricted

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

standard error of measurement

A

used to construct a confidence interval around a measured (obtained) score.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Content Validity

A

test will be used to obtain information about an examinee’s familiarity with a particular content or behavior domain. Determined by experts

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Construct validity

A

the test will be used to determine the extent to which an examinee possesses a particular hypothetical trait

17
Q

Criterion-related Validity

A

the test will be used to estimate or predict an examniee’s standing or performance on an external criterion

18
Q

Face validity

A

whether or not a test looks like it measures what it is intended to measure

19
Q

convergent and discriminate validity are used to assess ___ validity

20
Q

a squared factor loading provides a measure of

A

shared variablity

21
Q

when factors are orthogonal, a test’s communality can be calculated by ___

A

squaring and adding the test’s factor loadings

22
Q

Two types of criterion-related validity

A

concurrent and predictive

23
Q

standard error of the estimate

A

is used to construct a confidence interval around a predicted (estimated) criterion score.

24
Q

Base rate

A

true positives + false negatives/ total number of people

25
Sensitivity
percent of people in the validation sample who have the disorder and were accurately identified by the predictor as having the disorder. (true positives/ true positive + false negatives)
26
specificity
percent of people in the validation sample who do not have the disorder and were accurately identified by the predictor as not having the disorder. (true negatives/ true negatives + false positives)