Chapter 5: Validity

Question 1

Validity

Answer 1

Appropriateness and accuracy of the interpretation of test scores
Can’t be measured in a single test (need a comparison basis)

Question 2

Construct underrepresentation

Answer 2

Test doesn’t measure important aspects of the specified construct
Similar to content sampling error

Question 3

Construct-irrelevant variance

Answer 3

Test measures features that are unrelated to the specified construct

Question 4

External threats to validity

Answer 4

Examinee characteristics (ex- anxiety, which hinders examinee)
Deviation from standard test administration and scoring
Instruction and coaching
Standardization sample isn’t representative of population taking test

Question 5

3 types of validity

Answer 5

Content validity
Criterion-related validity
Construct validity

Question 6

Content validity

Answer 6

Degree to which the items on the test are representative of the behavior the test was designed to sample

Question 7

How content validity is determined

Answer 7

Expert judges systematically review the test content

Evaluate item relevance and content coverage

Question 8

Criterion-related validity

Answer 8

Degree to which the test is effective in estimating performance on an outcome measure

Question 9

Criterion

Answer 9

Comparison basis for a test

Question 10

Predictive validity

Answer 10

Time interval between test and criterion

Example: ACT and college performance

Question 11

Concurrent validity

Answer 11

Test and criterion are measured at same time

Example: language test and GPA

Question 12

Considerations in test-criterion studies

Answer 12

Selecting a criterion
Criterion contamination (taking test changes later performance)
Decision-theory models (circumstances surrounding test that need to be made aware of)
Validity generalization (does the test actually predict things similar to the test criteria)

Question 13

Sensitive test

Answer 13

Everyone of interest is identified, but lots of false positives

Question 14

Specific test

Answer 14

Very accurate in identification, but lots of false negatives

Question 15

Construct validity

Answer 15

Degree to which test measures what it is designed to measure

Question 16

Determining convergent validity

Answer 16

Correlate test scores with tests of same or similar construct

Question 17

Determining discriminant validity

Answer 17

Correlate test scores with tests of dissimilar construct

Question 18

Multitrait-multimethod approach to determining construct validity

Answer 18

Use multiple measures for same constructs to check for convergence as well as measures for other constructs to check for divergence

Question 19

Contrasted group study approach to determining construct validity

Answer 19

Create 2 separate and different groups: administer test and look for differences between them

Question 20

Factor analysis

Answer 20

Used to determine if test is measuring factors related to the given construct
Assign factor loadings (similar to correlation coefficients): variables should have high loadings on only 1 factor

Question 21

Evidence based on response processes

Answer 21

Is the manner of responses consistent with the construct being assessed?

Question 22

Evidence based on consequences of testing

Answer 22

If the test is thought to result in benefits, are those benefits being achieved?

Question 23

Incremental validity

Answer 23

Determines if the test provides a gain over another test

Question 24

Face validity

Answer 24

Determines if the test appears to measure what it is designed to measure
Not a true form of validity
Problem with tests high in these: can fake them

Question 25

Internal vs. external validity

Answer 25

Internal: Does the measure work in ideal conditions?
External: Does it work in the real world?