W3 - reliability and validity

Question 1

Types of validity

Answer 1

• ecological validity
• face validity

Validity evidence:

• content validity
• criterion validity
• construct validity

Question 2

Face validity

Answer 2

• do the items on the test appear to the test-take to be measuring what they claim to measure
• high face validity appears to obviously measure what it claims to measure, low face validity means the intent is obscured
• lack of face validity may undermine accuracy of answers due to honesty and motivation as well as affect test completion

Question 3

Content validity

Answer 3

• how adequately do the items represent the construct domain (universe)?
• how comprehensive is the coverage of the items in the test?
  e. g. mid trimester exam. does it draw on weeks 1-4 or only 2+3?

Question 4

Criterion-related validity

Answer 4

• how well does our test correlate with external criterion (or outcome)
• what is the tests predictive power?
  e. g. does employee screening test predict actual job performance?

Concurrent and predictive validity

Question 5

Construct validity

Answer 5

• how well does our test correlate with other similar measures
• this can be other scales or constructs that are theoretically related (nomological network) or other evidence -> convergent and divergent validity

Question 6

Concurrent validity

Answer 6

• type of criterion-related validity
• compare test score with criterion measured at the same time
  e. g. is test of personality disorders consistent with the diagnosis made from structured clinical interview

Question 7

Predictive validity

Answer 7

• type of criterion-related validity
• compare test score with a criterion at some point in the future
  e. g. does OP test score predict future GPA performance at university

Question 8

Test bias

Answer 8

systematically prevents accurate and impartial measurement

• Measurement Bias
• > Intercept bias
• > Slope bias
• > Rating error (leniency errors, severity errors, central tendency error vs ranking)

Question 9

Test fairness

Answer 9

is the extent to which a test is used in an impartial, just, and equitable way
E.g., not using the test for the correct target population