Validity

Question 1

Validity

Answer 1

Degree to which evidence and theory support the interpretations of test scores for proposed uses.

• Not about the test itself but about the construct
• Not an all-or-none issue, it’s a degree
• Based on solid emperical evidence/theory

Question 2

Construct validity

Answer 2

Degree to which test scores can be interpreted as reflecting a particular psychological construct.

5 facets/measurement:
- Test content
- Response processes
- Internal structure
- Associations w other variables
- Consequence of use

Question 3

Test content

Answer 3

The nature of the construct should dictate the content for the test.
- the match between the actual content of the test and what should be included in the test
- evidence is expert rating of test content

Threats:
- construct-irrelevant content
- construct underrepresentation; not the full scope
- no face validity

Question 4

Internal structure

Answer 4

Dimensionality in which the parts of the test are related to each other.
- Number of factors should match the number of dimensions in a construct
- Associations among factors should match
- The meaning of each factor has to match actual components
- Evidence is factor analysis

Question 5

Response processes

Answer 5

Match between process actually used in the measurement and the one they should use
- Direct evidence; questionnaire, which thought process used
- indirect evidence: eye tracking, item-total cor.

Threat:
- Poorly disigned items (misinterpretation)
- Respondents; guessing/social desireable

Question 6

Associations w other variables

Answer 6

The measure’s actual association w other measures and the association the test should have w other measures
- Convergent evidence; correlation with measures of related construct
- Discriminant evidence’; equally uncorrelated w measures they shouldn’t correlate with

!! convergent is aboit associations w other constructs that you expect from theory. Criterion, concurrent and predictive validity are ab out the relation with some other VARIABLE, like age, GPA or income.

Question 7

Criterion, concurrent and predictive validity

Answer 7

Criterion validity; the criterion used for the test, eg the cut-off score for admission to uni, should be meaningfully correlated w what you should know

Concurrent validity; correlatioon w variables measured at the same time. Eg eindexamen compared to rest of the grades/;

Predictive validity; Correlation w variables measured in future time point, Eg eindexamen vs degrees in first year of uni

Question 8

Consequence of testing

Answer 8

Intended effect vs actual effect
- Eg when hiring new worker; You want the assessment to maximize probability of good outcome vs bad one.
- Differential output on group must be taken into account
- Possibility that test use might have effect on organizational systems.