Lecture 7 Flashcards
In 1927, Kelly stated that a test is valid …
if it measures what it purports to measure.
The validity concept was meainly applied to the predictive properties of tests, such as success in college or a job. This predictive validity concept worked for maximum performance tests but not for typical performance tests.
In 1955, Cronbach and Meehl said about test validity …
test validity relates to the links of the test to the construct-to-be-measured and to related constructs. This is the nomological network.
In 1989, Messick stated …
Messick stated that test validity concerns all interpretations of the test scores and decisions that are based on these scores.
What is the view of Borsboom, Mellenberg & van Heerden on test validity?
Test validity related to the causal relation between the theoretical attribute and the observed test scores.
–> validation is testing the hypothesis that the theoretical attribute has a causal effect on the observed test scores
What are the main research strategies to validate a test?
- content analysis studies
- item response process studies
- protocol analysis studies
- psychometric process model studies - experimental studies
- quasi-experimental studies
- correlational studies
What is content analysis studies?
experts study the content of the test to evaluate whether it is plausible that the test measures the construct that it should measure.
content analysis yields qualitative information on the plausibility but it is not enough for accepting the hypothesis that the test is measuring the construct.
What are item response process studies?
item response process studies yield information on test takers item response behavior. This process behavior yields information on the plausibility that the test construct causally effect the item responses.
Two types of studies can be done on item response behavior:
- protocol analysis studies
- psychometric modelling
protocols are made of test takers cognitive item response processes. Two methods are used to compose a model …
- introspection - the test takers are asked to think aloud while responding
- retrospection - where test takers are asked to report on their response processes after they answered each of the test items
–> these protocols yield qualitative information on the plausibility of the hypothesis that the test construct causally effects test takers item responses
Experimental studies have two characteristics …
- the researcher can manipulate the independent variable
2. the participants are randomly assigned to the conditions
How differs a quasi-experimental study form an experimental study …
in a quasi-experimental study, participants are not randomly assigned to the conditions of the independent variable
the multitrait-multimethod design is based on two principles …
on the principles of convergence and discrimination of measures
What is the aim of a multitrait-multimethod design?
to examine construct validity
What is convergent validation?
convergent validation means that the test scores are positively correlated with scores of the same construct form a different measurement mode
What is discriminant validation?
discriminant validation of a test means that the measurements of different constructs can be distinguished
What is predictive validity?
to what extent do scores predict criterion at a later point of time (CITO)
