12 Validity and reliability 1 Flashcards
What is content validity?
Whether scores represent the content area they’re supposed to represent. E.g. exam representative of course
What is criterion-related validity?
The degree to which a test correlates with one or more outcome or parallel criteria. Eg. other test, future success
What are the two types of criterion validity?
Concurrent validity - when the criterion is in the present - e.g. another test
Predictive validity - the criterion is in the future - e.g. Aptitude tests vs. future job performance
What is construct validity?
The degree to which a construct possesses a sane theoretical foundation, which is able to be operationalised through measurable descriptors
What are the two levels of construct validity?
Theoretical - does it have a coherent definition?
Empirical - can it be measured and make predictions about reality?
What is convergence validity?
High levels of correlation between items (elements of an assessment) that make up the same construct or related constructs
Should there be high or low correlation between items on the same assessment?
High (and thus high convergence validity), because all items are supposed to be measuring the same thing.
What is discriminant validity?
Low levels of correlation between items that make up unrelated constructs. Eg. Neuroticism and psychoticism – should be close to zero
What is external validity?
The extent to which the results of a study can be generalized to other situations and to other people.
What is ecological validity (a sub-type of external validity)?
The degree to which a score reflects reality. In “ecologically valid” experiments, experimental procedures should resemble “real-world” conditions.
What is internal validity?
The degree of confidence on the nature of asymmetric (X causes Y, Y doesn’t cause X) causal relations between the measured constructs. How confident we are that X causes Y. E.g, how confident we are that high neuroticism causes high N score on Big 5.
What are asymmetric causal relations?
X causes Y, Y doesn’t cause X
Would a naturalistic interview have high internal or external validity?
High external validity
What are the two dimensions in Anderson’s (2001) spatial representation of validity?
Focus of concern:
Outcome (practical applications)
Process (theoretical applications)
Generality
Internal (validity within given setting eg. clinical setting)
External (validity in other [external] setting)
What is the “distinct-groups approach” of empirical validation of a construct?
Assessing construct validity by testing dissimilar samples. E.g. levels of personality disorder between clinically diagnosed and normals.
