Validity Flashcards
Validity
- Accuracy, utility, meaningfulness
- Validation is a never-ending process, not an outcome
Measurement vs Research Design Validity
- Measurement Validity: interpretation of scores from that measure for the proposed use
- Research Design Validity: validity of research design for making various interpretations
Face Validity
- Measurement validity
- Whether test appears to evaluate the construct as judged by a lay person
Content Validity
- Measurement validity
- Whether test appears to evaluate the construct in its entirety (no gaps, no intrusions, full range) as determined by experts
Criterion-Related Validity
- Measurement validity
- Examines whether a measure behaves the way it should given an accepted theory of construct
- OBSERVABLE criterion
- Born from radical operationalism
- Two types: concurrent validity and predictive validity
Concurrent Validity
- Type of CRV, measurement validity
- Concurrent/simultaneous association between measure and criterion
- Ex: written and physical driving test (At same time)
Predictive Validity
- Type of CRV, measurement validity
- Association between measure and criterion at a later time
- Ex: SAT and college GPA
Construct Validity
- Measurement validity
- Extent to which a measure accurately assesses a target construct
- UNOBSERVABLE criterion
- Gives rise to nomological network and multi-trait multi-method matrix
Multi-trait Multi-method Matrix
- Used to evaluate construct validity
- Way to distinguish method variance from trait variance
- Distinguish variance of measures into variance due to measurement and variance due to construct (trait)
Convergent Validity
- Evaluated using MTMMM, construct validity (measurement validity)
- Extent to which a measure is associated with other measures of the same target construct
- Strong CV would show significant and high-ish diagonals on the MTMMM
Discriminant (Divergent) Validity
- Evaluated using MTMMM, construct validity (measurement validity)
- Extent to which a measure is NOT associated with measures of different constructs that are not theoretically expected to be related
Incremental Validity
- Type of CRV, measurement validity
- Extent to which a measure provides an increment in predictive efficiency over otherwise easily and cheaply available info
- Should explain variance that wasn’t explained before by other measures
- Measures treatment utility
Treatment Utility
- Measured by incremental validity
- Extent to which a measure is shown to contribute to beneficial treatment outcomes (CRV)
- Two types: manipulated assessment (not everyone gets the assessment) and manipulated use (not everyone is treated based on assessment)
Consequential Validity
- Measurement validity
- Evaluates societal consequences of the measure in the long run
Internal Validity
- Research-Design Validity
- Extent to which causal inference is justified from the research design
External Validity
- Research-Design Validity
- Extent to which findings can be generalized to the broader population and real world
- As internal validity increases, external decreases; always a trade off
Statistical Validity
- Research-Design Validity
- AKA conclusion validity
- Extent to which conclusions about the association among variables on the data are reasonable
Maximum Association Formula
max association between x and y = sqrt(reliability of x and y)
Association Attenuation & Formula
- Lower reliability = greater measurement error = weaker association = greater attenuation
- Observed association between x and y = (true association of constructs) x sqrt(reliability of x and y)
Disattenuation Concept & Formula
- Correction for attenuation because we don’t have true reliability because we don’t have an infinite sample
- True association of construct = (observed association between x and y) / (sqrt(reliability of x and y))
Monotrait Heteromethod
Convergent Validity
