validity part 3 Flashcards
construct validity
- Construct = an abstraction that cannot be directly observed
- Construct Validity = test actually measures the construct it was designed to measure
- There is no single definitive test of construct validity
- Evidence for construct validity builds over time forming a body of evidence known as a nomological net (Cronbach & Meehl)
methods of establishing construct validity
studying group differences, conducing research to test hypotheses about the construct, demonstrating correlations with existing tests of the construct
explain studying group differences
- Do scores on the test accurately distinguish between people who are believed to have different levels of the construct?
- Example: Is there a significant difference in scores on a test of creativity between professional artists and bankers?
explain conducting research to test hypotheses about the construct
- Following an experimental manipulation, do test scores change in the direction predicted by the theory underlying the construct?
- Example: Is there a significant difference in scores on a test of assertiveness between those who have completed an assertiveness class and those placed on the waiting list?
explain demonstrating correlations with existing tests of the construct
- Are test scores correlated with other tests that measure the same construct?
- Example: Do scores on a newly-proposed intelligence test correlate with scores on the WAIS-IV?
convergent and discriminant validity
1) Convergent Validity = test correlates with a criterion with which it SHOULD correlate
2) Discriminant Validity = test does NOT correlate with criteria with which it should not correlate
- Pertains to the specificity of the test in measuring the construct
- Tests that have convergent but not discriminant validity cannot differentiate between closely related constructs
what is the multitrait-multimethod matrix
- Method for assessing convergent and discriminant validity simultaneously
- Requires two or more traits that are each measured by two or more methods
explain 6 blocks within the matrix
- 3 Monomethod Blocks (red) = different traits measured by the same method
- 3 Heteromethod Blocks (green) = different traits measured by different methods
long diagonals
reliabilities of each trait-method combination
short diagonals
-Diagonals within each heteromethod block (bold) = convergent validity (same trait measured by two different methods)
we want high correlations for these entries
other diagonal entires in heteromethod blocks (italics)
discriminant validity = different traits measured by different measures
we want low correlations for these entries
Other (non-diagonal) entries in Monomethod Blocks (underlined)
-different traits measured by the same method
- These are estimates of Method Variance = the effect of using the same method to measure different traits
- We want LOW correlations for these entries because LOW method variance means that a method CAN distinguish between different traits: HIGH Method Variance is NOT GOOD.
multitrait-multimethod matrix desirable findings
- Large entries in long diagonal = adequate reliability for our assessment measures
- Large entries in short diagonals (diagonals in Heteromethod Blocks) = strong convergent validity
- Small(-er) entries everywhere else = evidence for discriminant validity
- Small entries for off-diagonal entries in Monomethod Blocks = minimal method variance = the particular method is able to distinguish among the traits
what is factorial validity
-Factor Analysis can be used to evaluate construct validity
- Two ways of using factor analysis
- Exploratory FA
- Confirmatory FA
exploratory factor analysis
Exploratory FA = goal is to discover underlying factor structure
-Study is exploratory because no specific hypotheses are being tested
