validity part 3 Flashcards
construct validity
- Construct = an abstraction that cannot be directly observed
- Construct Validity = test actually measures the construct it was designed to measure
- There is no single definitive test of construct validity
- Evidence for construct validity builds over time forming a body of evidence known as a nomological net (Cronbach & Meehl)
methods of establishing construct validity
studying group differences, conducing research to test hypotheses about the construct, demonstrating correlations with existing tests of the construct
explain studying group differences
- Do scores on the test accurately distinguish between people who are believed to have different levels of the construct?
- Example: Is there a significant difference in scores on a test of creativity between professional artists and bankers?
explain conducting research to test hypotheses about the construct
- Following an experimental manipulation, do test scores change in the direction predicted by the theory underlying the construct?
- Example: Is there a significant difference in scores on a test of assertiveness between those who have completed an assertiveness class and those placed on the waiting list?
explain demonstrating correlations with existing tests of the construct
- Are test scores correlated with other tests that measure the same construct?
- Example: Do scores on a newly-proposed intelligence test correlate with scores on the WAIS-IV?
convergent and discriminant validity
1) Convergent Validity = test correlates with a criterion with which it SHOULD correlate
2) Discriminant Validity = test does NOT correlate with criteria with which it should not correlate
- Pertains to the specificity of the test in measuring the construct
- Tests that have convergent but not discriminant validity cannot differentiate between closely related constructs
what is the multitrait-multimethod matrix
- Method for assessing convergent and discriminant validity simultaneously
- Requires two or more traits that are each measured by two or more methods
explain 6 blocks within the matrix
- 3 Monomethod Blocks (red) = different traits measured by the same method
- 3 Heteromethod Blocks (green) = different traits measured by different methods
long diagonals
reliabilities of each trait-method combination
short diagonals
-Diagonals within each heteromethod block (bold) = convergent validity (same trait measured by two different methods)
we want high correlations for these entries
other diagonal entires in heteromethod blocks (italics)
discriminant validity = different traits measured by different measures
we want low correlations for these entries
Other (non-diagonal) entries in Monomethod Blocks (underlined)
-different traits measured by the same method
- These are estimates of Method Variance = the effect of using the same method to measure different traits
- We want LOW correlations for these entries because LOW method variance means that a method CAN distinguish between different traits: HIGH Method Variance is NOT GOOD.
multitrait-multimethod matrix desirable findings
- Large entries in long diagonal = adequate reliability for our assessment measures
- Large entries in short diagonals (diagonals in Heteromethod Blocks) = strong convergent validity
- Small(-er) entries everywhere else = evidence for discriminant validity
- Small entries for off-diagonal entries in Monomethod Blocks = minimal method variance = the particular method is able to distinguish among the traits
what is factorial validity
-Factor Analysis can be used to evaluate construct validity
- Two ways of using factor analysis
- Exploratory FA
- Confirmatory FA
exploratory factor analysis
Exploratory FA = goal is to discover underlying factor structure
-Study is exploratory because no specific hypotheses are being tested
confirmatory factor analysis
- goal is to test a hypothesis about a theorized factor structure
- The experimenter groups the subtests according to the hypothesized factor structure, and then tests this hypothesized grouping against the data from factor analysis using goodness of fit statistics
goal of exploratory factor analysis
Discover the underlying factor structure (no hypotheses being tested)
goal of confirmatory factor analysis
Test a hypothesis about the factor structure
-investigator groups the subtests according
to the hypothesized factor structure, then
¥ tests this hypothesized grouping
against the data from factor analysis using
goodness of fit statistics
