L5. Validity Flashcards
validity
- all validity is model off the concept of construct validity
- the degree to which a test measures what its supposed to measure
- only the interpretation and uses of a test scores can be valid
- is a continuum of strong to weak
- qualitative and quantitative analysis of validity
contruct validity
- Construct validity refers to the degree to which test scores can be interpreted as reflecting a particular psychological construct.
- we have to use indirect measures
Content validity
-> domain sampling
-> standard measure of estimate
face validity
1: Content validity
- A test may be suggested to be associated with good content validity when the items cover the entire breadth of the construct and do not exceed its boundaries
- tough for higher-order abstract concepts eg personality needs precise definitions to create boundaries
- domain sampling: an assortment of items that represent each domain of the construct are included in the test
- by experts
- psychometrically integral
the standard measure of estimate
- degree of correspondence between short-form and a long-form
2: Face validity
- Face validity represents the degree to which the items associated with a measure appear to be related to the construct of interest.
- susceptible to response bias
- by participants
- not psychometrically integral
factorial validity
factor analysis
1: The number of dimensions measured by a test.
2: Whether the items of a test are related to the dimensions of interest.
3: Whether the dimensions of interest are related to each other.
- can be tested using various quantitative techniques.
- use factor analysis to evaluate the factorial validity of the scores derived from a test.
- There are two types of factor analysis:
1. Unrestricted factor analysis (we only cover)
2. Restricted factor analysis.
response process
- should be a close match between the psychological processes that the respondents actually use when completing a measure and the process that they should use.
- response bias eg cheating or soc des
convergent validity
consensual convergent
- It is the degree to which test scores are correlated with tests of similar constructs.
- Emotional intelligence should correlate positively with cognitive intelligence.
- when you get the corr between self report and rater report scores
discriminant validity
- the degree to which test scores are uncorrelated with tests of unrelated constructs.
- In the emotional intelligence case, there should be very little in the way of a correlation between EI scores and a personality dimension such as morningness/ eveningness.
- Constructs should not correlate with everything under the sun.
- If they do, then the construct boundaries are overly expansive.
- correlation should be low 0.2
- should never be higher than 80 or as high
concurrent validity
- when the scores from one measure correlate in a theoretically meaningful way with the scores of another measure considered to be the “gold standard”.
- Concurrent validity does not have to based on measures administered precisely at the same time.
- But the time period should be very close
- not impressive, minimum expectation
predictive validity
- Refers to the degree to which test scores are correlated with relevant variables that are measured at a future point in time
- Predictive validity evidence is very impressive
- But it is also relatively rare, because of the time and resources required to keep track of people over time.
- eg grades w future earnings
consequential validity
-it refers to the social/personal consequences associated with using a particular test.
- For example, if two tests were equally predictive of a criterion of interest, but one of the tests tended to yield scores that were biased against women, then we would consider the non-biased test to be associated with greater consequential validity.
Criterion validity
(criterion groups validity)
- The observation of an association between a psychometric measure and a relevant outcome variable, such as different groups
- For example, different levels of achievement (undergraduate student, PhD student, and Professor) on taking the psych3302 exam
- concurrent and predictive validity together kind of
- oldest form of validity
- form of construct validity
Induction-Construct Development Interplay
- measure is developed solely from an inductive perspective.
- eg measure of personality by including all of the “person-descriptive” adjectives in the dictionary. (e.g., gregarious, moody, unpredictable).
- People rate the degree to which all of the adjectives describe them.
- Then the researcher would factor analyse all of the responses to help uncover the common dimensions.
- This is how the Big Five personality model was discovered.
Measurement as Theory
- measurement is considered a fundamental theory development end in its own right.
- where the primary objective of validation research is to offer a theoretical explanation of the processes that lead up to the measurement outcome.”
- This implies that validity research that involves correlating scores with other tests is not useful.
- In this approach, constructs and well articulated theories play a primary role in measurement and in psychology more generally.
Validity vs reliability
- reliability may not be a trait that we are actually interested in
- Validity, by contrast, is directly related to the nature of the trait supposedly being assessed by the measure.
- Reliability is a property of test scores
- Validity is a property of test score interpretations
- Validity is closely tied to psychological theory (reliability is not)
- Reliability is a necessary but not sufficient condition for validity.
- In the absence of reliability, there can be no possibility of validity.
