Chapter 6: Validity Flashcards
this term refers to the judgment or estimate of how well a test measures what it claims to measure within a specific context.
is based on evidence regarding the
appropriateness of inferences drawn from test scores.
validity
this term refers to a logical conclusion or deduction made from test scores.
inference
this term refers to a correlation coefficient that measures the relationship between test scores and criterion
measures
validity coefficient
this term indicates the added explanatory power of an additional predictor variable beyond those already in use.
incremental validity
what are the characteristics of a criterion?
Relevant: Pertains to the matter at hand.
Valid: Suitable for its intended use.
Uncontaminated: Avoids bias from predictor measures
what are the three broad categories of validity?
- Content Validity
- Criterion-related Validity
- Construct-related Validity
this type of validity is the estimate of the sufficiency of the variable included in the test (test blueprint: syllabus/tos)
content validity
this type of validity is an estimate of how well the test fits a hypothesized theoretical framework
construct-related validity
this type of validity is an estimate of how well the test scores relate to a specific standard
criterion-related validity
This type of validity is an estimate of how well the test’s appearance fits its intended purpose
How relevant the test items appear to be
(Questions that ask about actions vs Inkblots on The Introversion/Extraversion Test)
face validity
this type of validity is an estimate of how well the test measures what it intends to measure at the time the variable is emitted
ecological validity
for content validity, what is the source of evidence, procedure, and result?
source of evidence: test blueprint
procedure: Subject-matter experts’ review and/or Content Validity Ratio
result: more than 0.75 CVR is acceptable
for criterion-related validity, what is the source of evidence, procedure, and result?
source of evidence: relationship with a criterion (standard)
procedure: correlate with a criterion presently available or will be available in the future (concurrent or predictive)
result: correlation is more than 0.60
what is a construct?
a construct is an informed scientific idea developed or hypothesized to describe or explain behavior
what are the two types of construct-related validity?
- Convergent Validity
- Divergent (Discriminant Validity)
this type of validity is the degree to which a test correlates with other measures of the same or similar constructs
convergent validity
this type of validity is the degree to which a test does not correlate with measures of unrelated constructs
divergent (discriminant) validity
this term refers to a mathematical procedure used to identify underlying factors or dimensions on which individuals differ.
Factor Analysis
this type of factor analysis involves estimating/extracting factors, determining how many to retain, and rotating them for better interpretability
exploratory factor analysis
what are the two types of factor analysis?
- Exploratory Factor Analysis
- Confirmatory Factor Analysis
this type of factor analysis tests how well a hypothetical factor model fits actual data
Confirmatory Factor Analysis (CFA)
this term represents the degree to which a factor influences test scores, akin to a metaphorical vehicle carrying varying amounts of abilities or traits
Factor Loading
this term refers to any inherent factor in a test that systematically hinders accurate and impartial measurement, indicating systematic variation.
Test Bias
what are the types of rating errors?
- Leniency Errors
- Severity Error
- Central Tendency Error
this type of error is also known as generosity error
occurs when a rater tends to score more
leniently than warranted, inflating
ratings.
Leniency Error
this type of error is an error where the rater is overly harsh, leading to lower ratings regardless of actual performance (e.g., movie critics who consistently give negative reviews
Severity Error
what does the ranking procedure do?
A ranking procedure requires raters to measure individuals relative to one another rather than against an absolute scale.
this type of error is characterized by a rater’s reluctance to assign extreme ratings, resulting in a clustering of ratings around the middle of the scale
Central Tendency Error
what is the purpose of a ranking procedure?
This method can help mitigate rating
errors like central tendency, leniency, and
severity by forcing the rater to prioritize
individuals (e.g., ranking first, second, third).
this term refers to a cognitive bias where a rater assigns higher ratings to a rate based on a favorable impression or aspect, failing to discriminate among different behaviors or characteristics
Halo Effect
this term, in a psychometric context, refers to how impartially, justly, and equitably a test is used in practice
Fairness
what is the importance of ensuring fairness?
Ensuring fairness is crucial for the
integrity of testing processes and the validity of test results.
