Validity Flashcards
A judgment or estimate of how well a test measures what it purports to
measure in a particular context.
Validity
The agreement between a test score or measure and the
quality it is believed to measure
Validity
The process of gathering and evaluating evidence about
validity.
Validation
require professional time and know-how, and
they may be costly.
Local validation studies
are absolutely necessary when the
test user plans to alter in some way the format, instructions, language, or
content of the test.
Local validation studies
The degree to which all accumulated evidence supports the intended
interpretation of test scores for the intended purpose.
Validity
The process of gathering and evaluating evidence about
validity.
Validation
require professional time and know-how, and
they may be costly.
Local Validity Studies
This measure of validity is based on an evaluation of
the subjects, topics, or content covered by the items in the test
Content validity
This measure of validity is obtained by
evaluating the relationship of scores obtained on the test to scores on
other tests or measures.
Criterion related Validity
This measure of validity is arrived at by executing a
comprehensive analysis of:
• How scores on the test relate to other test scores and measures.
• How scores on the test can be understood within some theoretical
framework for understanding the construct that the test was
designed to measure.
Construct Validity
Validity best answers which of the following questions?
A. How consistent is the test over time?
B. How accurate are the results of the test?
C. Does the test measure what it is supposed to measure?
D. What is the variability of the test?
C
be defined as the agreement between a test score or
measure and the quality it is believed to measure. Validity is
sometimes defined as the answer to the question, “Does the test
measure what it is supposed to measure?”
Validity
Aspects of Validity
Face Validity
- Content Validity
- Criterion-Related Validity (Concurrent and Predictive)
- Construct Validity
A judgment concerning how relevant the test items appear to be.
Face Validity
is not a statistical or numerical
measure; rather, it taps whether a test “feels like” it is a reasonable
measure of its associated criterion
Face Validity
it taps whether a test “feels like” it is a reasonable
measure of its associated criterion
Face Validity
A judgment of how adequately a test samples behavior
representative of the universe of behavior that the test was designed to sample.
Content Validity
A plan regarding the types of information to be covered by
the items, the number of items tapping each area of coverage, the
organization of the items in the test
Test Blueprint
The content validity of a test varies across
Culture and Time
Developers must show evidence that the domain was systematically analyzed and concepts are covered in correct proportion
Content Validity
Four-step process:
Step 1 - Survey the domain – Defining the performance domain of interest
• Step 2 - Content of test matches the domain – Selecting a panel of
qualified experts in the content domain
• Step 3 - Specific test items match the content – Providing a structured
framework (instruction) for the process of matching item (Question) to the
performance domain (Answer)
• Step 4 - Analyze relative importance of each objective (weight) –
Collecting and summarizing the data form the matching process.
Not a real type of content validity
– A quick look at “face” value of questions
– Sometimes questions may not seem to measure the content,
but do
• How might you show content validity for an instrument that
measures depression?
Content Validity
evidence tells us just how well a test corresponds
with a particular criterion
Criterion Validity
A judgment of how adequately a test score
can be used to infer an individual’s most probable standing on some
measure of interest (i.e., the criterion).
Criterion related validity
An index of the degree to which a test score is related to some criterion measure obtained at the same time
Concurrent Validity
Does the instrument relate to another criterion now
Concurrent Validity
An index of the degree to which a test score predicts some criterion, measure.
Predictive Validity
Does the instrument relate to another criterion in the future?
Predictive Validity
is the standard against which a test or a test score is evaluated.
Criterion
Characteristic of Criterion
Relevant to the matter at hand.
• Valid for the purpose for which it is being used.
• Uncontaminated, meaning it is not based on predictor measures.
evidence comes from assessing the
simultaneous relationship between a test and criterion
Concurrent Validity
correlation coefficient that provides a measure of
the relationship between test scores and scores on the criterion measure
Vadility Coefficient
Is the relationship between variation in the criterion and our knowledge of a test score
Validity coefficient
The degree to which an additional predictor explains something about the criterion measure that is not explained by predictors already in use.
Incremental validity
Judgment about the appropriateness of inferences drawn from test
scores regarding individual standings on a construct.
Construct Validity
Scores on the test undergoing construct validation tend to correlate highly in the predicted direction with scores on older, more established tests designed to measure the same (or a similar)
construct.
Convergent Validity
Validity coefficient showing little relationship between test scores and/or other variables with which scores on the test should not theoretically be correlated.
Discriminant Validity
Class of mathematical procedures designed to identify specific variables on which people may differ.
Factor Analysis
When two things converge, they are “coming together” in a
meaningful way
Convergent Validity
Defining the validity of a test is futile if it is not also reliable
T or F
T
