Validity Flashcards
____ refers to a test’s accuracy: A test is valid when it ____ what it is ____ to ____. The intended uses for most tests fall into one of three categories, and each category is associated with a ____ ____ for establishing validity.
Validity; Measures; Intended to Measure; Different Method
The test will be used to obtain information about an examinee’s familiarity with a particular content or behavior domain: ____ ____.
Content Validity
The test will be used to determine the extent to which an examinee possesses a particular hypothetical trait: ____ ____.
Construct Validity
The test will be used to estimate or predict an examinee’s standing or performance on an external criterion: ____-____ ____.
Criterion-Related Validity
For some tests, it is necessary to demonstrate only ____ ____ of ____; for others, it is desirable to establish ____ than ____ ____. For example, if an arithmetic achievement test will be used to assess the classroom learning of 8th grade students, establishing the test’s ____ ____ would be sufficient. If the same test will be used to predict the performance of 8th grade students in an advanced high school math class, the test’s ____ and ____-____ ____ will both be of concern.
One Type of Validity; More than One Type; Content Validity; Content and Criterion-Related Validity
Note that, even when a test is found valid for a particular purpose, it might not be valid for that purpose for ____ ____. It is quite possible for a test to be a valid measure of intelligence or a valid predictor of job performance for one group of people but not for ____ ____.
All People; Another Group
A test has ____ ____ to the extent that it adequately samples the content or behavior domain that it is designed to measure. If test items are not a good sample, results of testing will be ____. Although content validation is sometimes used to establish the validity of personality, aptitude, and attitude tests, it is most associated with ____-____ ____ that measure knowledge of one or more ____ ____ and with tests designed to assess a well-defined ____ ____. Adequate content validity would be important for a ____ ____ and for a ____ (job) ____ ____.
Content Validity; Misleading; Achievement-Type Tests; Content Domains; Behavior Domain; Statistics Test; Work Simple Test
Content validity is usually “built into” a test as it is constructed through a ____, ____, and ____ ____ that involves ____ ____ the ____ or ____ ____ to be ____ and then ____ or ____ ____ that ____ that ____. Once a test has been developed, the establishment of content validity relies primarily on the ____ of ____ ____ ____. When experts agree that test items are an ____ and ____ ____ of the ____ ____, this provides evidence of the test’s ____ ____.
Systematic, Logical, and Qualitative Process; Clearly Identifying; Content; Behavior Domain to be Sampled; Writing or Selecting Items; Represent that domain; Judgment of Subject Matter Experts; Adequate and Representative Sample of the Target Domain; Content Validity
Several methods for quantifying the judgments of subject matter experts are available. For example, the ____ ____ ____ (CVR) is the ratio of the subject matter experts who rated an item as essential to defining the content domain of interest to the total number of raters.
Content Validity Ratio
Content validity must not be confused with ____ ____. ____ ____ refers to the systematic evaluation of a test by experts who determine whether test items adequately sample the relevant domain, while ____ ____ refers simply to whether or not a test “looks like” it measures what it is intended to measure. Although face validity is ____ an actual type of validity, it is a desirable feature for many tests. If a test lacks face validity, examinees may not be motivated to respond to items in an ____ or ____ manner. A high degree of face validity does not, however, indicate that a test has ____ ____.
Face Validity; Content Validity; Face Validity; Not; Honest or Accurate; Content Validity
A test is valid when it (1) ____ what it is designed to measure. There are three main forms of validity: Content validity is of concern whenever a test has been designed to measure one or more (2) ____. (3) ____ validity is important when a test will be used to measure a hypothetical (4) ____, such as achievement, motivation. intelligence, or mechanical aptitude. (5) ____ validity is of interest when a test has been designed to estimate or predict performance on another measure.
(1) accurately measures; (2) content or behavior domains; (3) Construct; (4) trait (construct); (5) Criterion-related
A test has (6) ____ validity when its items are a representative sample of the domain(s) that the test is intended to measure. Content validity is usually built into a test as it is being constructed through the selection of a (7) ____ sample of items. After a test has been developed, its content validity is checked by having (8) ____ experts evaluate the test in a systematic way.
(6) content; (7) representative; (8) subject matter
a test has been found to measure the hypothetical trait (construct) it is intended to measure, the test is said to have ____ ____. A ____ is an abstract characteristic that cannot be observed directly but must be inferred by observing its ____. Intelligence, mechanical aptitude, self-esteem, and neuroticism are all ____.
Construct Validity; Construct; Effects; Constructs
There is no ____ ____ to establish a test’s construct validity. Instead, construct validation entails a systematic accumulation of evidence showing that the test ____ ____ the ____ it was ____ to ____. The various methods used to establish this type of validity answer a slightly different question about the construct.
Single Way; Actually Measures the Construct; Designed to Measure
________________: Do scores on individual test items correlate highly with the total test score; i.e., are all of the test items measuring the same construct?
Assessing the test’s internal consistency
_____________: Do scores on the test accurately distinguish between people who are known to have different levels of the construct?
Studying group differences
___________________________: Do test scores change, following an experimental manipulation, in the direction predicted by the theory underlying the construct?
Conducting research to test hypotheses about the construct
_________________________: Does the test have high correlations with measures of the same and related traits (convergent validity) and low correlations with measures of unrelated traits (discriminant validity)?
Assessing the test’s convergent and discriminant validity
_______________: Does the test have the factorial composition it would be expected to have — i.e., does it have factorial validity?
Assessing the test’s factorial validity
Construct validity is said to be the most ____-____ of the methods of test validation. The developer of a test designed to measure a construct begins with a ____ about the ____ of the ____, which then guides the test developer in ____ ____ ____ and in choosing the ____ for ____ the ____ ____. If the developer of a creativity test believes that creativity is unrelated to general intelligence, that creativity is an innate characteristic that cannot be learned, and that creative people can be expected to generate more alternative solutions to certain types of problems than non-creative people, she would want to determine the correlation between scores on the creativity test and a measure of intelligence, see if a course in creativity affects test scores, and find out if test scores distinguish between people who differ in the number of solutions they generate to relevant problems.
Theory-Laden; Theory; Nature; Construct; Selecting Test Items; Methods for Establishing the Test’s Validity
____ and ____ ____: As noted above, one way to assess a test’s construct validity is to ____ test scores with ____ on ____ that do and do not purport to ____ the ____ ____. High correlations with measures of the same and related traits provide evidence of the test’s ____ ____, while low correlations with measures of unrelated characteristics provide evidence of the test’s ____ ____ (which is also known as divergent validity).
Convergent and Discriminant Validity; Correlate; Scores on Measures; Assess the Same Trait; Convergent Validity; Discriminant Validity
The ____-____ ____ (Campbell & Fiske, 1959) is used to systematically organize the data collected when assessing a test’s convergent and discriminant validity. The multitrait-multimethod matrix is a ____ of ____ ____, and, as its name suggests, it provides information about the ____ of ____ between ____ or ____ ____ that have each been assessed using ____ or ____ ____. When the correlations between different methods measuring the same trait are larger than the correlations between the same and different methods measuring different traits, the matrix provides evidence of the test’s ____ and ____ ____.
Multitrait-Multimethod Matric; Table of Correlation Coefficients; Degree of Association; Two or More Traits; Two or More Methods; Convergent and Discriminant Validity
____-____ ____ (“same trait-same method”): The ____-____ ____ (coefficients in italics) are reliability coefficients and indicate the correlation between a measure and itself. Although these coefficients are not directly relevant to a test’s convergent and discriminant validity, they should be ____ for the matrix to provide useful information.
Monotrait-Monomethod Coefficients; Monotrait-Monomethod Coefficients; Large
____-____ ____ (“same trait-different methods”): These coefficients (underlined coefficients) indicate the correlation between different measures of the same trait. When these coefficients are ____, they provide evidence of ____ ____.
Monotrait-Heteromethod Coefficients; Large; Convergent Validity