VALIDITY Flashcards
a judgement or estimate of how well a test measures what it purports to measure in a particular context
a. Utility
b. Reliability
c. Validity
c. Validity
the process of gathering and evaluating evidence about validity.
a. Validation
b. Content Validity
c. Face Validity
a. Validation
may yield insights regarding a particular population of test takers as compared to the norming sample described in a test manual.
a. Validation
b. Local validation studies
c. construct validity
b. Local validation studies
What are the three categories that often conceptualize validity?
- Content validity
- Criterion-related validity
- Construct validity
What is the Trinitarian view?
Classic Conception of Validity
- Content
- Criterion
- Construct
this measure of validity is based on an evaluation of the subjects, topics, or content covered by the items in the test.
a. content validity
b. criterion-related validity
c. construct validity
a. content validity
this measure of validity is obtained by evaluating the relationship of scores obtained on the test to scores on other tests or measures
a. content validity
b. criterion-related validity
c. construct validity
b. criterion-related validity
How scores on the test relate to other test scores and measures.
a. content validity
b. criterion-related validity
c. construct validity
c. construct validity
a judgment concerning how relevant the test items appear to be.
a. Validation
b. Face Validity
c. Test blueprint
b. Face Validity
a judgement of how adequately a test samples behavior representative of the universe of behavior that the test was designed to sample.
a. Content validity
b. test blueprint
c. content validity ratio
a. Content validity
Necessary to consider a content-valid measure of test
a. clinical setting
b. industrial setting
c. educational setting
c. educational setting
Give examples of topics where it may have different validations from different people/places.
Politics, History,Cultures
a plan regarding the types of information to be covered by the items, the number of items tapping each area of coverage, the organization of the items in the test, etc.
a. content validity
b. test blueprint
c. content validity ratio
b. test blueprint
measures agreement among raters regarding how essential an individual test item is for inclusion in a test
a. content validity
b. test blueprint
c. content validity ratio
c. content validity ratio
Values of content validity ration can range from
-1 to +1
True or False
Closer to +1 = majority of experts agree there is an association between the item and the domain
True
the standard against which a test or a test score is evaluated.
a. Norm
b. Score
c. Criterion
c. Criterion
What are the characteristics of an adequate criterion?
- RELEVANT to the matter at hand
- VALID for the purpose for which it is being used
- UNCONTAMINATED
A judgment of how adequately a test score can be used to infer an individual’s most probable standing on some measure of interest (i.e., the criterion).
a. criterion-relation validity
b. concurrent validity
c. predictive validity
a. criterion-relation validity
An index of the degree to which a test score is related to some criterion measure obtained at the same time.
a. criterion-relation validity
b. concurrent validity
c. predictive validity
b. concurrent validity
An index of the degree to which a test score predicts some criterion, measure.
a. criterion-relation validity
b. concurrent validity
c. predictive validity
c. predictive validity
If a criterion contamination occurs, should you continue your study?
No, because the study cannot be taken seriously. There are no methods or statistics to correct for such contamination.
“How well does Test A compare with Test B?” Which of the test is used as the validating criterion?
Test B
percentage of people hired under the existing system for a particular position extent to which a particular trait, behavior, characteristic or attribute exists in the population expressed in proportion;
a. Hit rate
b. Miss Rate
c. Base Rate
d. Selection Ratio
c. Base Rate
Proportion of people a test accurately identifies a possessing/exhibiting a particular trait, behavior/ characteristic/attribute
a. Hit rate
b. Miss Rate
c. Base Rate
d. Selection Ratio
a. Hit Rate
Proportion of people the test fails to identify as having/ not having a particular characteristic/ attribute
a. Hit rate
b. Miss Rate
c. Base Rate
d. Selection Ratio
b. Miss Rate
numerical value that reflects the relationship between the number of people to be hired and the number of people available to be hired;
a. Hit rate
b. Miss Rate
c. Base Rate
d. Selection Ratio
d. Selection Ratio
Miss Rate can be subdivided into:
False Positive (Type 1) and False Negative (Type 2)
a miss wherein the test predicted that the examinee did possess the particular characteristic/ attribute being measured when the examinee did not.
a. Type 1 Error
b. Type 2 Error
a. Type 1 Error (False Positive)
a miss wherein the test predicted that the examinee did not possess the particular characteristic/ attribute being measured when the examinee did.
a. Type 1 Error
b. Type 2 Error
b. Type 2 Error (False Negative)
A correlation coefficient that provides a measure of the relationship between test scores and scores on the criterion measure.
a. Validity coefficient
b. Incremental validity
a. Validity coefficient
The degree to which an additional predictor explains something about the criterion measure that is not explained by predictors already in use
a. Validity coefficient
b. Incremental validity
B. Incremental validity
Commonly used correlation coefficient to determine the validity between the two measures.
a. Pearson
b. Spearman rho
c. Chi-square
a. Pearson
Correlation coefficient that is used for ordinal values
a. Pearson
b. Spearman rho
c. Chi-square
b. Spearman rho
Correlation coefficient that is used for nominal values
a. Pearson
b. Spearman rho
c. Chi-square
c. Chi-square
Correlation coefficient that is used for ratio/interval values
a. Pearson
b. Spearman rho
c. Chi-square
a. Pearson
How high should a validity coefficient be for a user or a test developer to infer that the test is valid?
There are no rules. However, Cronbach & Glesser said that validity coefficients need to be large enough and high enough to allow accurate decisions.
A quantitative estimate of incremental validity can be obtained using a statistical procedure called ___________
hierarchical regression
Predictive validity that can measure academic success in college
a. Validity coefficient
b. Incremental Validity
b. incremental validity
Judgement about the appropriateness of inferences drawn from test scores regarding individual standings on a construct
a. construct validity
b. criterion-related validity
a. construct validity
True or False
If a test is a valid measure of a construct, then high scorers and low scorers should behave as theorized.
True
An informed, scientific idea developed or hypothesized to describe or explain a behavior.
a. Attitude
b. Theory
c. Construct
c. Construct
True or False
Constructs are observable
False
Constructs are unobservable
How uniform a test is in measuring a single concept.
a. Evidence of homogeneity
b. Evidence of changes with age
c. Evidence of pretest - posttest changes
d. Evidence from distinct groups
a. Evidence of homogeneity
Some constructs are expected to change over time (e.g., reading rate).
a. Evidence of homogeneity
b. Evidence of changes with age
c. Evidence of pretest - posttest changes
d. Evidence from distinct groups
b. Evidence of changes with age
Test scores change as a result of some experience between a pretest and a posttest (e.g., therapy).
a. Evidence of homogeneity
b. Evidence of changes with age
c. Evidence of pretest - posttest changes
d. Evidence from distinct groups
c. Evidence of pretest - posttest changes
Scores on a test vary in a predictable way as a function of membership in some group.
a. Evidence of homogeneity
b. Evidence of changes with age
c. Evidence of pretest - posttest changes
d. Evidence from distinct groups
d. Evidence from distinct groups
Scores on the test undergoing construct validation tend to correlate highly in the predicted direction with scores on older, more established tests designed to measure the same (or a similar) construct.
a. Convergent evidence
b. Discriminant evidence
a. Convergent evidence
Validity coefficient showing little relationship between test scores and/or other variables with which scores on the test should not theoretically be correlated
a. Convergent evidence
b. Discriminant evidence
b. Discriminant evidence
The matric or table that results from correlating variables (traits) within and between methods.
a. convergent validity
b. multitrait-multimethod matrix
b. multitrait-multimethod matrix
the correlation between measures of the same trait but different methods
a. convergent validity
b. discriminant validity
c. method variance
a. convergent validity
the correlations of different traits via different methods are near zero
a. convergent validity
b. discriminant validity
c. method variance
b. discriminant validity
Correlations of different traits via the same method and the similarity of scores is due to the use of the same method
a. convergent validity
b. discriminant validity
c. method variance
c. method variance
Class of mathematical procedures designed to identify specific variables on which people may differ.
a. convergent evidence
b. discriminant evidence
c. factor analysis
c. factor analysis
occurs when the use of a predictor results in consistent underprediction or overprediction of a specific group’s performance or outcomes.
a. intercept bias
b. slope bias
a. intercept bias
occurs when a predictor has a weaker correlation with an outcome for specific groups
a. intercept bias
b. slope bias
b. slope bias
A factor inherent in a test that systematically prevents accurate, impartial measurement.
a. bias
b. rating error
c. halo effect
d. fairness
a. bias
numerical or verbal judgement that places a person or an attribute along a continuum identified by a scale of numerical or word descriptions known as rating scale
rating
A judgment resulting from the intentional or unintentional misuse of a rating scale.
a. bias
b. rating error
c. halo effect
d. fairness
b. rating error
an error in rating that arises from the tendency on the part of the rater to be lenient in scoring, marking and or grading
a. leniency error
b. centrality tendency error
c. severity error
a. leniency error
Movie critics who pan just about everything they review may be guilty of ______
a. leniency error
b. centrality tendency error
c. severity error
C. severity error
the rater exhibits a general and systematic reluctance to giving ratings at either the positive or negative extreme
a. leniency error
b. central tendency error
c. severity error
b. central tendency error
To overcome what might be termed restriction-of-range rating errors is to use _____
rankings
A tendency to give a particular person a higher rating than he or she objectively deserves because of a favorable overall impression.
a. bias
b. rating error
c. halo effect
d. fairness
c. halo effect
The extent to which a test is used in an impartial, just, and equitable way.
a. bias
b. rating error
c. halo effect
d. fairness
d. fairness
True or False
It is bias if the the test takers are not from the standardized sample.
False.
The sheer fact that no members of a particular group were included in the standardization sample does NOT in itself invalidate the test for use with that group
