F&B Validity Flashcards
What is meant by validity?
the degree to which evidence and theory support the interpretations of test scores entailed by the proposed uses
What part of a test is seen as valid or invalid
The interpretation of the score and its implications; not any part of the test
How is validity traditionally a three faceted concept?
Content validity- the degree to which test scores can be interpreted as as reflecting a particular psychological construct
Criterion validity - the degree to which test scores can predict specific criterion variables.
Construct validity
What five types of evidence relevant for establishing the validity of test score interpretation was published in the Standards for Educational and Psychological Testing?
Internal structure Response processes Test Content Consequences of use Associations with other variables
Name 2 threats to content validity
Construct-irrelevant content
Construct underrepresentation
What is meant by face validity?
the degree to which a measure appears to be related to a specific construct, in the judgment of non-experts, such as test takers and representatives of the legal system
Why may test validity be relevant?
The apparent meaning and relevance of a lest’s content might influence lest takers’ motivation to respond in a serious and honest manner.
What is meant by the internal structure of a test?
The way the parts of the test are related to each other (correlated items or clusters etc)
What should the internal structure of a test match if is to be internally valid?
Internal structure of the theory
What test examines the internal structure of a test?
Factor analysis
How are the results of a factor analysis displayed for “global” and for multidimensional tests
Global- scree plot (eiganvalues)
Multidimensional- orthogonal rotation
What is the facet “associations with other variables” concerned with?
The match between a measure’s actual associations with other variables and the associations is actually has with other variables
When evaluating the pattern of validity associations between a measure and other measures, what evidence should we consider? (2)
Convergent evidence- the degree to which scores are correlated to tests of related (or the same) constructs
Discriminant evidence- the degree to which test scores are uncorrelated with tests of unrelated constructs
Distinguish between concurrent validity evidence and predictive validity evidence
Concurrent validity evidence refers to the degree to which test scores are correlated with other relevant variables that are measured at the same time as the primary test of interest.
Predictive validity evidence refers to the degree to which test scores are correlated with relevant variables that are measured at a future point in time.
concurrent validity evidence and predictive validity evidence are variants of what type of what type of evidence
Convergent evidence
Name two big differences between the contemporary and traditional perspective on validity
-the contemporary perspective emphasizes the primacy of construct validity over content
validity and criterion validity
-The assertion from the contemporary perspective that the social consequences are a facet of validity
According to the contemporary view, what should criterion validity be categorised under?
Construct validity
How would an inductive view of testing validity look?
by examining the associations between test scores and a large set of potentially
important and relevant psychological variables rather than testing the hypothesis of what is is assumed to test
When would an inductive method usually be utilised?
In a research context
What is meant by the nomological network?
The interconnections between a construct and other related constructs
Name four methods for evaluating convergent and discriminant validity
Focused associations
Sets of Correlations
Multitrait-Multimethod Matrices
What is involved in focused associations?
When measures have clear relevance for very few very specific variables
What are meant by validity coefficients?
Correlations between test scores and crucial variables
What is meant by validity generalisation?
Process of evaluating a tests validity coefficients across a large set of studies, it is a form of meta-analysis- it studies a range of settings, times, situations etc