SSR exam 5 Flashcards
testlets
multiple items that have the same context
facet method
item construction by conceptual analysis of the construct (using exising theory/knowledge) every aspect (facet) of the construct is measured in a systematic way
validity
degree to which a test measures what it is supposed to measure.
construct validity
the degree to which test cores can be interpreted as reflecting a particular psychological construct/ the extentto which relationships between observations are aligned with the nomoloigcal network
face validity
the degree to which a measure appears to be related to a specific construct, in the judgment of nonexperts such as testtakers and representatives of the legal system
content validity
degree to which the content of a measure truly reflects the full domain of the construct for which it is being used no more and no less
internal structure of a test
the way the parts of a test are related to each other
factors
items that are highly correlated with eachother form clusters of items
convergent evidence
the degree ot which test scores are correlated with tests of related constructs
discriminant evidence
the degree to which test scores are uncorrelated with tests of unrelated constructs
concurrent validity evidence
refers to the degree to whcih test scores are correlated with ther relevant variables that are measured at the same time as the primary test of interest
predictive validity evidence
refers to the degree to which test scores are correlated with relevant variables that are measured at a future point in time
criterion validity
extent to which a measure is related to an outcome
inductive approach
proceeds by examining the associations between test scores and a large set of potentially important and relevant psychological variables
deductive approach to validity
test evaluation proceeds by deducing a particular hypothesis from the theoretical basis of a construct and then empirical evaluating the accuracy of the hypothesis
item writing guidelines
dont ask two questions in one, use independent item content (no dependency between questions), avoid items that deliberately deceive test takers, use three options unless it is easy to write plausible distractors, avoid negatives, use one option that is unambiguously the correct or best option, avoid giving clues in the correct answer(long answer),
construct validity steps to follow
- define a construct in terms of a network of relationships
- operationalize every cnstruct with a measurement
- see if the relationships between measurements are aligned with the theory
- if yes : there is support for a theory in question
interpretationist account redefine validity
as an assessment of evidence with regards to proposed interpreations of the test scores
causal account of validity
test X is valid for attribute Y when differenes in Y cause differences in X. we have to establish a causal relationship between construct and the test only then can we conclude that a test is valid
response processes
the match between the psychological process that respondents actually use when completing a measure and the process that they should use
test utility
refers to using a test as a tool
criterion
what we want to predict (good employees or good students)
predictor
the test we use to predict
predictive validit
defined as the correlation between the predictor/test score and the criterion
base rate
cut off for the criterion (when we truly know someone is competent).
percentage of correct conlcusions when not selecting TP+FN/N
selection ratio
cut off for the predictor (at what predictor score we want to select someone)
hit rate
(percentage of correct conclusions) TP+TN/N
benefit
hit rate-base rate
sensitivity
the percentage of eligible candidates that will be selected = TP/(FN+TP)
specififcity
percentage of inapt candidates that will be rejected = TN/(TN+FP)
two factors influencing the cell values “the proportion satisfactory among those selected”
predictive validity, selection ratio
increase validity
higher reliability(more items deleting bad items), use better predictor, less restriction of range
principle cmponent analysis
statistical procedure that uses an orthogonal transofmration to converta set of observations of posssibly correlated variables into a set of values of linearly uncorrelated variables called principle components
nomonoligical network
network of “meaning” surrouring a construct
validity generalization
process of evaluating a tests validity coefficients across a large set of studies
shared method variance
the correlatin is positive because it is based on two measures derived from the same source