Utility - Doc Joyce Discussion Flashcards
this term refers to the usefulness or practical value of testing to improve efficiency
test utility
What is psychometric soundness?
the reliability and validity of a test
what does an index of utility tell us?
an index of utility can tell us something about the practical value of the information derived from scores on the test
if a test is psychometrically sound, is it automatically “useful”?
no, a test can be psychometrically sound but have little utility
what is cost in the context of test utility?
cost refers to disadvantages, losses, or expenses in both economic and noneconomic terms.
what is benefit in the context of test utility?
benefit refers to profits, gains, or
advantages in both economic and noneconomic terms.
this term may be be broadly defined as a family of techniques that entail a cost–benefit analysis designed to yield information relevant to a decision about the usefulness and/or practical value of a tool of assessment
is an umbrella term covering various possible methods, each requiring various kinds of data to be inputted and yielding various kinds of output
utility analysis
this type of utility analsis instrument can provide an indication of the likelihood that a testtaker will score within some interval of scores on a criterion measure—an interval that may be categorized as “passing,” “acceptable,” or “failing.”
expectancy table/chart (scatterplot and analysis regression)
what does a hit (true positive) mean?
a test score correctly identifies the condition of interest and implies that a qualified employee was hired
what does a correct rejection (true negative) mean?
the test score correctly identifies the absence of the condition of interest and implies that an unqualified employee was not hired
what does a miss (false negative) mean?
the test score failed to identify a trait or condition of interest and implies that a qualified employee was not hired
what does a false alarm (false positive) mean?
the test score incorrectly identifies the condition of interest and implies that a unqualified employee was hired
what is base rate (prevalence)?
base rate is the proportion of individuals with the condition of interest
this term refers to an estimate of the benefit (monetary or otherwise) of using a particular test or selection method.
utility gain
this type of utility analysis instrument provides an estimate of the extent to which inclusion of a particular test in the selection system will improve selection.
provides the personnel officer with
an estimate of how much using the test would improve selection over existing methods.
determines the increase over current procedures
Taylor-Russell Tables
this term is a numerical value that reflects the relationship between the number of people to be hired and the number of people available to be hired
For instance, if there are 50 positions
and 100 applicants, then the ________ _____ is 50/100, or .50
selection ratio
what is selection ratio?
selection ratio is the proportion of individuals indicating the presence of the condition of interest
this utility analysis instrument provided an indication of the difference in average criterion scores for the selected group as compared with the original group in response to the potential problems of the Taylor-Russell tables
the use of this entails obtaining the difference between the means of the selected and unselected groups to derive an index of what the test (or some other tool of assessment) is adding to already established procedures.
determines the increase in average score on some criterion measure
Naylor-Shine tables
this term refers to the percentage of people hired under the existing system for a particular position
For example, a firm employs 25 computer programmers and 20 are considered successful, the ____ ____ would be .80
base rate
what are the limitations of the Taylor-Russell tables?
- if job performance levels off at some point in time, then the Taylor-Russell tables would be inappropriate no matter how high the score is
- the potential difficulty of identifying a criterion score that separates “successful” from “unsuccessful” employees
this type of utility analysis instrument is used to calculate the dollar amount of a utility gain resulting from the use of a particular selection instrument under specified conditions
Brogden-Cronbach-Gleser
this utility analysis instrument is also used to predict performance and is under the expectancy tables or charts category
regression analysis
