UTILITY Flashcards
usefulness of some things or some process.
UTILITY
In the context of psychometrics, it pertains to the usefulness of a test.
UTILITY
Practical value of testing to improve efficiency/training program or intervention.
UTILITY
SOME FREQUENTLY RAISED UTILITY ISSUES
COST EFFICIENCY
SAVINGS IN TIME
COMPARATIVE UTILITY
CLINICAL UTILITY
DIAGNOSTIC UTILITY
PERSONNEL SELECTION
FACTORS THAT AFFECT A TEST’S UTILITY
PSYCHOMETRIC SOUNDNESS
COSTS
BENEFITS
Higher criterion-related validity of test scores for making a particular decision, the higher the utility of the test is likely to be
PSYCHOMETRIC SOUNDNESS
a reliable and valid test is a useful test
PSYCHOMETRIC SOUNDNESS
In the context of test utility, it refers to disadvantages, losses, or expenses in both economic and noneconomic terms.
COSTS
Expenditures associated with testing or not testing.
COSTS
refers to profits, gains, or advantages.
Some benefits that can be acquired for using a well-designed testing program
BENEFITS
BENEFITS THAT CAN BE ACQUIRED USING WELL-DESIGNED TESTING PROGRAM
An Increased in the quality of the workers’ performance
An increased in the quantity of the workers’ performance
A decrease in the time needed to train workers
A reduction in the number of accidents
A reduction in worker turnover
family of techniques that entails a cost-benefit analysis designed to yield information relevant to a decision about the usefulness of a tool assessment
UTILITY ANALYSIS
evaluates whether the benefits of using a test outweigh the costs
UTILITY ANALYSIS
It helps in making decisions regarding whether:
➔ one test is preferable to another test for use for a specific purpose
➔ one tool of assessment is preferable to another tool of assessment for a specific purpose
➔the addition of one or more tests to one or more tests that are already in preferable for purpose
➔ no testing or assessment is preferable to any testing or assessment
UTILITY ANALYSIS
an indication of the likelihood that a testtaker will score within some interval of scores on a criterion measure—an interval that that may be categorized as “passing”, “acceptable”, or “failing”.
EXPECTANCY DATA
WHAT DATA IS USE TO CONDUCT UTILITY ANALYSIS
EXPECTANCY DATA
It provides an estimate of the extent to which inclusion of a particular test in the selection system will improve selection
TAYLOR-RUSSEL TABLES
● It provides an estimate of the percentage of the employees hired by the use of a particular test who will be successful at their jobs, given different combinations of three variables: test’s validity, selection ratio, and base rate.
TAYLOR-RUSSEL TABLES
number of available positions ÷ total number of applicants
SELECTION RATIO
used for obtaining the difference between the means of the selected and unselected groups to derive an index of what the test is adding to already established procedures
NAYLOR-SHINES TABLE
the proportion of people that an assessment tool accurately identifies as possessing or exhibiting a particular trait, ability, behavior, or attribute
HIT RATE
The proportion of people that an assessment inaccurately identifies
MISS RATE
● a specific type of miss whereby an assessment tool falsely indicates that the testtaker possesses or exhibits a particular trait, ability, behavior, or attribute
FALSE POSITIVE
a specific type of miss whereby an assessment tool falsely indicates that the testtaker does not possesses or exhibits a particular trait, ability, behavior, or attribute
FALSE NEGATIVE
developed by Hubert Brodgen, is a formula for the utility gain resulting from the use of a particular selection instrument under specified conditions
BRODGEN-CRONBACH-GLESER FORMULA
refers to an estimate of the benefit of using a particular test or selection method
UTILITY GAIN
refers to an estimated increase in work output
PRODUCTIVITY GAIN
SOME PRACTICAL CONSIDERATION
THE POOL OF JOB APPLICANTS
THE COMPLEXITY OF THE JOB
CUT SCORE IN USE
There is a limitless supply of potential employees looking for employment
THE POOL OF JOB APPLICANTS
Some jobs have require unique skills or demand great sacrifice that there are relatively few people who would even apply, let alone be selected
THE POOL OF JOB APPLICANTS
May also vary depending on economic climate
THE POOL OF JOB APPLICANTS
Same sorts of approaches to utility analysis are put to work for positions that vary greatly in terms of complexity
COMPLEXITY OF THE JOB
reference point derived as a result of a judgement and used to divide a set of data into two or more classifications.
CUTOFF SCORE
reference point that is set based on norm-related considerations rather than on the relationship of test scores into a criterion
RELATIVE CUT SCORE
a reference point that is typically set with reference to a judgement concerning minimum level of proficiency required to be included in a particular classifications.
FIXED CUT SCORE
the use of two or more cut scores with reference to one predictor for the purpose of categorizing testtakers
MULTIPLE CUT SCORE
a cut score is in place for each predictor used.
MULTIPLE HURDLE
an assumption is made that high scores on one attribute can “balance out” or compensate for low scores on another attribute.
COMPENSATORY MODEL OF SELECTION
METHODS OF SETTING CUT SCORES
ANGOFF METHOD
IRT- BASED METHODS
BOOKMARK METHOD
this method for setting fixed cut scores can be applied to personnel selection tasks as well as to questions regarding the presence or absence of a particular trait, attribute, or ability.
ANGOFF METHOD
cut scores are typically set based on testtakers’ performance across all the items on the test.
IRT-BASED METHODS
cut scores are typically set based on testtakers’ performance across all the items on the test.
IRT-BASED METHODS
a standard setting method used to establish one or more cut scores associated with interpretable levels of performance on an assessment
BOOKMARK METHOD
