Week 6 - Cut Off Scores Flashcards
Biddle, R. E. (1993). How to set cutoff scores for knowledge tests used in promotion, training, certification, and licensing. Public Personnel Management, 22, 63-79.
- Cutoff scores in knowledge tests delineate pass or fail for promotions, certifications, and licensing.
- Litigation influences cutoff setting processes, emphasizing job analysis and adverse impact analysis.
- The Angoff Method calculates minimum competency scores using Subject Matter Expert estimates, modified by statistical considerations.
- The Uniform Guidelines on Employee Selection Procedures provide vague guidance on setting cutoff scores.
- Legal and professional considerations influence the determination of cutoff scores, including job relatedness and the minimization of adverse impact.
- Statistical and practical significance in rate differences and pool differences guide the evaluation of adverse impact in test outcomes.
- Recommendations for setting cutoffs include using Subject Matter Expert opinions, considering job relatedness, consequences of not knowing test information, and statistical adjustments to mitigate adverse impact.
- The Supreme Court and Uniform Guidelines influence the process, emphasizing job relatedness, adverse impact analysis, and legal defensibility.
Cascio, W.F., Alexander, R.A., & Barrett, G.V. (1988). Setting cutoff scores: Legal, psychometric, and professional issues and guidelines. Personnel Psychology, 41(1), 1-24.
- Cutoff scores are widely used in personnel decisions, but guidance on their derivation and use is dispersed across legal, psychometric, and professional fields.
- This paper critically analyzes and integrates information from these diverse literatures to clarify the use and misuse of cutoff scores.
- It examines alternative methods for setting cutoff scores in the context of legal precedents, psychometric principles, and professional guidelines.
- The paper suggests directions for future research to address gaps in understanding about cutoff scores.
- The authors conclude with recommendations for acceptable professional practice regarding the establishment and application of cutoff scores.
Cizek, G. J. (2006). Standard setting. In S. M. Downing and T. M. Haladyna (Eds.). Handbook of test development (pp. 225-258). Lawrence Earlbaum.
Ock & Oswald (2018) The Utility of Personnel Selection Decisions
The document by Ock & Oswald (2018) compares the utility of compensatory and multiple-hurdle selection models in personnel decisions. It highlights:
- Compensatory selection models generally offer higher reliability and utility than multiple-hurdle models.
- Despite higher costs, compensatory models are preferable for their overall effectiveness in selecting high-performing applicants.
- Multiple-hurdle models may be favored for cost and time efficiency, especially in less selective contexts.
- The study uses Monte Carlo simulations to illustrate trade-offs between reliability, cost, and utility in different selection scenarios.
- Results support compensatory models for higher complexity jobs where the return on selection accuracy is greatest.
- In lower complexity jobs or when selection costs are a concern, multiple-hurdle models may offer sufficient utility.
- This study provides insights into the strategic choice between selection models based on job complexity, cost considerations, and expected utility.