Week 6 - Cut Off Scores Flashcards

1
Q

Biddle, R. E. (1993). How to set cutoff scores for knowledge tests used in promotion, training, certification, and licensing. Public Personnel Management, 22, 63-79.

A
  • Cutoff scores in knowledge tests delineate pass or fail for promotions, certifications, and licensing.
  • Litigation influences cutoff setting processes, emphasizing job analysis and adverse impact analysis.
  • The Angoff Method calculates minimum competency scores using Subject Matter Expert estimates, modified by statistical considerations.
  • The Uniform Guidelines on Employee Selection Procedures provide vague guidance on setting cutoff scores.
  • Legal and professional considerations influence the determination of cutoff scores, including job relatedness and the minimization of adverse impact.
  • Statistical and practical significance in rate differences and pool differences guide the evaluation of adverse impact in test outcomes.
  • Recommendations for setting cutoffs include using Subject Matter Expert opinions, considering job relatedness, consequences of not knowing test information, and statistical adjustments to mitigate adverse impact.
  • The Supreme Court and Uniform Guidelines influence the process, emphasizing job relatedness, adverse impact analysis, and legal defensibility.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Cascio, W.F., Alexander, R.A., & Barrett, G.V. (1988). Setting cutoff scores: Legal, psychometric, and professional issues and guidelines. Personnel Psychology, 41(1), 1-24.

A
  • Cutoff scores are widely used in personnel decisions, but guidance on their derivation and use is dispersed across legal, psychometric, and professional fields.
  • This paper critically analyzes and integrates information from these diverse literatures to clarify the use and misuse of cutoff scores.
  • It examines alternative methods for setting cutoff scores in the context of legal precedents, psychometric principles, and professional guidelines.
  • The paper suggests directions for future research to address gaps in understanding about cutoff scores.
  • The authors conclude with recommendations for acceptable professional practice regarding the establishment and application of cutoff scores.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Cizek, G. J. (2006). Standard setting. In S. M. Downing and T. M. Haladyna (Eds.). Handbook of test development (pp. 225-258). Lawrence Earlbaum.

A
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Ock & Oswald (2018) The Utility of Personnel Selection Decisions

A

The document by Ock & Oswald (2018) compares the utility of compensatory and multiple-hurdle selection models in personnel decisions. It highlights:

  • Compensatory selection models generally offer higher reliability and utility than multiple-hurdle models.
  • Despite higher costs, compensatory models are preferable for their overall effectiveness in selecting high-performing applicants.
  • Multiple-hurdle models may be favored for cost and time efficiency, especially in less selective contexts.
  • The study uses Monte Carlo simulations to illustrate trade-offs between reliability, cost, and utility in different selection scenarios.
  • Results support compensatory models for higher complexity jobs where the return on selection accuracy is greatest.
  • In lower complexity jobs or when selection costs are a concern, multiple-hurdle models may offer sufficient utility.
  • This study provides insights into the strategic choice between selection models based on job complexity, cost considerations, and expected utility.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly