01b_Criterion Measures Flashcards
Steps in Evaluating Criterion-Related Validity
Job analysis to select predictor and criterion
Administer predictor and criterion, corrleate Scores
Check for adverse impact
Evaluate incremental validity
Cross-validate
Evaluating Criterion-Related Validity:
Adverse impact
When a procedure results in a substantially different selection, placement, or promotion rate for members of a certain group
80% rule
Determination of Adverse Impact
Multiply majority group hiring rate by 80%
= minimum hiring rate for the minority group
Factors that Increase likelihood of Adverse impact:
Differential validity
Difference in validity coefficients for subgroups
Correlations in one or both groups are significantly different from zero
Factors that Increase likelihood of Adverse impact:
Unfairness
When lower predictor scores are not related to differences in scores on the criterion
Incremental validity
Increase in decision-making accuracy by using the predictor to make selection decisions
Validity coefficient of .20 or .30 can still increase decision-making accuracy
Selection Ratio
Job openings : Job applicants
e.g. 1:50 = 1 job for every 50 applicants
Low selection ratio is preferred by employers since it is more selective
Base rate
Percent of employees performing satisfactorily without use of the proposed predictor
Range of 0 to 1.0
Why are moderate base rates associated with greatest incremental validity?
Because a new predictor will have little effect when base rates are very high or very low
base rate = naturally occurring performance, therefore extremes are less affected by new predictors
Taylor-Russell Tables
Estimates percent of new hires that will be successful
True or False?
Multiple regression is compensatory:
exceptional performance on one predictor can offset poor performance on another predictor.
True
Combining Predictors:
Multiple cutoff method
Non-compensatory
Minimum score on each predictor must be obtained before someone is considered
*Can be used as a preselection process before implementing multiple regression
Combining Predictors:
Multiple Hurdles
Predictors administered one at a time in a predetermined order
Proceed only if the previous predictor has been successfully met
Predictor that consistently produces the highest validity coefficients across jobs and job settings
General Mental Ability Tests (GMA)
aka Cognitive Ability Tests
Job Knowledge Tests:
Comparison with GMA
Good predictors of job performance, similar to GMA
However, measures are job specific, and more relevant for applicants with previous job training/experience
Interest Inventories:
Limitations
Susceptible to faking
Do not predict Job Sucess
More valuable for predicting job choice, satisfaction and persistence
Biographical data
Best predictor of future behavior is past behavior
Biographical Information Blank (BID)
Assessment of job history and family background
Lacks face validity
Applicants may resist answering items because of perceived invasion of privacy
Interviews are the most common method for obtaining information.
What is their main limitation?
Interviews generally have lower levels of reliability and validity
Interviews:
How to maximize Accuracy
Training in observation and interpersonal skills
Interviews:
How to maximize Reliability and Validity
Structured interview
Standardized questions
Combine interview with GMA
Single interviewer instead of a panel
True or False?
Hypothetical situation interviews are more valid than behavior-descriptive interviews that explore actual situations that occurred in applicant’s past.
False.
Behavior description interviews are more valid
What is a primary cause of job turnover?
Disillusionment from mismatch of expectations and reality
Assessment Centers:
Primary Purpose
Evaluation of managerial-level personnel
Determine whether to promote lower-level managers
In-Basket Test
Situational test designed to assess managerial skills
Leaderless Group Discussion
Situational Test
Five or six participants work together without an assigned leader to solve a job-related problem
Taylor-Russell Tables:
Components
Selection ratio
Base Rate
Predictor Validity coefficient
e.g., low selection rate + moderate base rate increases likelihood that predictor with low validity can still improve decision-making accuracy
Realistic Job Previews
Technique used to reduce unrealistic expectations
Goal is to increase job satisfaction/commitment
