Predicting Employee Performance:The Predictor Problem

Question 1

To be effective as predictors of job performance, tests and other techniques must have acceptable levels of both reliability and ______. In Industrial/Organizational Psychology, organizations typically use predictors to predict or estimate scores or status on a criterion(performance) measure, a predictor’s criterion-related validity is usually of particular interest.

Answer 1

Validity.

Question 2

Describe the steps in Evaluating Criterion-Related Validity?

Answer 2

Step#: Action: Description

1. Conduct a Job Analysis: Identify the knowledge, skills, abilities, and other characteristics required for successful job performance
2. Select or develop the predictor: Select or develop a predictor that measures the attributes identified by the job analysis
3. Administer the predictor and criterion: Administer the predictor to a sample of job applicants (predictive validity) or current employees (concurrent validity); obtain criterion information for all individuals in the sample
4. Correlate predictor and criterion scores: Calculate the criterion-related validity coefficient to determine if there is a statistically significant relationship between predictor and criterion scores.
5. Check for adverse impact: Determine if the predictor will have adverse impact-i.e., if use of the predictor will discriminate against a legally protected class of individuals
6. Evaluate incremental validity: Determine if use of the predictor will increase the proportion of correct decisions
7. Cross-validate: Perform steps 3 through 6 with a new sample. Because all of the chance factors that maximized the original validity coefficient are not present in the second sample, the cross-validation coefficient is usually smaller than the original coefficient. This phenomenon is referred to as shrinkage.

Question 3

When does adverse impact happen?

Answer 3

It occurs when use of a selection test or other employment procedure results in discrimination against members of a legally protected (minority) group than for the majority group

Question 4

Describe the 80%(Four-Fifths) Rule and how to its used?

Answer 4

It is used to determine if adverse impact is occurring. EX: The hiring rate for the majority group is multiplied by 80% to determine the minimum hiring rate for the minority group. If use of a selection test results in a 50% hiring rate for African Americans and a 90% rate for Whites, the test would be having an adverse impact for African Americans because 90% times 80% is 72%, which is greater than the hiring rate for African Americans. A shortcoming is that the 80% Rule can be used only to determine if a person has been discriminated against only when he/she is a member of a specific group. For a finding of adverse impact, the 80% Rule requires that a pattern of discrimination be established.

Question 5

What are 2 causes of adverse impact?

Answer 5

Differential Validity: when a measure is valid for one group but is not valid (or is significantly less valid) for another group). The solution is to use different predictor cutoff scores for the groups.
Unfairness: members of the minority group consistently score LOWER ON A PREDICTOR but perform approximately the same on the criterion as members of the majority group.

Question 6

What is incremental validity and how does it improve an employer’s accuracy in making selection decisions?

Answer 6

Incremental validity refers to the increase in decision-making accuracy an employer will achieve by using the predictor to make selection decisions. A validity coefficient of .20 or .30 can increase decision making accuracy and depends on other factors such as selection ratio and base rate.

Question 7

Describe how base rate and selection ratio impact incremental validity?

Answer 7

Selection ratio is the ratio of job openings to job applicants.
EX: 1:50 one job opening for every 50 applicants
A low selection ratio is preferred as it allows employer to be selective when making hiring decisions and employer can raise the predictor cutt off score and reduce the risk of false positives (individuals who score high on the predictor but low on the criterion)

Base rate: is the percent of employees who are performing satisfactorily without use of the proposed predictor and ranges in value from 0 to 1.0. Moderate base rate close to .50 are associated with the greatest incremental validity.
Base rate is high, current selection procedure is adequate.
Base rate is low, suggests that something other than selection procedure is the problem and is selecting worst applicants and they need adequate training.

Question 8

When are multiple predictor scores combined to facilitate job selection?

Answer 8

Multiple Regression: predictor scores are weighted and summed to yield an estimated criterion score, with each predictor’s weight being determined by it’s correlation with the criterion and with the other predictors.

Multiple Cuttoff: is noncompensatory or a minimum score on each predictor must be obtained before a job applicant will be considered for selection. Selecting only people who score above a minimum cutoff point on all predictors and then using the multiple regression equation to predict each selected person’s criterion score.

Multiple (Successive Hurdles: Predictors re administered one at a time in a predetermined order, with each predictor being administered only if the previous one has been successfully passed or completed. An advantage, is that it saves time and money since all of the predictors are not administered to all individuals.

Question 9

What are the 8 main predictors commonly used in organizations for selection and other employment decisions?

Answer 9

Objective Tests, application forms, interviews, and work samples.

1. General Mental (Cognitive) Ability Tests: (GMA) tests consistently produce the HIGHEST validity coefficients across jobs and job settings. average correlation coefficients of .51 between GMA and job performance and .56 between GMA and performance in job training programs.
2. Job Knowledge Tests: Good predictors of job performance, with validity coefficients similar to or even larger than for measure of GMA. Unlike GMA tests, measures of job knowledge are job specific and are useful ONLY when applicants have had previous job training or experience.
3. Personality tests: Useful predictors in organizational settings, studies not consistent w/regard to relative validities of global and specific personality traits. Big Five-Conscientiousness has been found to be BEST PREDICTOR of Job Performance across different jobs, job settings, and criterion measures.
4. Interest Tests: Premise that applicants whose interest profile resembles those of successful employees will perform best on the job. However, usefulness of these tests as selection tools is limited because they are susceptible to faking and because they have been found to be MORE VALID for Predicting JOB CHOICE, Satisfaction, and Persistence than job success.
5. Bio data(Biographical): as a selection tool is based on assumption that the best predictor of future behavior is past behavior, and this has been confirmed by research and among most valid predictors across organizations, occupations, and samples. The Validity of biodata is maximized when items are derived empirically. EX: The Biographical Information Blank (BIB) a type of empirically derived form has multiple choice questions that lack face validity and do not look like they are related to job performance. As a consequence, job applicants may resist answering items because they perceive them to be an invasion of privacy.
6. Interviews: Most commonly used method for obtaining information about job applicants, generally have lower levels of reliability and validity than other selection techniques. Can be improved by: Training interviewers in observation and interpersonal skills, having a structured interview, asking sam questions to all applicants, including a measure of general mental ability, behavior description interviews-actual situations that occurred in applicant’s past are more valid than situational interviews(hypothethical situations), and maybe multiple interviewers.BUT structured interviews are most reliable, valid, cost-effective when entail a single interviewer.
7. Work(Job) Sampes: require an individual to perform tasks similar or identical to those actually performed on the job and are most commonly used to select applicants for jobs requiring clerical, mechanical, or technical skills.Reaslistic Job preview-may include written description of the job, video, discussion with current workers so there is no disillusionment with the job and avoid turnover.
8. Assessment Centers:Used to evaluate managerial-level personnel to determine if lower-level managers should be promoted to higher-level positions or to identify a manager’s needs for training and personal development. Participants in an assessment center are evaluated by a team of evaluators on a number of dimensions (e.g., leadership, oral and written communication, decision-making, flexibility, cooperativeness) with a variety of techniques including structured interviews, written tests, and situational tests (work samples). The in-basket test is best situational test.It requires participants to take action on letters, memos, brief reports, that are typical of those actually encountered by managers. Leaderless group discussion-involves 5-6participants work together without an assigned leader to solve a job-related problem or discuss a job-related issue.