Exam 3 Flashcards

1
Q

Be able to calculate, understand, and interpret z scores.

A

Z-Score = (raw score – mean) / standard deviations

  • The z-score is essentially a calculation of how many standard deviations a raw score is away from the mean
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Why are measurement and statistics so important to staffing?

A

Because talent analytics are based on scores which are obtained from measures

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What are the four scales of measurement?

A
  1. Nominal
  2. Ordinal
  3. Interval
  4. Ratio
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Nominal scale

A

exist when the scores represent simple categories but provide no additional information

  • Ex. Gender, race, or country
  • Scores only tell us which category something is from
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Ordinal scale

A

provide slightly more information, telling us about category membership and rank order

  • Ex. Forced-ranking performance assessment programs, school grades, or “top ten cities”
  • These scales don’t tell us whether the differences between each rank are the same
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Interval scale

A

account for both rank order and ensures the differences between each rank order are the same

  • Ex. Common 5-point or 7-point scales used in surveys and performance management systems
  • These scales do not have a true zero
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Ratio scale

A

possess all the characteristics of an ordinal scale, but also include a true zero point

  • Ex. Money, temperature, or weight
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Understand why scales that provide more information are more valuable for staffing.

A

the more information provided by a measure, the more insight that we have into the concept, the better distinctions we can make, and the more sophisticated analytical models can be used to make predictions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Describe how correlations are used in selection to estimate criterion-related validity?

A
  • correlation measures the relationship between 2 variables and is used to assess validity to evaluate how accurately a test measures the outcome it was designed to measure (between a predictor and outcome)
  • concurrent = present
  • predictive = future
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is selection?

A

refers to the strategies, processes, methods, and practices used to assess applicants on the job- and organizationally related KSAOs

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Why is selection strategically valuable? (S.I.C.E.E)

A
  • it shapes the nature of talent resources
  • implements the firm’s strategy to drive competitive advantage
  • changes or reinforces the company culture
  • effects of selection are difficult and costly to duplicate
  • enhances analytics and quantitative insights
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is the basic logic of selection?

A

A process to assess those applicants in order to determine who meets or exceeds qualification standards and thus should receive job offers

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

How is “selection” a bet on the future?

A

Based on the incomplete and imperfect information available today

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What is validity?

A

Accuracy- are you measuring what you’re actually trying to measure?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What are the 3 types of validity?

A
  1. criterion-related
  2. content
  3. construct
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Criterion-related validity

A
  • Empirical relationship between predictor and outcome (criterion) variables. Usually estimated using correlation/regression
  • Ex. Wanting to know if a college entrance exam will predict future academic performance… first semester GPA would serve as a criterion-related variable since it is an accepted example of academic performance.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Content validity

A
  • Expert judgment about the conceptual overlap between predictors and outcomes
  • how well a test covers all of the relevant parts of the construct it aims to measure
  • Ex. Taking a driving test to obtain your driver’s license
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Construct validity

A
  • Whether scores on some assessments represent the underlying attribute they are intended to represent.
  • Ex. A measurement of the human brain; to measure intelligence
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What are predictors?

A

(selection methods)
- Provide scores about latent talent used for predicting performance outcomes
- Ex. Sign and samples

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What is criteria?

A

Standards to measure candidates to determine whether or not they have the qualifications to perform the job

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

How are predictors and criteria related?

A

To determine the quality of hire

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

What are the 7 steps of selection?

A

1- identify desired talent
2- ensure compliance w/ legal and political guidelines
3. determine cultural influences
4. determine selection methods
5. structure the selection process to balance trade-offs
6. make selection decisions
7. evaluate the selection process

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

What are the 2 predictor methods?

A
  • sign; homogeneous
  • sample; heterogeneous
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Sign KSAOs

A

homogeneous
- cognitive ability
- knowledge and skill
- personality
- work interests, styles, and values
- biodata
- fit
- situational judgment test

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

Sample KSAOs

A

heterogeneous
- situational judgment test
- interviews
- assessment centers
- work samples
- simulations

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

Why do signs and samples matter?

A

Provides information on an associated dependent (Y-axis) variable regarding a particular outcome

Sign= usually self-reported surveys
Sample= behavioral simulations

27
Q

What does it mean to say the best predictor of future behavior is past behavior?

A

Suggests that a person’s previous actions and behaviors are a strong indicator of how they are likely to behave in the future

28
Q

What types of predictors are based on “future behavior is past behavior” principle?

A

Interviews: (behavioral and situational questions)

Simulations: (situational judgment tests, assessment centers, work samples)

29
Q

What is cognitive ability?

A

Refers to how well individuals can manipulate, process and remember data

30
Q

What does cognitive ability predict?

A
  • Job performance
  • Occupational “prestige”
  • Salary
  • Promotion rate
  • Basically, every desirable criterion of interest!
31
Q

Cognitive ability is among the highest validities of anything we use, so what is the problem with using them?

A
  • Cognitive ability often produces larger demographic mean differences than nearly any other test
  • Using the “most valid” test will result in lower diversity
32
Q

What is personality and how is it used in selection?

A

Consistency in behavior and people’s personalities are set

33
Q

Why is personality related to performance?

A

Adverse impact tends to be low

34
Q

What is the five-factor model? Be sure to describe and give examples of each trait.

A
  • Neuroticism (emotional stability) - refers to one’s level of emotional instability, anxiety, and negative emotionality
  • Extraversion - This trait refers to one’s level of sociability, assertiveness, and positive emotionality
  • Agreeableness - refers to one’s level of cooperativeness, kindness, and empathy
  • Openness to experience – willing to try new experiences
  • Conscientiousness - refers to one’s level of organization, self-discipline, and responsibility.
35
Q

What is the underlying logic of the interview?

A
  • Past behavior is the best predictor of future behavior.
  • If interviews ask Qs to gauge applicant’s level of competence on job-related KSAOs (from job analysis), then this maximizes the capture of job-relevant info and minimizes time spent on irrelevant Qs
36
Q

Why are interviews so frequently of poor quality?

A
  • Not properly developed and administered to be structured and streamlined across candidates, such as basing interview Qs on job analysis and asking the same Qs of all applicants.
  • Interviewers are not trained and process fails to use multiple interviews and interviewers
37
Q

What is interview structure?

A
  • ask same Qs for all – minimal deviations from the Qs
  • minimize use of prior info (such as interviewer knowing the details of schools attended, experience, socioeconomic factors, etc)
  • Use behavioral rating scales
  • Interviewers are trained
38
Q

How does Structure interviews relate to validity, diversity, etc.?

A

the most valid predictor method, also most accurate, consistent, legally defensible, and able to reduce bias and discrimination

39
Q

Behavioral interview questions

A

(experience-based), ask Qs about past behavior, validity .30+

Example: Tell me about a time you… (teamwork, problem solving, etc.)

40
Q

Situational interview questions

A

ask Qs about a hypothetical behavior, validity .25+

Example: Imagine you were in a situation where…How would you handle it?

41
Q

Job knowledge interview questions

A

ask Qs about specific job skills, validity .25+

Example – How do you develop a presentation using x software? Talk about processes, program-specific knowledge, etc.

42
Q

What do interviews measure?

A
  • general cognitive ability
  • knowledge and skills
  • personality, social skills
  • interests/preferences
  • org fit
  • physical abilities
43
Q

What kinds of KSAOs should we use interviews to measure?

A

Interviews should be used to measure JOB-RELEVANT KSAOs (from job analysis)

44
Q

How strongly correlated are the different types of interview questions?

A

Almost zero correlation! (between situational and behavioral)

We know behavioral interviews predict better

45
Q

What kinds of interviews do applicants/managers tend to prefer? Why?

A

tend to prefer UNstructured, so it is important to explain why a structured interview is being used

Why they prefer unstructured – feel like they can’t get to know people on a personal level, very boring to ask the same Qs over and over to different candidates

46
Q

Know (very well!) the guidelines for conducting effective interviews.

A

5 best practices:
- Base Qs on job analysis
- Ask same Qs
- Train interviewers
- Use Systematic & Behavioral scoring
- Multiple Interviewers & Interviews

47
Q

Describe the basic purpose and logic of simulations.

A

tests are SIGNS of behavior, simulations are SAMPLES of behavior.

48
Q

What is fidelity?

A

applicants complete similar/identical tasks to those required on the job

Higher fidelity = higher validity

49
Q

How does fidelity factor into our discussion of simulations?

A

lower fidelity = situational/judge
higher fidelity = work sample

50
Q

What is a situational judgment test (SJT)? Do you understand the key characteristics of SJTs, validity, etc?

A

SJT – present applicant with work-related situation, ask how they will handle it

Example: Coworkers are not doing certain tasks that are needed to get done. Would you: A) Perform tasks yourself, B) Tell your boss, C) Convince coworkers to split up the tasks. Correct answer would get +1 points, others would get 0 or -1

51
Q

What is an assessment center?

A

applicants participate in multiple exercises (avg of 5), each exercise measures multiple competences (avg of 10). Raters evaluate the competences across exercises.

52
Q

Give some examples of different assessment center methods.

A
  • In-basket
  • Leaderless group discussion
  • Roleplay
  • Mock presentation
  • Business game
53
Q

What is the distinction between exercises and dimensions?

A
  • exercises ARE the methods to elicit info about the candidate’s level of the desired behaviors/competencies
  • dimensions ARE the KSAOs/competencies
54
Q

What are the most common exercises used in selection?

A
  • work samples
  • gamification
55
Q

What are the most common dimensions measured in assessment centers?

A
  • interpersonal skills
  • communication (oral & written)
  • motivation & achievement
  • persuasion & influence
  • organization & planning
  • problem solving
56
Q

What is the validity, diversity, legal issues, etc., of assessment centers?

A

Pros: high validity 0.30+, adverse impact is lower, candidate experience is favorable

Cons: can be expensive to develop/administer, can look dated very quickly

57
Q

Be familiar with the guidelines for effective use of assessment centers/simulations.

A

6 best practices

1-Predictor method gives scores that are construct valid and reliable;
2-Predictor method gives scores that are job-related;
3-Use multiple Qs/items that are the same across applicants;
4- Qs are understandable and instructions are clear;
5-When assessors are used they are: trained, multiple, avoid prior info, held accountable for ratings;
6-Continually monitor scores of assessments for reliability and validity

58
Q

Explain compensatory

A

best used when high scores on one predictor are the most important factor and can trump or make up for low scores on another predictor

Advantages: minimizes errors, optimal predictor combo bc can use regression, very flexible
Disadvantages: must have large sample size, must assess all applicants on all predictors

59
Q

Explain noncompensatory

A

(two types: multiple cutoffs or multiple hurdles)
Multiple cutoffs – best used when you must cut people who score too low on certain predictors due to legal/safety reasons
- Adv: narrows applicant pool to only those minimally qualified, easy to use and explain, low cost/efficient, sometimes ADA compliance
- Disadv: only identifies those with minimal qualifications, must assess all applicants on all predictors

Multiple hurdles – best used when KSAOs are not compensatory or training is infeasible
- Adv: Same as multiple cutoffs but even less costly, and only assess candidates who pass earlier hurdles
- Disadv: need established validity of each predictor and justify cutoff for each, increased time required for implementation

60
Q

Understand the benefits of using regression for selection decision making. What are the limitations or potential risks with using regression?

A

Benefits – it is the best way to combine the result of multiple predictors to predict performance (better than human experts!); can help to identify the most important predictors out of a set

Limitations/risks of regression – bc it’s only based in statistics, it doesn’t account for possible adverse impact or cost of using certain predictors

61
Q

Why is the best approach a hybrid of rational (expert judgment) and empirical (regression) approaches?

A
  • Uses regression first to identify the most important predictors
  • and then use expert judgement to identify which of those chosen predictors have issues that make them less desirable to use
62
Q

Understand the logic of cut scores, and how raising/lowering a cut score will affect the number of true positives, true negatives, false positives, and false negatives.

A
  • Setting cut scores means there is a certain minimum score applicant much reach to be considered for the job
  • Pro of raising the cut score: decreases false positives (people who would be hired but can’t do the job)
  • Con of raising the cut score: increases false negatives (people who you don’t hire but could have done the job)
63
Q

Why should we generally avoid setting cut scores?

A
  • Cut scores are a major concern for legal reasons – adverse impact
64
Q

When are cut scores sometimes necessary?

A
  • Cut scores should be used when there are minimal qualifications for safety or legal reasons, or when there are large # of applicants who will never be hired