lecture 7- occupational assessment Flashcards
complexities of personnel selection
Seems simple issue?
* Use valid assessment techniques to
select job applicants with the best
profile of performance
But: legal, practical and psychometric
pitfalls.
* What are you looking for in a good
employee?
* How do you choose the assessment
techniques?
* Employers ‘just know’ how to select
employees.
* Subjective influences and
unconscious biases
job analysis matrix
what the worker does
- duties
-tasks
- responsibilities
how the worker does it
- methods
- tools
-techniques
why the worker does it
- products
-services
worker qualifications
- skills
- knowledge
- abilities
- physical demands
job analysis
- Understand nature of job task requirements
- Job description and person specification
- Job analysis methods
- Observations, records, questionnaires,
interviews, diaries - E.g. Position Analysis Questionnaire
- Updated 2008, adapted for different settings
- Self-report, Employer/manager report
- Covers areas such as: Knowledge & skills,
organisational relationships, responsibilities,
decision-making, working environment,
qualifications, primary purpose of job.
position analysis questionnaire
- Reliability
- Analysis of 92 jobs by two independent groups
- Inter-rater reliability coefficient of 0.79
- Different perspectives (e.g. employee v employer)
suggest acceptable reliability - Validity
- Evidence of reasonable validity
- But relatively few studies, small numbers in studies, and
potential sources of subjective bias - Sanchez & Levine (2012) Annual Review of Psychology
steps in recruitment process
- job analysis
- job and person description
- advertise
- select best candidate
selection techniques
- autobiographical data
- interviews
- cognitive ability tests
- personality and integrity tests
- assessment centres
autobiographical data (biodata)
- Application form, curriculum vitae, references
- Factual info: education, work experience, skills…
- Tends to be quite reliable, faking levels low
- Biodata good predictor of job performance
- Meta-analysis by Schmidt & Hunter
- Useful for screening candidates (Cole et al., 2003)
- For psychology graduates Sulastra et al. (2015) found that key biodata predictor:
- …of getting a job on graduation is work experience
- ….of getting a psychology-based job is GPA
purpose of interviews
- Interviews are a form of
assessment - Gather data about
people - Make predictions about
their behaviour - Most frequent assessment in job selection
- Initial method of data collection
- Clinical psychology and educational psychology
employment interviews
early studies on interviews:
- poor inter-rater reliability (.5) and predective validity (eg. Wagner, 1949)
but:
- panel interviews better than single interviews
- structured interviews better reliability and predictive validity compared to unstructured interviews. (Wiesner and cronshaw, 1988)
structured verses unstructured interviews
Unstructured interviews
* Free-ranging and unplanned
* Halo effects
* Influenced by social situation and biases
Structured interviews
* Standardised questions focused on job
* Relate to job analysis and person specification
* Consistent panel with transparent scoring
* Reduce biases
Validity of structured versus unstructured
employment interviews
- Levashina et al. (2014) review of literature
- 12 meta-analyses confirm structured interviews considerably higher
predictive validity and reliability than unstructured interviews - Structured interviews provide incremental validity over personality and
cognitive ability tests - Structured interviews reduce bias against women and ethnic minority groups
cognitive ability tests
- Intelligence, verbal skills, numerical ability, spatial
perception, problem-solving, speed of processing…. - General ability tests
- Wonderlic Personnel Test
- Multiple Aptitude Batteries
- Specific job-relevant abilities:
- Mechanical, motor, perceptual, clerical tests
cognitive ability tests
- Specific versus general tests?
- Often group multiple-choice tests
- Excellent reliability
- Generally good predictive validity:
- Bertua et al. (2005) meta-analysis: cognitive test scores predict job
performance: r = 0.5 - 0.6. - Ryan & Ployhard (2014): almost no novel research in last decades - cognitive
ability well established predictor job performance.
personality tests
- Five factor model of personality
- Costa & Macrae
- Openness, conscientiousness,
extraversion, agreeableness, neuroticism
(OCEAN) - Measured by NEO-PI questionnaire
- Good reliability but low predictive validity
- Hurtz & Donovan (2000)
- Meta-analysis of 45 studies (8000+
employees) - Predictive validity ranged from .04
(openness) to .26 (conscientiousness)
maybe it depends what type of job?
- Judge & Zapata (2015): Personality x situation interactions
- Conscientiousness predicts job performance where the job requires
independence - Neuroticism is negatively related to job performance when dealing with
difficult customers - Extraversion: positively related to job performance when social skills are
required, but negatively when the job requires attention to detail. - Conclude predictive validity of personality traits for appropriate jobs double that
of predictive validity for jobs as a whole.
integrity tests
- Detect undesirable behaviours and attitudes
- E.g. attitudes to theft, disregard for fellow employees
- Predictive validity of integrity tests
- Ones et al. (1993) meta-analysis
- Integrity & job performance: r = .41
- Criticized: dependent on test publisher information
- Updated meta-analysis Van Iddekinge et al. (2012)
- Predictive validity r = .14 for job performance
- r = .30 for counterproductive work behaviour
assessment centres
- Key elements in assessment centres:
- Leaderless group discussions
- Role play
- Presentation skills
- Interview
- Work sample or situational judgement test
- Case analysis
gale et al. (2010): assessment centre for anaesthetics
- 244 candidates for anaesthesia training assessed
- Assessments used: Structured interview, portfolio review,
presentation, medical simulation (work sample) - 68 appointed to posts
- Psychometric results:
- Inter-rater reliability = .62-.77
- Cronbach’s α = .88-.91
- Predictive validity: Correlations assessment scores and:
- In-theatre performance = .42
- Annual review of competence = .48
Combining assessments to predict job
performance: incremental validity
- Best combinations to predict job success:
- Schmidt & Hunter (1998): Ability tests + work
sample - Schmidt (2016): Ability tests + integrity tests +
structured interviews - Emotional intelligence (EI)?
- O’Boyle et al. (2010): Emotional intelligence
predicts job success (r from .24 to .30) - Incremental prediction to cognitive ability and
personality - But EI overlaps with cognitive ability and
personality traits - May explain relationships with job success
(Joseph et al, 2015)
evidence from the uk
- Meta-analytic evidence job prediction in UK (Bertua et
al., 2011) - Similar pattern to US/worldwide data
- Both general and specific cognitive tests good
predictors - Assessment stronger predictor for more
complex/managerial jobs - What do UK businesses actually do?
- Zibarras & Woods (2010) surveyed 579 UK businesses
- Interviews: structured 69%, unstructured 41%
- Biodata: CV 85%, application form 60%, references
72% - Ability test 39%, personality test 26%
- Work sample 19%, assessment centre 17%
main points from this topic
- Job analysis important to inform selection
methods - Key selection techniques to predict job
performance: - Biodata reliable and reasonable predictor
- Interviews reliable and reasonably valid if
structured - Cognitive ability tests and work samples good
predictors - Personality tests only predictive in specific types
of job - Best to use combination of validated
methods
