Assessment: CRV & Validity and Reliability of Assessment Methods

Question 1

Different Validity: Test Scores vs Job Performance

Answer 1

If there is a large difference in rejection rates between different groups, need to look at relationship between test scores and job performance. If low predictor scores are no reflected with job performance, this therefore shows the test has no validity.

Question 2

Validity Generalisation

Answer 2

The transportability of validity coefficients to similar groups e.g. similar jobs, this precluding the need for further CRV.

This is done when samples are not large enough to carry out a validity study.

Question 3

Validity and Reliability of Assessment Methods: Types of Validity and Reliability

Answer 3

Face validity refers to the subjects perceived relevance of the measure

Content validity is representativeness of a measure

Reliability as the consistency with which a measure provides results. I.e its stability and freedom from random variation.

Inter rater reliability - degree of agreement between 2 or more raters

Intra rater reliability - how far a single rater is consistent across time.

Question 4

Psychometric Tests and Inventories

Answer 4

A standardised sample of behaviour which can be described by a numerical scale - Cronbach (1984).

Objective rather than subjective via standardisation (instructions, time, content, scoring and interpretation).

Question 5

Psychometric Tests

Answer 5

Wechsler Adult Intelligence Scales (WAIS)

Myers-Briggs Type Indicator (MBTI)

NEO PI-R five factor model (FFM) (Costa and McCrea 1990) OCEAN

Question 6

Psychometric Tests: OPQ

Answer 6

9 interlocking questionnaires consisting of 30 concepts, relating to thinking style, feelings and relationships with people. Number of factors in shorter versions derived from 30 concepts.

Question 7

V and R for Psychometric Tests and Inventories

Answer 7

All these tests have been subjected to CRV studies for a range of occupations. Results vary for particular traits/occupations, but this is only to be expected. A test can only be valid if its use is based on a through job analysis.

Once reliable always reliable.

Validity depends on appropriate use - little validity if communication skills were used to predict perceptual motor performance - even if test were valid when used appropriately

Also problems with: culture fairness of items, definition of intelligence and personality, social desirability/faking/impression management

Question 8

Interviews

Answer 8

Interviewer: aim to gather data, evaluate it and select or reject. An intuitive regression equation must combine and weigh data to come to a decision. How well this done determines validity.

Importance to level of training and experience and biases. May be due to personal factors such as age, sex, race intelligence and personality factors. Also stereotypes held and set.

Question 9

Interviews Personal Communications

Answer 9

A conversation with a purpose. Behaviour of interviewer can affect response of interviewee. Agreeing, paraphrasing silence or disagreement. Behaviour influenced by perception of each other.

Group vs individual interview.

Question 10

Interview V and R

Answer 10

Very little predictive power for interviews (Mayfield 1964; Arvey and Campion 1982)

Thousands of studies showing range of validities

Shown poor inter rater reliability

Even intra rater reliability

Question 11

Interviews can be improved by: 5 Point Plan & 7 Point Plan

Answer 11

For example, the 5-point plan: 1) Impact on Others 2) Qualifications 3) Experience 4) Motivation 5) Adjustment.

Also the 7-point plan: 1) Physical makeup 2) Attainments 3) Intelligence 4) Aptitudes 5) Interests 6) Disposition 7) Circumstances

Question 12

Situational interviews

Answer 12

Use job analyses to provide job related incidents for job related questions - what would you do.

Training of interviewers - to use job related questions asked to graduates based on requirements

Question 13

Work Sample Tests

Answer 13

Any assessment which involves the assessee being asked to perform a behaviour which is the same to those performed in real life

Psychomotor
Job knowledge tests
Individual decision making
Group discussions
Indoor/outdoor/leaderless/with leader
If done correctly good validity and reliability

Question 14

Assessment Centre

Answer 14

Seek to account for all the variance by using multiple methods of assessment with multiple assessors over several days

Question 15

Assessment: Synthetic Validityy

Answer 15

‘…the inferring of validity in a specific situation from a logical analysis of jobs into their elements, a determination of test validity for these elements, and a combination of elemental validities into a who

Reliability and validity dependent on those of assessment methods used and combination thereafter.