Ch. 4 Predictors: Psychological Assessments Flashcards
What is a predictor?
- Any variable used to forecast/predict a criterion
- For example, you want someone who will show up on time for work (criterion)
What is Reliability
- Consistency and stability of measurement
- Three types of reliability used for different reasons
NOT interchangeable - Regardless, good scores are r = .70 and above
Test-retest Reliability: Coefficient of stability
- If a person took the assessment again in a month, would they get the same scores?
- If we think the assessment is tapping into something enduring or trait-like, it should not vary wildly across time
Internal-Consistency Reliability: Homogeneous content
- Degree to which individual items of an assessment relate to one another
- Split-half Reliability: Divide test into two and see how well their parts relate to one another
- Cronbach’s Alpha or Kuder-Richardson 20 (KR20): Each individual item is related to all other items and degree of agreement among items is assessed
- Split-half Reliability - Divide test into two and see how well their parts relate to one another
Inter-Rater Reliability: Conspect reliability
- If three separate interviewers rate the performance of an interviewee, you can evaluate the degree to which they agree with one another
- Did everyone see the candidate in the same way?
- Disagreement needs to be discussed and understood
Validity
- Accuracy of measurement
- Are we measuring what we seek to measure?
Construct Validity
- The degree to which a test is an accurate measure of the construct it is trying to measure
Convergent validity
- The degree to which our test relates to what it should theoretically relate to
- Happiness should relate to optimism and negative affect (inversely)
Discriminant validity
- The degree to which the construct does not relate to things it should not theoretically relate to
- Happiness does not relate to intelligence
Criterion-Related Validity
- Another way of assessing construct validity
The degree to which a predictor relates to a criterion - Concurrent criterion-related validity
- Predictive criterion-related validity
- Determine both of these in a sample of employees for whom we have these scores
Validity Coefficient
- The correlation between predictor scores and a criterion
Desired (and common) range is .30 to .40 - Squaring the correlation tells us variance we can explain in the criterion variable
- r = .40, we are explaining 16% of the variance
Content Validity
- Another way to assess construct validity
- No statistics
- An evaluation of how well the test represents the domain you seek to assess
- Only assessing knowledge of one chapter would lead to poor content validity if the criterion is knowledge of I/O psychology in general
- Typically assessed by Subject Matter Experts
Content of assessment needs to relate to the content of a job (as outlined by the Work Analysis)
Similar type of “validity” - Face Validity
- Items appear appropriate for purpose of assessment
- Book says this is assessed by test-takers
Predictors: Measured via a test
- Give potential mechanics various questions assessing knowledge of cars and how to fix them
Predictors: Measured via a sampling of behavior
- Give potential mechanics a broken car to assess and fix
Predictor: Present-oriented
- Job interview may assess level of interpersonal skills
Predictor: Past-oriented
- Job interview may ask “tell me about a time in which you…”
- Letters of recommendation
Intelligence
- Concept of g vs. multiple intelligences
- Racially charged history and controversy
- Book says “single best predictor job performance” which is true numerically, but likely not practically or ethically
Mechanical Aptitude
- Assess the recognition of mechanical principles like sound and heat conductance, velocity, gravity, and force.
- Predictive of success in manufacturing/production type jobs
- Women tend to perform worse
- Example: Bennett-Mechanical Test
Series of pictures that illustrate various mechanical concepts and principles
Personality
- No “right v. wrong” answers
- Scale scores used to predict job success
Big 5 Personality Theory
Openness to experience, Conscientious, Extraversion, Agreeableness, Neuroticism (OCEAN or CANOE)
Dark Triad
- Machiavellianism, narcissism, psychopathy
Integrity
- Overt (transparent)
- Asks questions about attitudes toward theft and other forms of dishonesty, endorsement of common rationalizations of theft, or admission of theft
- Incentive for employee to lie
- Personality-based
- Take a personality measure that makes no overt reference to theft, but has been found to be predictive of theft
- More predictive of counterproductive work behavior than job performance or turnover
- Incentive to lie
Situational Judgment
- Responses are not scored simply as right or wrong
- Designed to reflect reality of making decisions in life