Validity – Criterion-Related Validity Flashcards
An organizational psychologist conducts a predictive validity study to obtain the data she needs to evaluate a new selection test’s incremental validity. The data she collects indicates that 45 of the employees in her study are “true negatives,” which means that they obtained:
A. low scores on both the predictor and criterion.
B. high scores on both the predictor and criterion.
C. high scores on the predictor and low scores on the criterion.
D. low scores on the predictor and high scores on the criterion.
Answer A is correct. To identify the correct answer to this question, you need to know that a person’s score on the predictor determines if he/she is a “positive” or “negative”: If the person received a high score on the predictor, that person is a positive; if the person received a low score on the predictor, he/she is a negative. Knowing that narrows the choices to answers A and D since the question is asking about true negatives. To choose between these two answers, you need to know that a person’s score on the criterion determines if he or she is a true or false positive or negative: When the person obtains low scores on the predictor and the criterion, he/she is a true negative (answer A); when the person obtains a low score on the predictor but a high score on the criterion, he/she is a false negative (answer D).
A psychologist was hired to develop a new selection test that the company will add to its current selection procedure. The psychologist determines that the test has adequate test-retest reliability and concurrent validity. However, before the company begins using the test to hire applicants, the psychologist will want to make sure that its use also produces an increase in decision-making accuracy. In other words, the psychologist will want to make sure that the test has adequate:
A. predictive validity.
B. differential validity.
C. incremental validity.
D. construct validity.
Answer C is correct. A predictor has incremental validity when its use produces an adequate increase in decision-making accuracy.
When a predictor’s reliability coefficient is .49, its criterion-related validity coefficient can be no greater than:
A. .70.
B. .51.
C. .49.
D. 30.
Answer A is correct. The maximum criterion-related validity coefficient for a predictor is equal to the square root of its reliability coefficient: The square root of .49 is .70.
On cross-validation, a multiple correlation coefficient (R) is most likely to “shrink” when:
A. the original validation sample was small and the number of predictors is small.
B. the original validation sample was small and the number of predictors is large.
C. the original validation sample was large and the number of predictors is small.
D. the original validation sample was large and the number of predictors is large.
Answer B is correct. Shrinkage refers to the reduction in the size of a criterion-related correlation coefficient on cross-validation and is greatest when the initial sample is small and, for the multiple correlation coefficient, the number of predictors is large.
A test developer administers her newly developed job knowledge test to 50 applicants for sales positions at a large insurance company. All 50 applicants are hired regardless of their scores on the test and their scores are correlated with the scores they receive on a job performance measure six months later. This correlation coefficient provides information about the job knowledge test’s:
A. concurrent validity.
B. predictive validity.
C. convergent validity.
D. discriminant validity.
Answer B is correct. Predictive validity is a type of criterion-related validity that’s evaluated to determine how well predictor scores predict future criterion scores. It’s assessed by correlating scores on the predictor (e.g., job knowledge test) with scores obtained on the criterion (e.g., job performance measure) at a later time.
The sensitivity of a diagnostic screening test is:
A. the ability of the test to identify clients who do not have the disorder.
B. the ability of the test to identify clients who have the disorder.
C. the probability that a client who tests negative on the test does not actually have the disorder.
D. the probability that a client who tests positive on the test actually has the disorder.
Answer B is correct. Sensitivity is the proportion of people with the disorder who are identified by the test as having the disorder. It’s calculated by dividing the true positives by the true positives plus the false negatives (TP/TP + FN).
When a predictor’s criterion-related validity coefficient is .40, this means that:
A. 40% of variability in predictor scores is true score variability.
B. 16% of variability in predictor scores is true score variability.
C. 40% of variability in criterion scores is explained by variability in predictor scores.
D. 16% of variability in criterion scores is explained by variability in predictor scores.
Answer D is correct. Like other correlation coefficients for two different measures, the criterion-related validity coefficient can be squared to determine the amount of variability in one measure that’s explained by or shared with the other measure: .40 squared is .16. When a predictor’s criterion-related validity coefficient is .40, this means that 16% of variability in criterion scores is explained by variability in predictor scores.
Based on his score on a selection test, a job applicant obtains a predicted job performance score of 70. The job performance measure’s standard error of estimate is 5, which means that there’s a 95% chance that his true job performance score is between:
A. 65 and 75.
B. 60 and 80.
C. 55 and 85.
D. 50 and 90.
Answer B is correct. The 95% confidence interval for a predicted criterion score is calculated by adding plus and minus two standard errors of estimate to the obtained criterion score: 70 + 5(2) = 60 to 80.
