Utility Flashcards
The practical value of using a test to aid in decision making and improve efficiency. It also refers to the practical value of a training program or intervention.
Utility
The practical value of the information derived from scores of the test.
Index of utility
The higher the _ validity of test scores for making a particular decision, the higher the utility of the test is likely to be.
Criterion- related validity
The disadvantages, losses of expenses in both economic and non economic terms. It is the expenditures associated with testing or not testing.
Costs
It refers to profits, gains or advantages. It should justify the costs of administering, scoring and interpreting the test.
Benefits
It may be undertaken for the purpose of evaluating whether the benefits of using a test outweight the costs.
Utility analysis
Provides an estimate of the extent to which inclusion of a particular test in the selection system will improve selection. The tables provide an estimate of the percentage of employees hired by the use of a particular test who will be successful at their jobs.
Taylor-Russell tables
The computed validity coefficient in Taylor-Russell tables.
Test’s validity
The numerical value that reflects the relationships between the number of people to be hired and the number of people available to be hired in Taylor-Russell tables.
Selection Ratio
It is the percentage of people hired under the existing system for a particular position, in the Taylor-Russell tables.
Base Rate
What are the 3 variables in Taylor-Russell tables?
Test’s validity
Selection Ratio
Base rate
One limitation of the Taylor-Russell tables is that the relationship between the predictor (test) and the criterion (rating of performance) must be _.
Linear
Entails obtaining the difference between the means of the selected and unselected groups to derive an index of what the test is adding to already established procedures. Used in utility analysis.
Naylor-Shine tables
The Naylor-Shine tables determine the increase in average score on some criterion measure while the Taylor-Russell determine the increase over _ procedures.
Current
The validity coefficient with both the Taylor-Russell and the Naylor-Shine tables must be obtained by _ validation procedures.
Concurrent validation
It is used to calculate the dollar amount of a utility gain resulting from the use of a particular selection instrument under specified conditions.
The Brogden-Cronbach-Gleser formula
It refers to an estimate of the benefit of using a particular test or selection method.
Utility gain
The estimated increase in work output.
Productivity gain
The value of a test lies in the extent to which its use improves the _ that exists without its use.
Hit rate
It means correct classification.
Hit
Incorrect classification.
Miss
The proportion of people that an assessment tool accurately identifies as possessing or exhibiting a particular trait, ability, behavior or attribute.
Hit rate
The proportion of people that an assessment tool inaccurately identifies as possessing or exhibiting a particular trait, ability, behavior or attribute.
Miss rate
A specific type of miss whereby an assessment tool falsely indicates that the testtaker possesses or exhibit a particular trait, ability, behavior or attribute.
False positive
A specific type of miss whereby an assessment tool falsely indicates that the testtaker does not possess or exhibit a particular trait, ability, behavior or attribute.
False negative
If the selection ratio is high, the cut-off score will be higher. There will be many _.
False positives
If the selection ratio is low, the cut-off score will be higher. There will be many _.
False negatives
_ theory provides guidelines for setting an optimal cutoff score.
Decision theory
Period of high unemployment there are significantly more people in the pool of possible job applicants than in periods of high employment.
Economic climate
It is usually a numerical reference point derived as a result of a judgment and used to divide a set of data into two or more classifications, with some to be taken or some inference to be made on the basis of these classifications.
Cut score
A type of cut score. It is the reference point that is set based on norm-related consideration.
Relative cut score or norm-referenced cut score.
A type of cut score. It is a reference point that is typically set with reference to a judgment concerning a minimum level of proficiency required to be included in a particular classification.
Fixed cut score or absolute cut score
A type of cut score that refers to the use of two or more cut scores with reference to one predictor for the purpose of categorizing testtakers.
Multiple cut score
It is one collective element of a multistage decision-making process in which the achievement of a particular cut score on one test is necessary in order to advance to the next stage of evaluation in the selection process.
Multiple Hurdle
High scores on one attribute can balance out or compensate for low scores on another attribute.
Compensatory model of selection
It is the statistical tool that is ideally suited for making selection decisions within the framework of a compensatory model.
Multiple regression
Different types of cut scores:
Relatively cut score or norm referenced cut score
Fixed cut score or absolute cut scores
Multiple scores
Multiple Hurdle
What are the methods for setting cut scores?
The Angoff Method
The Known Groups Method or the Method of contrasting Groups
IRT Based Methods
Methods of Predictive yield
Discriminant Analysis or Discriminant Function Analysis
The Angoff Method was devised by _.
William Angoff
It is a method of setting cut scores that can be applied to personnel selection tasks as well as to questions regarding the presence of absence of a particular trait, attribute or ability.
The Angoff Method
In Angoff Method, they are the one who provide estimates regarding how testtakers who have atleast minimal competence for the position should answer the items correctly.
Experts
The weakness of Angoff Method is when there is a low _ reliability.
Inter-rater reliability
A method for setting cut scores that entails collection of data on the predictor of interest from groups known to possess and not to possess a trait, attribute or ability of interest. The cut score is set on the test that best discriminates the 2 group’s performance.
The Known Groups Method or the Method of Contrasting Groups
Cut scores are based on testtaker’s performance across all items on the test. Some portion of the total number of items on the test must be scored “correct” in order for the testatker to pass the test.
Classical test score theory
Within this theory framework-based methods of setting a cut score, each item is associated with a particular level of difficulty. The test takers must answer items that are deemed to be above some minimum level of difficulty which is determined by experts and serves as the cut score.
Item-response theory (IRT)
What are the 2 IRT-based methods for determining the difficulty level reflected by a cut score?
Item-mapping method
Bookmark method
IRT based method that includes arrangement of items in a histogram with each column in the histogram containing items deemed to be of equivalent value.
Item-mapping method
An IRT based method that begins with the training of experts with regard to the minimal knowledge, skills and/or abilities that testtakers should possess in order to pass. The experts are given a book of items, arranged in an ascending order of difficulty. They place bookmarks between the 2 pages that are deemed to separate minimal knowledge from those who have not. Bookmark served as the cut score.
Bookmark method
A technique for setting cut scores which took into account the number of positions to be filled, projections regarding the likelihood of offer acceptance, and the distribution of applicant scores.
Method of Predictive Yield
A method for setting cut scores that is used to shed light on the relationship between identified variables and two naturally occurring groups.
Discriminant Analysis or Discriminant Function Analysis
