UTILITY Flashcards
the practical value of testing to improve efficiency
a. Reliability
b. Validity
c. Utility
c. Utility
“How useful is this test as compared to another test”
a. Comparative utility
b. Treatment utility
c. Diagnostic Utility
a. Comparative utility
“Is the use of this test followed by better intervention results?”
a. Comparative utility
b. Treatment utility
c. Diagnostic Utility
b. Treatment Utility
“How useful is it for classification purposes”
a. Comparative utility
b. Treatment utility
c. Diagnostic Utility
c. Diagnostic Utility
What are the factors affecting utility?
Psychometric soundness, costs and benefits
How can you say that a test score has utility?
a. it can be used everyday
b. it is cost-effective
c. it can be used in work
b. it is cost effective
What is the relationship between validity and utility?
a. Direct
b. Inverse
c. Not Sure
a. Direct (The higher the criterion-related validity of test scores for making a particular decision, the higher the utility of the test is likely to be.)
True or False
Valid tests are not always useful tests.
True
One of the most basic elements of utility analysis is the financial cost associated with a test
a. Psychometric Soundess
b. Costs
c. Benefits
b. Costs
To which do you allocate funds for in a test?
a. to a particulat test
b. a supply blank test protocols
c. computerized test processing scoring and interpretation
d. all of the above
d. all of the above
Which is harder to calculate in terms of costs?
a. Effective tests
b. Ineffective tests
b. Ineffective tests
Refers to the profits, gains or advantages
a. Psychometric Soundess
b. Costs
c. Benefits
c. Benefits
A family of techniques that entail a cost–benefit analysis designed to yield information relevant to a decision about the usefulness and/or practical value of a tool of assessment.
a. Utility Analysis
b. Taylor-Russell Tables
c. Brogden-Cronbach-Gleser Formula
a. Utility Analysis
What is the endpoint of an utility analysis?
to make an educated decision about which of many possible courses of action is optimal.
An assumption is made that high scores on one attribute can “balance out” or compensate for low scores on another attribute.
a. Selection of Compensatory Model
b. Model of Selection
c. Compensatory Model of Selection
c. Compensatory Model of Selection
The likelihood that a test taker will score within some interval of scores on a criterion measure.
a. Criterion Data
b. Expectancy Data
c. Norms Data
b. Expectancy Data
Provide an estimate of the percentage of employees hired by the use of a particular test who will be successful at their jobs
a. Utility Analysis
b. Taylor-Russell Tables
c. Brogden-Cronbach-Gleser Formula
b. Taylor-Russell Tables
What is in the Table of Taylor-Russell?
Validity of Coefficient, Selection Rate, and the Base Rate
used to calculate the dollar/peso amount of a utility gain resulting from the use of a particular selection instrument under specified conditions.
a. Utility Analysis
b. Taylor-Russell Tables
c. Brogden-Cronbach-Gleser Formula
c. Brogden-Cronbach-Gleser Formula
What are the limitations of Taylor-Russell Tables?
- the relationship between the the predictor (the test) and the criterion (rating of the performance on the job) must be LINEAR
- difficult to identify between successful and unsuccessful employees
help obtain the difference between the means of the selected and unselected groups to derive an index of what the test (or some other tool of assessment) is adding to already established procedures.
a. Taylor-Russell Tables
b. Naylor-Shine Tables
b. Naylor-Shine Tables
Utility gain is the
a. total benefit (or monetary) of using a particular test or selection method
b. estimate of the benefit (or monetary) of using a particular test or selection method
b. estimate of the benefit (or monetary) of using a particular test or selection method
refers to an estimated increase in work output
productivity gain
If the selection rate is higher, should the cut off score be higher as well?
No, cut-off score should be lower
Which among the following is not part of the practical considerations when conducting utility analyses?
a. the pool of job applicants
b. the complexity of the job
c. the cut-off score
d. the difficulty of the test
d. the difficulty of the test
Some utility models are based on the assumption that that there will be a ready supply of viable applicants from which to choose and fill positions.
a. The pool of job applicants
b. the complexity of the job
c. the cut-off score
a. the pool of job applicants
How does utility estimates the pool of job applicants?
a. It assumes that most applicants will not accept the job
b. It assumes that the selected applicants will actually accept the offer of employment thus tend to overestimate the utility measurement tool
c. It randomize applicants who will accept the job so that the utility of measurement will be impartial
b. It assumes that the selected applicants will actually accept the offer of employment thus tend to overestimate the utility measurement tool
True or False
The more complex the job, the more people differ on how well or poorly they do that job
True
Why is relative cut score also called as norm-referenced score?
a. because this type of cut score is set with reference to a criterion
b. because this type of cut score is set with a new reference made by test developers
c. because this type of cut score is set with reference to the performance of a group
c. because this type of cut score is set with reference to the performance of a group
minimum level of proficiency required to be included in a particular classification.
a. relative cut score
b. fixed cut score
b. fixed cut score
used in a distribution of test score to divide a set of data into two or more classifications
a. relative cut score
b. fixed cut score
c. all of the above
d. none of the above
c. all of the above
A high score can in fact, “balance out” or compensate for low scores on another attribute is from what model?
a. compensatory model of selection
b. los angeles models
c. multiple model of scores
a. compensatory model of selection
The statistical tool that is ideally suited for making such selection decisions within the framework of a compensatory model is called
a. pearson correlation coefficient
b. chi-squared test
c. multiple regression
c. multiple regression
Angoff Method and Known Groups Method are under
a. Classical Test Score Theory
b. IRT-Based Method
c. Method of Predictive Yield
d. Discriminant Analysis
a. Classical Test Score Theory
The judgments of the experts are averaged to yield cut scores for the test.
a. Angoff Method
b. Known Groups Method
c. Item-Mapping Method
d. Bookmark Method
a. Angoff Method
What’s the weakness of Angoff Method?
a. There is no criteria
b. If disagreement arises between raters
c. If the raters same scores
b. If disagreement arises between raters
Entails collection of data on the predictor of interest from groups known to possess, and not to possess, a trait, attribute, or ability of interest.
a. Angoff Method
b. Known Groups Method
c. Item-Mapping Method
d. Bookmark Method
b. Known Groups Method
Weakness of Known Groups Method
a. no guidelines for choosing contrasting groups
b. cut score can be placed anywhere
c. both a and b
d. no weakness
c. both a and b
Item-Mapping and Bookmark Method is under
a. Classical Test Score Theory
b. IRT-Based Method
c. Method of Predictive Yield
d. Discriminant Analysis
b. IRT-Based Method
In order to “pass” the test, the test taker must answer items that are deemed to be above some minimum level of difficulty, which is determined by experts and serves as the cut score.
a. Classical Test Score Theory
b. IRT-Based Method
c. Method of Predictive Yield
d. Discriminant Analysis
b. IRT-Based Method
entails arrangement of items in a histogram with each column containing items deemed to be of equivalent value
a. Angoff Method
b. Known Groups Method
c. Item-Mapping Method
d. Bookmark Method
c. Item-Mapping Method
experts place a bookmark between 2 items deemed to separate test takers who have acquired minimal knowledge, etc.
a. Angoff Method
b. Known Groups Method
c. Item-Mapping Method
d. Bookmark Method
d. Bookmark Method
Typically used in academic set up
a. Item-Mapping Method
b. Bookmark Method
b. Bookmark Method
took into account the number of positions to be filled, projections regarding the likelihood of offer acceptance, and the distribution of applicant scores.
a. Method of Predictive Yield
b. Discriminant analysis
c. Discriminant function analysis
a. Method of Predictive Yield
A family of statistical techniques used to shed light on the relationship between identified variables and two naturally occurring groups
a. Method of Predictive Yield
b. Discriminant analysis
c. Personnel Psychology
b. Discriminant Analysis
in Cognitive tests, a statistic indicating how many tests takers responded correctly to an Item
in personality tests, a statistic indicating how many test takers responded to an item in a particular direction
a. index of item difficulty
b. index of item discrimination
c. index of item reliability
d. index of item validity
a. index of item difficulty
a statistic designed to indicate how adequately a test item discriminates between high and low scorers.
a. index of item difficulty
b. index of item discrimination
c. index of item reliability
d. index of item validity
b. index of item discrimination
is equal to the product of the item-score standard deviation (s) and the correlation (r) between the item score and the total test score.
provides an indication of the internal consistency of a test
a. index of item difficulty
b. index of item discrimination
c. index of item reliability
d. index of item validity
c. index of item reliability
statistic indicating the degree to which a test measures what it purports to measure;
the higher the item-validity index, the greater the test’s criterion-related validity.
a. spiral omnibus format
b. index of item discrimination
c. index of item reliability
d. index of item validity
d. index of item validity
a test where the themes are spread out through the test and not together in one section.
a. spiral omnibus format
b. index of item discrimination
c. index of item reliability
d. index of item validity
a. spiral omnibus format
