Ch. 6 - Validity

Question 1

base rate

Answer 1

an index, usually expressed as a proportion, of the extent to which a particular trait, behavior, characteristic, or attribute exists in a population

Question 2

bias

Answer 2

as applied to tests, a factor inherent within a test that systematically prevents accurate, impartial measurement

Question 3

central tendency error

Answer 3

a type of rating error wherein the rater exhibits a general reluctance to issue ratings at either the positive or negative extreme and so all or most ratings cluster in the middle of the rating continuum

Question 4

concurrent validity

Answer 4

a form of criterion-related validity that is an index of the degree to which a test score is related to some criterion measure obtained at the same time

Question 5

confirmatory factor analysis

Answer 5

a class of mathematical procedures employed when a factor structure that has been explicitly hypothesized is tested for its fit with the observed relationships between the variables

Question 6

construct

Answer 6

an informed, scientific idea developed or generated to describe or explain behaviors…

i.e., intelligence, personality, anxiety, job satisfaction

Question 7

construct validity

Answer 7

a judgement about the appropriateness of inferences drawn from test scores regarding individual standings on a variable called a construct

Question 8

content validity

Answer 8

describes a judgement of how adequately a test samples behavior representative of the universe of behavior that the test was designed to sample

Question 9

convergent evidence

Answer 9

with reference to construct validity…

data from other measurement instruments designed to measure the same or a similar construct as the test being construct-validated and that all point to the same judgement or conclusion with regard to a test or other tool of measurement

contrast with discriminant evidence

Question 10

convergent validity

Question 11

criterion

Answer 11

the standard against which a test or a test score is evaluated

may take many forms, including a specific behavior or set of behaviors

Question 12

criterion contamination

Answer 12

a state in which a criterion measure is itself based, in whole or part, on a predictor measure

Question 13

criterion-related validity

Answer 13

a judgement regarding how adequately a score or index on a test or other tool of measurement can be used to infer an individual’s most probable standing on some measure of interest

Question 14

discriminant evidence

Answer 14

with reference to construct validity…

data from a test or other measurement instrument showing little relationship between test scores or other variables with which the scores on the test bring construct-validated should not theoretically be correlated

contrast with convergent evidence

Question 15

expectancy chart

Question 16

expectancy data

Question 17

exploratory factor analysis

Answer 17

a class of mathematical procedures employed to estimate factors, extract factors, or decide how many factors to retain

Question 18

face validity

Answer 18

a judgement regarding how well a test or other tool of measurement measures what it purports to measure that is based solely on “appearances”, such as the content of the test’s items

Question 19

factor analysis

Answer 19

a class of mathematical procedures, frequently employed as data reduction methods, designed to identify variables on which people may differ (or factors)

Question 20

factor loading

Answer 20

in factor analysis, a metaphor suggesting that a test (or an individual test item) carries with it or “loads” on a certain amount of one or more abilities that, in turn, have a determining influence on the test score (or on the response to the individual test item)

Question 21

fairness

Answer 21

as applied to tests, the extent to which a test is used in an impartial, just, and equitable way

Question 22

false negative

Answer 22

a specific type of miss characterized by a tool of assessment indicating that the test-taker does not possess or exhibit a particular trait, ability, behavior, or attribute when in fact, the test-taker DOES possess or exhibit a particular trait, ability, behavior, or attribute

Question 23

false positive

Answer 23

an error in measurement characterized by a tool of assessment indicating that the test-taker possesses or exhibits a particular trait, ability, behavior, or attribute when in fact the test-taker does not

Question 24

generosity error

Answer 24

aka leniency error

a less than accurate rating or evaluation by a rater due to that rater’s general tendency to be lenient or insufficiently critical

contrast with severity error

Question 25

halo effect

Answer 25

a type of rating error wherein the rater views the object of the rating with extreme favor and tends to bestow ratings inflated in a positive direction a set of circumstances resulting in a rater's tendency to be positively disposed and insufficiently critical

Question 26

hit rate

Answer 26

the proportion of people who are accurately identified as possessing or not possessing a particular trait, ability, behavior, or attribute based on test scores

Question 27

homogeneity

Answer 27

describes the degree to which a test measures a single trait

Question 28

incremental validity

Answer 28

used in conjunction with predictive validity, and index of the explanatory power of additional predictors over and above the predictors already in use

Question 29

inference

Answer 29

a logical result or deduction in a reasoning process

Question 30

intercept bias

Answer 30

occurs when the use of a predictor term results in a consistent under-prediction or over-prediction of a specific group's performance or outcomes

Question 31

leniency error

Answer 31

aka generosity error

Question 32

local validation study

Answer 32

the process of gathering evidence, relevant to how well a test measures what it purports to measure, for the purpose of evaluating the validity of a test or other measure tool typically undertaken in conjunction with a population different from the population for whom the test was originally validated

Question 33

method of contrasted groups

Answer 33

aka known groups method a system of collecting data on a predictor of interest from groups known to possess (and not possess) a trait, attribute, of ability of interest

Question 34

miss rate

Answer 34

the proportion of people a test of other measurement procedure fails to identify accurately with respect to the possession or exhibition of a particular trait, ability, behavior, or attribute a "miss" in this context is an inaccurate classification or prediction, may be subdivided into false positives and false negatives

Question 35

multitrait-multimethod matrix

Answer 35

a method of evaluating construct validity by simultaneously examining both convergent and divergent evidence by means of a table of correlations between traits and methods

Question 36

predictive validity

Answer 36

a form of criterion-related validity that is an index of the degree to which a test score predicts some criterion measure

Question 37

ranking

Answer 37

the ordinal ordering of persons, scores, or variables into relative positions or degrees of value

Question 38

rating

Answer 38

a numerical or verbal judgement that places a person or attribute along a continuum identified by a scale of numerical or word descriptors called a rating scale

Question 39

rating error

Answer 39

a judgment that results from the intentional or unintentional misuse of a rating scale two types of rating error are leniency (aka generosity) error and severity error

Question 40

rating scale

Answer 40

a system of ordered numerical or verbal descriptors on which judgments about the presence/absence or magnitude of a particular trait, ability, behavior, or attribute, emotion, or other variable are indicated by raters, judges, examiners, or (when the rating scale reflects self-reports) the assessee.

Question 41

severity error

Answer 41

less than accurate rating or error in evaluation due to the rater's tendency to be overly critical contrast with generosity error

Question 42

slope bias

Answer 42

it occurs when a predictor has a weaker correlation with an outcome for specific groups

Question 43

test blueprint

Answer 43

a detailed plan of the content, organization, and quantity of the items that a test will contain

Question 44

validation

Answer 44

the process of gathering and evaluating validity evidence

Question 45

validation study

Question 46

validity

Answer 46

a general term referring to a judgment regarding how well a test or other measurement tool measures what it purports to measure judgment has important implications regarding the appropriateness of inferences made and actions taken on the basis of measurements

Question 47

validity coefficient

Answer 47

a correlation coefficient that provides a measure of the relationship between test scores and scores on a criterion measure