Chapter 6 - Validity

Question 1

Bias

Answer 1

As applied to tests, a factor inherent within a test that systematically prevents accurate, impartial measurement.

Question 2

Central Tendency Error

Answer 2

A type of rating error wherein the rater exhibits a general reluctance to issue ratings at either the positive or negative extreme and so all or most ratings cluster in the middle of the rating continuum.

Question 3

Concurrent Validity

Answer 3

A form of criterion-related validity that is an index of the degree to which a test score is related to some criterion measure obtained at the same time.

Question 4

Confirmatory Factor Analysis

Answer 4

A class of mathematical procedures employed when a factor structure that has been explicitly hypothesized is tested for its fit with the observed relationships between the variables.

Question 5

Construct

Answer 5

An informed, scientific idea developed or generated to describe or explain behavior; some examples of constructs include “intelligence,” “personality,” “anxiety,” and “job satisfaction.”

Question 6

Construct Validity

Answer 6

A judgment about the appropriateness of inferences drawn from test scores regarding individual standings on a variable called a construct.

Question 7

Content Validity

Answer 7

A judgement regarding how adequately a test (or other tool of measurement) samples behavior representative of the universe of behavior it was designed to sample.

Question 8

Content Validity Ratio

Answer 8

A formula, developed by C. H. Lawshe, used to gauge agreement among raters regarding how essential an individual test item is for inclusion in a test

Question 9

Convergent Evidence

Answer 9

With reference to construct validity, data from other measurement instruments designed to measure the same or a similar construct as the test being construct-validated and that all point to the same judgment or conclusion with regard to a test or other tool of measurement.

Question 10

Convergent Validity

Answer 10

Data indicating that a test measures the same construct as another test purporting to measure the same construct.

Question 11

Criterion

Answer 11

The standard against which a test or a test score is evaluated; this standard may take many forms, including a specific behavior or set of behaviors.

Question 12

Criterion Contamination

Answer 12

A state in which a criterion measure is itself based, in whole or in part, on a predictor measure.

Question 13

Criterion-Related Validity

Answer 13

A judgment regarding how adequately a score or index on a test or other tool of measurement can be used to infer an individual’s most probable standing on some measure of interest (the criterion).

Question 14

Discriminant Evidence

Answer 14

With reference to construct validity, data from a test or other measurement instrument showing little relationship between test scores or other variables with which the scores on the test being construct-validated should not theoretically be correlated.

Question 15

Expectancy Chart

Answer 15

Graphic representation of an expectancy table.

Question 16

Expectancy Data

Answer 16

Information, usually in the form of an expectancy table, illustrating the likelihood that an individual testtaker will score within some interval of scores on a criterion measure.

Question 17

Expectancy Table

Answer 17

Information presented in tabular form illustrating the likelihood that an individual testtaker will score within some interval of scores on a criterion measure.

Question 18

Exploratory Factor Analysis

Answer 18

A class of mathematical procedures employed to estimate factors, extract factors, or decide how many factors to retain.

Question 19

Face Validity

Answer 19

A judgment regarding how well a test or other tool of measurement measures what it purports to measure that is based solely on “appearances,” such as the content of the test’s items.

Question 20

Factor Analysis

Answer 20

A class of mathematical procedures, frequently employed as data reduction methods, designed identify variables on which people may differ (or factors).

Question 21

Factor Loading

Answer 21

In factor analysis, a metaphor suggesting that a test (or an individual test item) carries with it or “loads” on a certain amount of one or more abilities that, in turn, have a determining influence on the test score (or on the response to the individual test item).

Question 22

Fairness

Answer 22

As applied to tests, the extent to which a test is used in an impartial, just, and equitable way.

Question 23

False Negative

Answer 23

A specific type of miss characterized by a tool of assessment indicating that the testtaker does not possess or exhibit a particular trait, ability, behavior, or attribute when in fact, the testtaker does possess or exhibit this trait, ability, behavior, or attribute.

Question 24

False Positive

Answer 24

An error in measurement characterized by a tool of assessment indicating that the testtaker possesses or exhibits a particular trait, ability, behavior, or attribute when in fact the testtaker does not.

Question 25

Generosity Error

Answer 25

Also referred to as leniency error, a less than accurate rating or evaluation by a rater due to that rater’s general tendency to be lenient or insufficiently critical.

Question 26

Halo Effect

Answer 26

A type of rating error wherein the rater views the object of the rating with extreme favor and tends to bestow ratings inflated in a positive direction; a set of circumstances resulting in a rater’s tendency to be positively disposed and insufficiently critical.

Question 27

Hit Rate

Answer 27

The proportion of people who are accurately identified as possessing or not possessing a particular trait, behavior, characteristic, or attribute based on test scores.

Question 28

Homogeneity

Answer 28

a

Question 29

Incremental Validity

Answer 29

Used in conjunction with predictive validity, an index of the explanatory power of additional predictors over and above the predictors already in use.

Question 30

Inference

Answer 30

A logical result or a deduction in a reasoning process.

Question 31

Intercept Bias

Answer 31

A reference to the intercept of a regression line exhibited by a test or measurement procedure that systematically underpredicts or overpredicts the the performance of members of a group; contrast with slope bias.

Question 32

Leniency Error

Answer 32

Also referred to as a generosity error, a rating error that occurs as the result of a rater’s tendency to be too forgiving and insufficiently critical.

Question 33

Local Validation Study

Answer 33

The process of gathering evidence, relevant to how well a test measures what it purports to measure, for the purpose of evaluating the validity of a test or other measurement tool; typically undertaken in conjunction with a population different from the population for whom the test was originally validated.

Question 34

Method of Contrasted Groups

Answer 34

A system of collecting data on a predictor of interest from groups known to possess (and not to possess) a trait, attribute, or ability of interest.

Question 35

Miss Rate

Answer 35

The proportion of people a test or other measurement procedure fails to identify accurately with respect to the possession or exhibition of a trait, behavior, characteristic, or attribute; a “miss” in this context is an inaccurate classification or prediction; may be subdivided into false positives and false negatives.

Question 36

Multitrait-multimethod Matrix

Answer 36

A method of evaluating construct validity by simultaneously examining both convergent and divergent evidence by means of a table of correlations between traits and methods.

Question 37

Predictive Validity

Answer 37

A form of criterion-related validity that is an index of the degree to which a test score predicts some criterion measure.

Question 38

Ranking

Answer 38

The ordinal ordering of persons, scores, or variables into relative positions or degrees of value.

Question 39

Rating

Answer 39

A numerical or verbal judgment that places a person or attribute along a continuum identified by a scale of numerical or word descriptors called a rating scale.

Question 40

Rating Error

Answer 40

A judgment that results from the intentional or unintentional misuse of a rating scale; two types of rating error are lenience error (or generosity error) and severity error.

Question 41

Rating Scale

Answer 41

A system of ordered numerical or verbal descriptors on which judgments about the presence/absence or magnitude of a particular trait, attitude, emotion, or other variable are indicated by raters, judges, examiners, or (when the rating scale reflects self-report) the assessee.

Question 42

Severity Error

Answer 42

Less than accurate rating or error in evaluation due to the rater’s tendency to be overly critical.

Question 43

Slope Bias

Answer 43

a

Question 44

Test Blueprint

Answer 44

A detailed plan of the content, organization, and quantity of the items that a test will contain.

Question 45

Validation

Answer 45

The process of gathering and evaluating validity evidence.

Question 46

Validation Study

Answer 46

Research that entails gathering evidence relevant to how well a test measures what it purports to measure for the purpose of evaluating the validity of a test or other measurement tool.

Question 47

Validity

Answer 47

A general term referring to a judgment regarding how well a test or other measurement tool measures what it purports to measure; this judgment has important implications regarding the appropriateness of inferences made and actions taken on the basis of measurements.

Question 48

Validity Coefficient

Answer 48

A correlation coefficient that provides a measure of the relationship between test scores and scores on a criterion measure.

Question 49

Base Rate

Answer 49

An index, usually expressed as a proportion, of the extent to which a particular trait, behavior, characteristic, or attribute exists in a population.