Chapter 4 Part 1: Reliability, Validity, and Utility

Question 1

The agreement between a test score or measure and the quality it is believed to measure.

Answer 1

Validity.

Question 2

This refers to the degree to which an assessment tool produces consistent results when repeated measurement are made.

Answer 2

Consistency.

Question 3

True or False. A weighing scale always measures a person to weigh 67kg when they are in fact 60kg.

The weighing scale is consistent but not valid.

Answer 3

True.

Question 4

The four (4) different aspects of validity.

Answer 4

1. Face validity.
2. Content-related evidence for validity.
3. Criterion-related evidence for validity.
4. Construct-related evidence for validity.

Question 5

This is not really validity at all because it does not offer evidence to support conclusions drawn from test scores.

Answer 5

Face validity.

Question 6

This considers the adequacy of representation of the conceptual domain the test is designed to cover.

Answer 6

Content-related evidence for validity.

Question 7

True or False: You are taking an examination with coverage on the first 6 chapters of Theories of Personality (example). Suddenly, a test item about a chapter not yet discussed in class showed up.

The test no longer has content-related evidence for validity.

Answer 7

True.

Question 8

This tells how well a test corresponds with a particular criterion, and such evidence is provided by high correlations between a test and a well-defined criterion or measure.

Answer 8

Criterion-related evidence for validity.

Question 9

The two (2) types of criterion-related evidence for validity.

Answer 9

1. Concurrent validity
2. Predictive validity.

Question 10

This refers to the forecasting function of tests.

For example, the NMAT might predict how well a high school student will perform in college.

Answer 10

Predictive validity.

Question 11

In this kind of validity, the measures and criterion measures are taken at the same time.

For example, a student with a learning disability might take a diagnostic test and their school tests in the same day to determine the relationship between their learning disability and performance in school.

Answer 11

Concurrent validity.

Question 12

This is established through a series of activities in which a researcher simultaneously defines some construct and develops the instrumentation to measure it.

Answer 12

Construct-related evidence for validity.

Question 13

This involves assembling evidence about what a test means by showing the relationship between the test and other tests and measures.

Answer 13

Construct validation.

Question 14

The two (2) types of construct-related evidence for validity are?

Answer 14

1. Convergence evidence.
2. Discriminant evidence.

Question 15

This refers to when a measure correlates well with other tests believed to measure the same construct.

Answer 15

Convergence evidence.

Question 16

This refers to when a test should have low correlations with measures of unrelated constructs, or evidence for what the test does not measure.

Answer 16

Discriminant evidence / divergent validation.

Question 17

Six (6) guidelines for item development / item writing.

Answer 17

1. Define clearly what you want to measure.
2. Generate an item pool.
3. Avoid exceptionally long items.
4. Keep the level of reading difficulty appropriate for your intended test-takers.
5. Avoid “double-barreled” items.
6. Consider mixing positively and negatively worded items.

Question 18

A term for a test item that conveys two (2) or more ideas at the same time.

Answer 18

Double-barreled item.

Question 19

An item format that offers two (2) alternatives for each item.

Answer 19

Dichotomous format.

Question 20

Resembles the dichotomous format except that each item has more than two (2) alternatives.

Answer 20

Polytomous format.

Question 21

A popular format for attitude and personality scales that requires a respondent to indicate the degree of agreement with a particular attitudinal question.

Answer 21

Likert format.

Question 22

Similar to the Likert format, but has even more choices.

Answer 22

Category format.

Question 23

A format common in personality measurement in which a subject receives a long list of adjectives and indicates whether each one is characteristic of himself/herself or someone else.

Answer 23

Adjective checklist (checklists and q-sorts).

Question 24

In this format, a subject is given statements and asked to sort them into nine (9) piles. This can be used to describe oneself or to provide ratings of others.

Answer 24

Q-sort.

Question 25

A general term for a set of methods used to evaluate test items.

Answer 25

Item analysis.