Chapter 4 Part 1: Reliability, Validity, and Utility Flashcards
The agreement between a test score or measure and the quality it is believed to measure.
Validity.
This refers to the degree to which an assessment tool produces consistent results when repeated measurement are made.
Consistency.
True or False. A weighing scale always measures a person to weigh 67kg when they are in fact 60kg.
The weighing scale is consistent but not valid.
True.
The four (4) different aspects of validity.
- Face validity.
- Content-related evidence for validity.
- Criterion-related evidence for validity.
- Construct-related evidence for validity.
This is not really validity at all because it does not offer evidence to support conclusions drawn from test scores.
Face validity.
This considers the adequacy of representation of the conceptual domain the test is designed to cover.
Content-related evidence for validity.
True or False: You are taking an examination with coverage on the first 6 chapters of Theories of Personality (example). Suddenly, a test item about a chapter not yet discussed in class showed up.
The test no longer has content-related evidence for validity.
True.
This tells how well a test corresponds with a particular criterion, and such evidence is provided by high correlations between a test and a well-defined criterion or measure.
Criterion-related evidence for validity.
The two (2) types of criterion-related evidence for validity.
- Concurrent validity
- Predictive validity.
This refers to the forecasting function of tests.
For example, the NMAT might predict how well a high school student will perform in college.
Predictive validity.
In this kind of validity, the measures and criterion measures are taken at the same time.
For example, a student with a learning disability might take a diagnostic test and their school tests in the same day to determine the relationship between their learning disability and performance in school.
Concurrent validity.
This is established through a series of activities in which a researcher simultaneously defines some construct and develops the instrumentation to measure it.
Construct-related evidence for validity.
This involves assembling evidence about what a test means by showing the relationship between the test and other tests and measures.
Construct validation.
The two (2) types of construct-related evidence for validity are?
- Convergence evidence.
- Discriminant evidence.
This refers to when a measure correlates well with other tests believed to measure the same construct.
Convergence evidence.
This refers to when a test should have low correlations with measures of unrelated constructs, or evidence for what the test does not measure.
Discriminant evidence / divergent validation.
Six (6) guidelines for item development / item writing.
- Define clearly what you want to measure.
- Generate an item pool.
- Avoid exceptionally long items.
- Keep the level of reading difficulty appropriate for your intended test-takers.
- Avoid “double-barreled” items.
- Consider mixing positively and negatively worded items.
A term for a test item that conveys two (2) or more ideas at the same time.
Double-barreled item.
An item format that offers two (2) alternatives for each item.
Dichotomous format.
Resembles the dichotomous format except that each item has more than two (2) alternatives.
Polytomous format.
A popular format for attitude and personality scales that requires a respondent to indicate the degree of agreement with a particular attitudinal question.
Likert format.
Similar to the Likert format, but has even more choices.
Category format.
A format common in personality measurement in which a subject receives a long list of adjectives and indicates whether each one is characteristic of himself/herself or someone else.
Adjective checklist (checklists and q-sorts).
In this format, a subject is given statements and asked to sort them into nine (9) piles. This can be used to describe oneself or to provide ratings of others.
Q-sort.
A general term for a set of methods used to evaluate test items.
Item analysis.
