TEST DEVELOPMENT & NORM Flashcards by Ariel IPAD

The process of developing a test occurs in five stages:

test conceptualization (idea)
test construction (writing up)
test tryout (pilot testing)
item analysis
test revison

test administration (final) 50

How well did you know this?

Not at all

Perfectly

What stage of test development involves the initial idea and purpose of the test, including determining what the test will measure?

Test Conceptualization

(1st Stage)

How well did you know this?

Not at all

Perfectly

1st Stage - Test Conceptualization

What is the term for the preliminary research that surrounds the creation of a test prototype?

Pilot Work

How well did you know this?

Not at all

Perfectly

1st Stage - Test Conceptualization

What process involves evaluating test items to determine if they should be included in the final version of the test?

Pilot Study

How well did you know this?

Not at all

Perfectly

1st Stage - Test Conceptualization

What are the three key questions a test developer must ask during test conceptualization?

a) What is the test designed to measure?
b) What is the objective of the test?
c) Is there a need for this test?

How well did you know this?

Not at all

Perfectly

What stage in test development involves writing and structuring the test items based on the conceptualized idea?

Test Construction

(2nd Stage)

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

What is the process of setting rules for assigning numbers in measurement?

Scaling

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

Scaling Methods:

a. rating scale
b. Likert scale (Likert, 1932)
c. Method of Paired Comparisons
d. Comparative Scaling
e. Categorical Scaling
f. Guttman scale (1944, 1947)
g. Scalogram Analysis

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

What type of scale involves grouping words, statements, or symbols to indicate the strength of a trait, attitude, or emotion?

Rating Scale

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

A survey asks students to rate their stress level on a scale from 1 to 10, with 1 being “Not Stressed at All” and 10 being “Extremely Stressed.” What type of scaling method is used?

Rating Scale

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

Which scale presents test-takers with a five-point or seven-point response format to measure attitudes?

Likert Scale

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

A researcher asks participants to respond to the statement “I feel confident in my ability to solve problems” using the following options:
1 - Strongly Disagree
2 - Disagree
3 - Neutral
4 - Agree
5 - Strongly Agree
What type of scaling method is used in this survey?

Likert Scale

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

In which scaling method are test-takers presented with two stimuli and asked to compare them?

Method of Paired Comparisons

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

A hiring manager shows a candidate two job descriptions and asks, “Which role do you find more suitable for your skills?” What scaling method is being applied?

Method of Paired Comparisons

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

What type of scaling requires judgments of a stimulus in comparison with every other stimulus on the scale?

Comparative Scaling

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

A group of coffee lovers is asked to rank five coffee brands from least favorite to most favorite. What type of scaling method is used?

Comparative Scaling

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

What type of scaling involves categorizing stimuli into two or more alternatives, such as “Agree” or “Disagree”?

Categorical Scaling

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

A researcher presents a list of common workplace behaviors and asks employees to classify them as:

Ethical
Unethical
Depends on the Situation

What scaling method is used in this classification?

Categorical Scaling

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

Which scale arranges items sequentially from weaker to stronger expressions of an attitude, belief, or feeling?

Guttman Scale

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

A survey asks participants whether they agree or disagree with the following statements:

“All citizens should have access to basic healthcare.”
“The government should provide healthcare for low-income individuals.”
“Private insurance should be replaced with a universal healthcare system.”

If a participant agrees with statement 3, they are likely to agree with statements 1 and 2.

What type of scaling method is used?

Guttman Scale

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

What is the item-analysis procedure that involves a graphic mapping of a test-taker’s responses?

Scalogram Analysis

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction

A psychologist visually maps responses to a questionnaire on self-esteem to analyze patterns and consistency in participants’ answers. What type of scaling method is used?

Scalogram Analysis

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction (Writing Items)

What to consider when writing items?

What range of content should the items cover?
Which of the many different types of item formats should be employed?
How many items should be written in total and for each content area covered?

How well did you know this?

Not at all

Perfectly

2nd Stage - Test Construction (Writing Items)

What is the term for the reservoir of items from which the final test items will be selected?

Item Pool

How well did you know this?

Not at all

Perfectly

# 2nd Stage - Test Construction What term refers to the form, structure, and arrangement of individual test items?

Item Format

# 2nd Stage - Test Construction (Writing Items) Two Types of Item Format.

- Selected-Response Format - Constructed-Response Format

# 2nd Stage - Test Construction (Writing Items) Which type of item format requires test-takers to choose an answer from a given set of options?

Selected-Response Format

# 2nd Stage - Test Construction (Writing Items) Which type of item format requires test-takers to supply or create their own answer?

Constructed-Response Format

In which stage of test development is a prototype of the test administered to a sample group to assess its effectiveness before finalization?

Test Tryout | (3rd Stage)

# 3rd Stage - Test Tryout During test tryout, the test should be administered to individuals who are similar to the target test population. Why is this important?

To ensure the test is valid and appropriate for its intended users.

# 3rd Stage - Test Tryout How many subjects should ideally be included in a test tryout per test item?

No fewer than five, and preferably ten per item.

# 3rd Stage - Test Tryout Why should the conditions of a test tryout be as identical as possible to the standardized test administration conditions?

To maintain uniformity and reliability in test results.

What stage of test development involves selecting the best items from a pool of tryout items

Item Analysis | (4th Stage)

# 4th Stage - Item Analysis What statistical measure indicates the proportion of test-takers who answered an item correctly and helps determine whether an item is too easy or too difficult?

The Item-Difficulty Index

# 4th Stage - Item Analysis Which index provides an indication of a test’s internal consistency and helps determine if the test is reliable?

The Item-Reliability Index

# 4th Stage - Item Analysis What statistical tool is used to determine whether test items measure the same construct and help identify items that need to be revised or eliminated?

Factor Analysis

# 4th Stage - Item Analysis Which index provides an indication of how well a test measures what it is supposed to measure and contributes to a test’s criterion-related validity?

The Item-Validity Index

# 4th Stage - Item Analysis What measure evaluates how well a test item differentiates between high and low scorers, with a higher value indicating better discrimination?

The Item-Discrimination Index

# 4th Stage - Item Analysis If an item has a negative discrimination index (d-value), what does it indicate about low and high scorers?

It indicates that low scorers are more likely to answer the item correctly than high scorers, meaning the item should be revised or eliminated.

# 4th Stage - Item Analysis What graphical representation shows both item difficulty and discrimination?

The Item-Characteristics Curve

# 4th Stage - Item Analysis What type of item analysis involves non-statistical procedures such as interviews and discussions to explore how individual test items work?

Qualitative Item Analysis

# 4th Stage - Item Analysis What cognitive assessment technique involves having respondents verbalize their thoughts as they complete a test to provide insight into their reasoning process?

Think-Aloud Administration

# 4th Stage - Item Analysis What qualitative item analysis method involves gathering specialists to review test items and provide feedback on their quality?

Expert Panels

# 4th Stage - Item Analysis Types of Item Analysis and Index

a. The Item-Difficulty Index b. The Item-Reliability Index c. The Item-Validity Index d. The Item-Discrimination Index e. Item-Characteristics Curve f. Qualitative Item Analysis

It involves refining test items based on item analysis results, such as difficulty, reliability, validity, and discrimination indices, to improve the overall quality of the test.

Test Revision | (5th Stage)

# 5th Stage - Test Revision What process involves revalidating a test on a different sample of test-takers to ensure its predictive validity remains consistent?

Cross-validation

# 5th Stage - Test Revision What test validation process is conducted on two or more tests using the same sample of test-takers?

Co-Validation

What is the final stage in test development where the test is officially administered under standardized conditions to the target population?

Test Administration

Behavior that is usual, average, normal, standard, expected, or typical.

NORM

# NORM What refers to the test performance data of a specific group used as a reference for evaluating individual test scores?

Norms

# NORM What is the term for a group of individuals whose test performance is analyzed to serve as a reference for others?

Normative sample

# NORM What is the process of deriving norms called?

Norming

# NORM It may be modified to describe a particular type of norm derivation.

Norming

# NORM Types of Norms

* Percentiles * Developmental Norms: Age Norms & Grade Norms * National Norms * Subgroup Norms * Local Norms

# Types of Norms Which type of norm expresses a test score relative to the percentage of people who scored lower?

Percentiles

# Types of Norms Which type of norm is based on the typical performance of individuals at a specific age or grade level?

Developmental Norms

# Types of Norms What are the two types of developmental norms?

Age Norms & Grade Norms

# Types of Norms What type of norms represent the test performance of individuals from across a country?

National Norms

# Types of Norms What type of norms are established for specific groups within a larger population, such as gender or ethnicity?

Subgroup Norms

# Types of Norms What type of norms are created based on test performance within a particular region or institution?

Local Norms