Chapter 10 - Developing a Test

Question 1

What are the 10 steps of the test development process?

Answer 1

1. Define the test universe, target audience, and testing purpose.
2. Develop a test plan.
3. Compose the test items.
4. Write administration instructions.
5. Conduct pilot tests.
6. Conduct item analysis.
7. Revise the test.
8. Validate the test.
9. Develop the norms.
10. Compile test manual.

Question 2

How do they define the test universe?

Answer 2

• The developer prepares a working definition of the construct that the test will measure.
• May involve a literary review of a psychological construct.

Question 3

How do they define a target audience?

Answer 3

• The developer makes a list of the characteristics of the persons who will take the test.
• Particularly those characteristics that affect how test takers will respond to each question.
• May also have to consider the reading level, potential disabilities, or motivations.

Question 4

How do they define the test purpose?

Answer 4

• The purpose includes not only what the test will measure, but also how the test users will use the test scores.
• Could take the normative, ipsative, or criterion approach.

Question 5

Normative Approach

Answer 5

• Comparing test takers to other test takers.
• Ex: Who receives the highest score will get the job.

Question 6

Ipsative Approach

Answer 6

• Compre a test taker standing on one trait with their own standing on other traits.
• Ex: How many pushups someone can do in a minute.

Question 7

Criterion Approach

Answer 7

• Used to indicate achievement.
• Ex: Must get at least 50% to pass the exam.

Question 8

How do they create a test plan?

Answer 8

• Specify the characteristics of the test, including an operational definition of the construct and the content to be measured, the format of the questions, and the administration and scoring of the test.

Question 9

How do they define a construct?

Answer 9

• Reviewing the literature about the construct and any available tests.
• Creating a concise definition of the construct.
• Includes operationalizing the construct in terms of observable and measurable behaviours.
• Also provides boundaries for the test / what should be tested and what should be excluded from the content.

Question 10

How do you choose the test format?

Answer 10

• Refers to the type of questions that the test will contain.
• Test formats provide two elements: A stimulus to which the test taker responds and a mechanism for response.

Question 11

Objective Test Format

Answer 11

• One response that is designated as “correct” or provides evidence of a specific construct.
• Ex: Multiple choice questions

Question 12

Subjective Test Formats

Answer 12

• Do not have a single response that is designated as “correct”.

Question 13

Projective Tests

Answer 13

• Subjective test formats because the stimuli for these tests are ambiguous pictures.
• Ex: Essay questions

Question 14

How do we choose how to administer a score a test?

Answer 14

• Important because the plan will influence the format and content of the test items.
• How will the test be taken, how long, will it be administered in a group or individually, how will the test be scored, and what data can we expect?

Question 15

The Cumulative Model of Scoring

Answer 15

• Probably the most common method for determining an individual’s final test score.
• Can yield interval-level data.

Question 16

The Categorical Model of Scoring

Answer 16

• Used to place test takers in a particular group or class.
• Typically yields interval-level data.

Question 17

The Ipsative Model of Scoring

Answer 17

• Differs from the cumulative and categorical models.
• The test taker is usually presented with two to four statements in a forced choice format.
• Each test item will contain statements associated with more than one trait or construct.
• The computer based on a total number of points each scale in the test received.

Question 18

Pilot Test

Answer 18

• A scientific evaluation of the test’s performance - followed by revisions to determine the final form that the test will take.

Question 19

Test Items

Answer 19

• The stimulus or questions in a test.

Question 20

How do they choose the item format?

Answer 20

• Based on information in the test plan, such as the target audience, method of administration, and requirements for scoring.

Question 21

Examples of Objective Items

Answer 21

• Multiple Choice
• True or False
• Forced Choice

Question 22

Examples of Subjective Items

Answer 22

• Essay Questions
• Interview Questions
• Projective Techniques
• Sentence Completion

Question 23

Performance Assessments

Answer 23

• Require test takers to directly demonstrate their skills and abilities to perform a group of complex behaviours and tasks.
• Complex Item

Question 24

Simulation

Answer 24

• Similar to a performance assessment in that it requires test takers to demonstrate their skills and abilities to perform a complex task.
• Not performed in the actual environment for safety or cost concerns.

Question 25

Portfolio

Answer 25

• Collection of work products that a person gathers over time to demonstrate his or her skills and abilities in a particular area.

Question 26

Response Sets

Answer 26

• Patterns of responding that result in false or misleading information.
• These sources of error limit the accuracy and usefulness of test scores.

Question 27

Social Desirability

Answer 27

• Test takers to provide or choose answers that are socially acceptable or that present themselves in a favorable light.

Question 28

Acquiescence

Answer 28

• The tendency to agree with any ideas or behaviours presented.

Question 29

Random Responding

Answer 29

• Responding to items in a random fashion by marking answers without reading or considering them.

Question 30

Faking

Answer 30

• Refers to the inclination of some test takers to try and answer items in a way that will cause a desired outcome or diagnosis.

Question 31

Constucts

Answer 31

• Tools that help us understand human behaviour.

Question 32

Brevity

Answer 32

• Use concise and exact language. Short questions reduce errors.
• Important when writing questions.