Chapter 13 Flashcards
Test items must always be based on specific learning objectives.
13
Kirkpatrick’s four levels of evaluation guide an instructor in determining whether a student has learned necessary material.
Reaction, learning, behavior, results
Both students’ and instructors’ work is easier when test sheets have proper formatting and layout.
13
Test items should evaluate the student’s ability at the level within the taxonomy that corresponds to the learning objective being evaluated.
13
The actual determination of test difficulty is the responsibility of a level III instructor.
13
In terms of time requirements, benchmarks for answering certain types of questions are as follows:
True-false - 15 seconds for true answers
True-false (correct the statement) 30-45 seconds
Seven-item matching - 60-90 seconds
Multiple-choice - 30-60 seconds
Problem solving- 30-60 seconds
Short answer- 30-60secs
Essay - 60 seconds
13
Tests should be an appropriate length to address the learning objectives that the test is intended to evaluate.
13
When time is a restrictive factor, tests can emphasize the most critical learning objectives and include a sampling of less important objectives. This method of test construction is called sampling.
13
In the fire and emergency services, bias is generally limited to use of jargon and differences in terminology.
13
Written tests measure students’ understanding and retention of technical information and evaluate their accomplishment of the cognitive learning objectives.
13
An objective test item is a question for which there is only one correct answer.
13
Objective items measure cognitive learning but typically only at the lower levels of remembering and understanding.
13
However, properly constructed objective test items can also be used to measure higher levels of cognitive learning such as evaluation or creation.
13
There are three main types of objective questions:
Multiple choice questions
True or false
Matching
13
A subjective test item has no single correct answer.
13
Subjective items are an effective way of measuring higher cognitive levels because they allow students the freedom to organize, analyze, revise, redesign, or evaluate a problem.
13
Questions that are published in study guides, test preparation guides, textbooks, and manuals should not be used for certification or summative tests.
13
A multiple-choice test item consists of either a question or an incomplete statement, called the stem, plus a list of several possible responses, which are referred to as choices or alternatives.
13
Distractors are used to discriminate between students who understand the subject matter well and those who are uncertain of the correct answer.
13
Write the stem in the form of a direct question or an incomplete sentence that measures only one learning objective.
13
One limitation of true-false questions is that students tend to remember the false items on the test as being true, a phenomenon known as the negative suggestion effect.
13
Make sure that stems and alternatives do not give students grammatical clues as to the correct response.
13
Instructors should be advised that matching test items may be more effectively and efficiently written as a series of multiple-choice questions.
13
The interpretive exercise is another subjective test item that measures higher level cognitive process.
13
When the purpose of the test is to determine knowledge, the questions should be closed, requiring only a single brief answer.
13
When the purpose is to determine how a student responds under pressure, the question should judge both accuracy and presentation.
13
Base oral questions on standard criteria and performance objectives.
13
Even though scoring is subjective and testing conditions may cause anxiety, an oral test is the most valid and reliable way to test a student’s ability to verbally communicate ideas, concepts, and processes.
13
A performance or skills test measures a student’s proficiency in performing any task that involves a psychomotor objective.
13
Assessment is based on either a speed standard such as timed performance, a quality standard, such as minimum acceptable performance, or both.
13
Performance tests require students to demonstrate psychomotor proficiency after appropriate practice or drill sessions.
13
Validity - A performance test is the only valid method of measuring a student’s ability to perform manipulative skills.
13
Reliability - A properly constructed performance test using specific criteria is a reliable measure of performance when coupled with an appropriate rating scale.
13
Using checklists for evaluation helps to ensure that students are rated against a standard and not against one another.
13
Performance tests motivate students to practice skills, both in and out of class.
Student motivation
Students who successfully complete well-prepared and carefully administered performance tests will understand their level of achievement and be proud of their accomplishments.
13
Test items are not normally written to evaluate the affective domain because affective objectives are often difficult to measure.
13
Learning objectives that include a change in affective behavior can be evaluated based on instructor observations during training.
13
The purpose of a test refers to the overall expectation of student performance.
13
Compare performance against appropriate minimum standards.
Criterion-referenced tests
Rate student performance compared to other students.
Norm-referenced tests
Norm-referenced tests are very effective for promotional examinations when ranking applicants is desirable.
13
Norm-referenced tests should never be used as end-of-course or certification tests.
13
Given at the beginning of instruction to establish a student’s current level of knowledge.
Prescriptive
Quizzes, pop tests, or question/answer periods in class that are given throughout the course or unit of instruction, typically measure improvement and give the instructor and students feedback on learning progress.
Formative
Measure student achievement in an entire area on a number of topics covered over a long period of time such as semester or other major segment of a course.
Summative
After determining the purpose and classification of the test, the instructor must identify the learning objectives the test will evaluate.
13
Learning objectives reflect the course outcomes, which are broad statements explaining what students should have learned at the completion of a course.
13
When a test includes items with a range of difficulties, a student’s overall knowledge is tested.
13
Test item difficulty is often a factor of whether the test is criterion-referenced or norm-referenced.
13
Criterion-referenced tests are developed with the intention of evaluating whether students in a class have a working knowledge of a skill or data set.
13
On a criterion-referenced test, all test items should be designed so knowledgeable students should be able to pass the test.
13
Criterion-referenced test items should match the cognitive level of the learning objectives in the course.
13
The appropriate number of items for any test depends on the following factors:
Purpose of the test
Types of test items or performance items
Desired level of reliability
13
An objective that took 30 minutes to teach in a 120- minute course would be addressed by 25 percent of the test items.
13
Usability is an important factor to consider when developing a test, or when selecting a test that another organization developed.
13
The two most important characteristics of a well-designed test are validity and reliability.
13
Validity is the extent to which a test measures what it is supposed to measure.
13
One way to ensure validity is to include an ample number of test items for each learning level and content area.
13
Reliability is the extent to which a test provides consistent, accurate measurements of student achievement.
13
The final step in creating a test instrument is determining how to score it, which includes establishing the criteria for passing and failing.
13
There are three main types of subjective test items:
Short answer or completion
Essay
Interpretive exercise
13
The most common test types are written, oral, and performance.
13
Oral tests may vary according to the interpretation of the instructor.
13
