Lec 8 Flashcards
what are the strengths of selected response items?
SR items can help identify student misconceptions and errors.
✓ SR items are useful when there are multiple possible distractors.
✓ A relatively large number of SR items can be answered in a
typical testing period.
✓ Scoring SR items is easy, objective, and reliable.
✓ Possible to test a wide range of knowledge and skills, including
higher-level thinking.
✓ SR items can eliminate the influence of students’ writing skills
on their responses.
What are the linitations of selected response items?
× Guessing becomes a major concern because
students can select the correct answer by chance.
× Writing items beyond the “remembering” and
“understanding” levels is difficult, and it is not possible
to build items at the “creating” level.
× Building high-quality SR items (especially multiple-
choice) is difficult and time-consuming.
× Poorly constructed SR items provide unreliable results
and invalid inferences, creating frustration and
confusion.
× Testwiseness can be a serious problem if items and
exams are not designed carefully.
3 types of selected response items:
- Alternative response: only two options (e.g., True/False)
- Matching: many options that need to be matched by the
student. - Multiple-choice: one correct and several incorrect response
options.
* Items without a source
* Items with a source (e.g., a table, a figure, an excerpt)
Alternative response items is also known as:
Binary- choice items
alternative response items are appropriate for measuring:
basic skills or knowledge at
the remembering or understanding levels of Bloom’s
taxonomy.
another limitation of altnerative response is that…
students may guess the answer
what are the characteristics of a quality alternate-response item?
Stem is appropriately worded & focused
▪ Use appropriate vocabulary and sentence structure.
▪ Avoid double negatives and try to word in the
positive.
▪ Ensure there is only one central idea in each
statement.
Best used when only two alternatives are possible
▪ Answer must be unequivocally correct.
Avoid making the true statements consistently
longer than the false items
how to enhance the reliability and validity of alternate response items?
Identify each item as either “T” (True) or “F” (False).
For items you identify as “F”, provide a corrected statement on the line below the item to make the item true
OR
Alternative version 2: An item measuring editing skills
- (A-Providing, B-Provided) that all homework is done, you may go to
the movie.
Strengths of alternative-response items?
Strengths:
* Useful when only two
alternatives are
possible
- Less demand placed
on reading ability
than multiple-choice - A relatively large
number of items can
be answered in a
testing period - Scoring is easy,
objective and
reliable
Limitations of alternative-response items?
Limitations
* Marking an item “false”
provides no evidence
that the student knows
what is the correct answer
- Difficult to write items
beyond the remembering
level that are free of
ambiguity - Scores influenced by
guessing more than any
other type of item
alternative response: An item is technically
sound when it…
An item is technically
sound when it…
✓ is free of verbal clues to
the answer
✓ is focused on a single,
central idea
✓ provides space in which to
record the answer
✓ has correct spelling and
grammar
alternative response: An item is equitable
when the item is…
An item is equitable
when the item is…
✓ free from racial, ethnic,
gender, sexual, religious or
other bias
✓ free of irrelevant material
✓ stated in appropriate and
clear language
✓ free from pop cultural
references that would not
be familiar to all students
matching items is appropriate for measuring what concepts of blooms taxonomy?:
Knowledge of related concepts at the understanding and applying levels of blooms taxonomy.
an idea matching item usually has
5 to 6 pairs of words, phrases, or sentences.
Matching items work best when the content has many
parallel or related concepts: such as:
o Terms and definitions
o Objects or pictures and labels
o Symbols and proper names
o Causes and effects
o Scenarios and responses
o Principles and scenarios to which they apply
Characteristics of a quality matching item:
✓Item is on one page
✓Directions are clear and concise
✓Does not make reading ability part of assessment
✓provide more responses than premises and/or
allow students to select a response once, more
than once or not at all
✓Premises and responses
*are brief, similar in length, and
ordered
* are homogeneous within their
column
* make sense when reading
* are at a suitable reading level for the students
strengths of matching items:
- A compact and efficient
assessment - Reading and response
time is short - Are easily constructed if
converted from multiple-
choice items having a
common set of
alternatives - Scoring is easy, objective
and reliable
limitations of matching items:
Limitations
* This item type is largely
restricted to simple
knowledge outcomes
based on association
* It is difficult to construct
items that contain enough
homogeneous responses
* Susceptibility to irrelevant
clues is greater than in
other item types
* Guessing is still possible but
less impactful than for
alternative response and
multiple-choice items.
A matching item is is technically
sound when it…
An item is technically
sound when it…
✓ is free of verbal clues to the
answer
✓ provides premises and
responses in an order that is
logical and easily understood
✓ allows for responses to be
used more than once or
provides more responses than
premises
✓ uses homogeneous material
✓ is contained on a single page
✓ has correct spelling and
grammar
a matching item is is equitable
when the item is…
An item is equitable
when the item is…
✓ free from racial, ethnic, gender,
sexual, religious or other bias
✓ free of irrelevant material
✓ stated in appropriate and clear
language
✓ free from pop cultural
references that would not be
familiar to all students
multiple choice items are arguably the most
Multiple-choice items are arguably the most difficult assessment
instruments to make well, and poorly made items can provide
unreliable and invalid results and leave test-takers frustrated.
the anatomy of a multiple choice question:
- Source
- Stem
3.Distracters
A stem of a selected response item may be worded as
a question or a statement
stems should be
concise but provide enough context so that test takers can visualize the correct response
stem item checklist:
- The stem is focused on a single and meaningful problem.
- The stem is concise and clear.
o Avoid lengthy, verbose stems in the items - The stem emphasizes keywords in the stem (e.g., “least”,
“most,” “not”).
o Either CAPITALIZE these keywords, use bold, or underline to highlight the word - The stem is free of verbal clues to the answer.
- The stem has correct spelling and grammar.
a multiple choice item is technically sound when it
✓is free of verbal clues to the
answer
✓is focused on a single,
meaningful problem
✓emphasizes key words in the
stem (e.g. “least” “most” “not”)
✓has plausible distracters
✓has alternatives that are
homogeneous and of similar
length
✓has alternatives that are in a
logical order
✓uses of ‘all of the above’ and
‘none of the above’ only if
necessary
✓has correct spelling and
grammar
a multiple choice item is equitable when the item is:
✓ free from racial, ethnic, gender,
sexual, religious or other bias
✓ free of irrelevant material
✓ stated in appropriate and clear
language
✓ free from pop cultural
references that would not be
familiar to all students
what is the overall advantages of selected response items?
Include their ease of scoring and the diagnostic information that can be gathered about students based on which distractors have been chosen.
