lec 7 Flashcards
The tasks that constructed-
response (CR) items require of the student can be….
can be as varied as
the skills measured
What are the two Restricted CR items?
- Completion (fill-in the blank) items
- Short answer items
Restricted CR items are typically used to measure
simple learning outcomes
Extended CR items response length
several paragraphs or more
what do extended CR items measure?
they measure student’s ability to analyse evaluate and/or create.
extended CR are typically used to measure:
complex learning outcomes
Completion items are based on a
single word, phrase, or
number, which may require
o basic calculations
o remembering a definition
o labeling a diagram, etc.
Completion items typically measure what levels of
Bloom’s Taxonomy?
First three levels
(remembering, understanding, and
applying).
strengths of Restricted Constructed Responses
*Guessing is less likely,
compared to selected-
response items.
* A range of outcomes can
be measured.
* Directions impose
boundaries and
constraints so that the
response is limited in scope
Limitations of restricted constructed responses:
- It is difficult to phrase
statements so that only one
answer is correct. - Scoring could be time-
consuming and
contaminated by spelling
ability. - This item type is not
adaptable to measuring the
higher levels of Bloom’s
Taxonomy.
How to develop extended CR items:
- Describe the task clearly. Directions should give some
freedom to determine the form and scope of response. - Provide the approximate time allotted.
- Assess an item’s quality before giving it to students by
composing, mentally or in writing, a possible response. - Identify the criteria which will be used to assess the answer.
- It is necessary to impose some restrictions for these items,
such as time, number of pages, and allowable content.
Restricted CR items: example
complete the following
dialogue between a hotel
receptionist and a guest.
The receptionist’s questions
are provided. You must
provide the guest’s
responses in Spanish.
extended CR item: example
Using your Spanish
language skills, create a
dialogue between a hotel
receptionist and a guest
who arrives seeking a room
for one week.
Restricted response Vs extended response example how to “Bump up”
Restricted Response:
With your dance
partner, perform the
ballroom dance steps
as learned in class.
Extended response:
Working in groups,
select a musical
composition and
create a line-dance
that reflects the spirit
of the music chosen.
when should guided words be provided? and what do they do?
Guiding words provided immediately after the assigned task can
help remind writers of things to be cognizant of as they create
their response.
level/order of planning for guiding words:
*Plan your composition
*Focus on the assigned ask
*Establish a clear position or thesis that will unify your composition
*Organize your composition in a manner that will best allow you
to defend your position
*Defend your position by using supportive and specific evidence
*Edit and proofread your composition`
analytical rubric for CR items:
Analytical Rubric
* Assigns points to each factor
* Identifies students’ strengths
and weaknesses
* Ignores overall quality of
response
* Usually best suited for an
extended-constructed
response
Holistic rubric for Extended CR items:
Holistic Rubric
* Focuses on the response as a
whole
* Reflects overall performance
* Usually best suited for a
restricted-constructed
response
Constructed-response items tend to have (in relation to reliability)
lower reliability than selected-response items because the
scoring is more subjective.
4 types of scoring erros:
1.Halo effect
2. Item to item carry over effects
3. test to test effects
4.slide effect
Halo effect:
Extraneous factors (i.e., student behavior,
penmanship, etc.) may influence the scoring.
Item-to-item carry-over effects:
A response to one item tends to influence the assessment of a student’s subsequent responses.
Test-to-test effects:
Tests may be judged against a previously-
scored test, rather than against established standard.
“Slide” effect:
Essay scores tend to decrease as the number of
essays scored increases because markers may get less tolerant if
they are seeing the same mistakes frequently repeated.
Extra advice when marking constructed response:
More advice when Scoring Constructed Responses
If students respond to several item, score all responses to one item
before scoring the next (to avoid item-to-item carry-over effect).
If possible, do not look at the identity of the students (remember the
halo and test-to-test effects).
Use multiple raters where possible (e.g., ask a colleague for a blind
review of the same task, at least for a few students).
For extended responses it is a good idea to do an initial scoring and
sorting of responses into “high” “mid” and ”low” groupings then
assign scores with a second look at papers in each initial grouping.
do not always start scoring with the same students, for example of
you marked the first task starting with student “A” and finished at
student “Z”, reverse the order for the next scoring sequence.
If a rubric needs improvements note the changes for future use.
Changing a rubric “midstream” is problematic given that students will
have used the rubric as a guide in the creating their response..
Inductive approach:
-Begin with a Question
-Gather Evidence
-Develop a Theory
A “Bottom-up” Approach
”Exploratory”
Deductive approach
-Begin with a Hypothesis
-Provide Evidence
-Confirm a Theory
A “Top Down” Approach
”Presumptive”
Types of learning outcomes measured in Selected Items
-(Mostly) Remembering,
understanding, and
applying levels.
-X Inadequate for
organizing and expressing
complex ideas
Types of Learning
Outcomes
Measured for COnstructed items:
-Best for ability to
organize, integrate, and
express ideas.
-X Inefficient and time-
consuming for measuring
the remembering leve
Amount of Content
Sampled Selected items:
Large number of items
results in a broad coverage
and makes representative
sampling of content
feasible
Amount of content sampled: Constructed items
Small number of items
limits coverage and
makes representative
sampling of content
unfeasible
Ease of item preparation: Selected response
difficult and time consuming
ease of item preparation:
Constructed items
Difficult but easier than
selected-response items
Ease of scoring: Selected items
Objective, simple, and
highly reliable
Ease of scoring constructed items:
More subjective,
difficult, and less reliable
Potential sources
of error: Selected items
Reading ability and
guessing
Potential sources
of error constructed items:
Writing ability and
bluffing
Influence on
student learning: Selected items
Encourages students to
remember, interpret, and
use the ideas learned in
class
Influence on
student learning: constructed items
Encourages students to
organize, integrate, and
express their own ideas
using what is learned in
class
