1-10 Flashcards
10 Steps of Test Development (first 5 in chapter 10)
1) defining the testing universe, audience, and purpose
2) Developing a test plan
3) composing the test items
4) writing the administration instructions
5) conducting the pilot test
issues related to “defining the test purpose”
normative approach vs criterion approach ??
how do we pick which decision to use
normative approach
the scores be used to compare test takers with other test takers
the scores be used to compare test takers with other test takers
normative approach
criterion approach
compare two groups, test used to indicate achievement
compare two groups, test used to indicate achievement
criterion approach
pilot testing
a scientific evaluation of the test’s performance, little experiment before the main one
a scientific evaluation of the test’s performance, little experiment before the main one
pilot testing
What are the step involved in developing a test plan?
defining the testing universe
defining the target audience
defining the test purpose
What characteristics of a test does the test plan specify?
defining the construct to be measured
defining the content to be measured
defining the construct to be measured
???
???
defining the construct to be measured
defining the content to be measured
???
???
defining the content to be measured
objective items: strength/weaknesses
multiple choice: scoring is easy/can guess right (this is offset by having a lot of questions)
true/false: scoring is easy/can guess right
forced-choice: good for personality and attitude tests, good for when the test-taker answers dishonestly/little face validity (no apparent connection with the state purpose of the test)
subjective items: strength/weaknesses
essay questions: have an in-depth response/harder to score
interview questions: wide range of responses/hard to score
response bias
patterns of responding that result in false or misleading information
limit the accuracy and usefulness
social desirability, acquiescence, random responding, faking
patterns of responding that result in false or misleading information
limit the accuracy and usefulness
social desirability, acquiescence, random responding, faking
response bias
social desirability
present in a favorable light
present in a favorable light
social desirability
acquiescence
tendency to agree with ideas presented
tendency to agree with ideas presented
acquiescence
random responding
marking answers randomly
marking answers randomly
random responding
faking
answering in a way to cause a desired outcome or diagnosis
answering in a way to cause a desired outcome or diagnosis
faking
Writing Effective Items
Write twice as many items as needed
Identify item topics by consulting test plan
Base each item on an important learning objective or topic
Write items that assess information or skills drawn only from the testing universe
Write each item in a clear and direct manner
Use vocabulary and language appropriate for target audience
Avoid using slang or colloquial language
Make all items independent
Ask someone else to review items to reduce unintended ambiguity andinaccuracies
writing effective items: multiple choice/true false
Avoid using negative stems and responses
Make all responses similar in detail and length
Make sure the items has only one correct / best answer
Avoid using words like “always” and “never”
Avoid overlapping responses
Avoid using inclusive distractors such as “all of the above”
Use random assignment to position correct response
Writing effective items: essay/interview
Use essay items appropriately
Consider the time necessary for response
Prepare an answer key
Score essays anonymously
Use multiple independent scorers
What should be included in the instructions?
Whether the test should be administered in a group and/or individually
Specific requirements for the test administration location, such as privacy; quiet; and comfortable chairs, tables, or desks
Required equipment such as No. 2 pencils, a computer with a removable storage device such as an SD card or flash drive and/or Internet access
Time limitations or the approximate time for completing the test when there are no time limits
A script for the administrator to read to test takers, including answers to questions that test takers are likely to ask
Credentials or training required for the test administrator
