Instructional Design & Item Analysis Flashcards
Recall /regurgitate facts without understanding. Exhibits previously learned material by recalling facts, terms, basic concepts and answers ?
Knowledge
To show understanding finding information from the text. Demonstrating basic understanding of facts and ideas ?
Comprehension
To use in a new situation. Solving problems by applying acquired knowl-edge, facts, techniques and rules in a different way.
Application
Key Words
Choose
Copy
Define
Duplicate
Find
How
Identify
label
List
Listen
Locate
march
Memorise
Name
Observe
Omit
Quote
Read
Recall
Recite
Recognise
Record
Relate
Remember
Repeat
Reproduce
Retell
Select
Show
Spell
Tell
Trace
What
When
where
Which
Who
Why
Knowledge
Key Words
Ask
Cite
Classify
Compare
Contrast
Demonstrate
Discuss
Estimate
Explain
Express
Extend
Generalise
Give examples
Illustrate illustrate
Indicate
Infer
Interpret
Match
Observe
Outline
Predict
Purpose
Relate
Rephrase
Report
Restate
Review
Show
Summarise
Translate
Comprehension
Key Words
Act
Administer
Apply
Associate
Build
Calculate
Categorise
Choose
Classify
Connect
Construct
Correlation
Demonstrate
Develop
Dramatise
Employ
Experiment with
Group
Identify
Illustrate
Interpret
Interview
Link
Make use of
Manipulate
Model
Organise
Perform
Plan
Practice
Relate
Represent
Summarise
Teach
Transfer
Translate
Use
Application
This is the most common, convenient, and concise method of reporting and provides a reasonable, interpretable evaluation report of a students’ performance. This is usually reported as a number (percentage) or in a small number of categories.
Single marks
This is the most commonly used, usually on a 100-point scale, with grades above a certain point(ex. 70 or 75%) considered as “passing” and those below as “failing.” It relates achievement to the extent of mastery of what was set out to be learned.
Percentage grades
Grades are reported only in two groups, such as pass or fail; satisfactory or unsatisfactory; credit and no credit. It is favored by some teachers because it eliminates or reduces competition for grades among students and its bad effects, such as rote learning and cramming. However, it may encourages students to explore new areas of study without fear of lowering their grade. This type of reporting may reduce their efforts to study because minimum effort is needed to obtain satisfactory grades. Also, incomplete information is provided, so they are difficult to average.
Two- step system
Three steps (honors, pass, fail); four steps (P+,P, P-, F); and five steps (A,B,C,D,E or numbers) may be utilized because they provide enough differentiation without having to go for very fine discriminations. Again, the grades have to be transformed to numbers before they can be averaged.
Three-or-more-step system
This system provides a descriptive summary of students’ overall performance without using numbers or letters. It provides more specific detailed information and emphasizes specific behavior resulting in reduced emphasis on grades. This system however, is not feasible in a large class and teachers may rely on a small number of vague phrases to describe students.
Narrative or descriptive system
The students’ performance is described in words and phrases, usually in the form of an informal letter to parents. They are comprehensive and flexible but the description of weaknesses is easily misinterpreted. They are recommended for use in a supplementary role to clarify specific points.
Narrative
This is a list of behavioral descriptions or performance objectives that are rated according to specific categories by teachers as they apply to each student. These scales are most useful in health professions education because of their ability to determine students’ skills and attitudes.
Checklist and rating scale
It is used to overcome the limited information that is provided by the traditional report card and to establish more cooperation between parents and teachers.
Parent- teacher conference
There are two general systems utilized for deciding the passing mark- the norm- reference system and the criterion reference system. (Please refer to table “Comparison of norm- reference and criterion- reference systems of grading” attached on last page).
Determining the passing mark
This is a derived scoring system in which the population f scores is converted to a scale of nine equal units, with each unit being one half of a standard deviation. In a normal curve, the mean is five and the standard deviation is two.
Stanines (derived from standard nines)
In a normal curve, the mean of stanine scores is five and the standard deviation is two.
this system indicates a learner’s position in a group by showing how far this raw score is above or below the mean. Usually this expresses test performance in terms of standard deviation units from the mean.
Standard score
this score indicates a learner’s relative position in a group in terms of the percentage of other students scoring below him. For example, a student with a raw score of 165 and places 90th percentile means that ninety percent of all scores are below 165. This scoring system is widely applied in the norm- reference grading system. Thus, when we interpret a percentile rank, we must always refer to the norm group on which it is based. Also, percentile units are not equal on all parts of the scale. For example, since in most tests a larger number of students tend to obtain scores near the middle rank, a percentile difference in the middle of the scale would represent a much smaller difference in test performance than the same percentile difference near the end of that scale.
Percentile mark
This is a numerical report of the tests performance in a score scale that has well defined characteristics and yields normative meaning. The most common are the percentile rank and standard score
Derived score
This is the number of points received by a student on a test when the test has been scored according to specified directions. It is merely a numerical summary of performance and provides little meaning since it is hard to interpret whether it is high or low, or passing or fail.
Raw score
What system utilizes a combination of reporting methods, thus combining the advantages and overcoming the limitations that may arise from the use of a single reporting method ?
Multiple reporting systems
This method is utilized only for multiple- choice questions. The steps are:
1. For each item, determine which option a borderline student can reject outright as wrong.
2. Determine the acceptability index (Ai) of the item where:
1
Ai= ——————————
No. of items not rejected
3. Determine the MPL of the test where
Ai for all items x 100 % MPL= —————————- Total no. of questions
Nedelsky’s method
This may be used for any type of objective question. The steps are:
1. Assess each question individually
2. Determine the percentage of question similar as the item analyzed as answerable by a borderline student, or determine the percentage of 100 borderline students who will be able to answer the item correctly.
3. Determine the MPL of the test as follows:
MPL of all items x 100 MPL= ——————————
No. of items
B. Angoff’s method
This may also be used for any type of tests The steps are:
- Classify items into:
Relevance: essential, important, acceptable, questionable, or
must- know, useful to know, nice to know Difficulty: easy, moderately hard, difficult - Determine the percent probability that a borderline student can answer each group of questions correctly
- Determine the MPL where
MPL= MPL for each group x no. of items in each group
Ebel’s method
What is defined as the process of applying statistical techniques to assess the difficulty and discriminating power of each item in order to improve their quality?
Item Analysis
This index indicates how easy or difficult a test question is, taking into account all the correct answers given by the examinees.
The difficulty index
This index indicates how significantly a question discriminates between “high” and “low” scoring students.
Key words:
Agree
Appraise
Argue
Assess
Award
Bad
Choose
Compare
Conclude
Consider
Convince
Criteria
Criticise
Debate
Decide
Deduct
Defend
Determine
Disprove
Dispute
Effective
Estimate
Evaluate
Explain
Give reasons
Good
Grade
How do we know?
Importance
Infer
Influence
Interpret
Judge
Justify
Mark
Measure
Opinion
Perceive
Persuade
Prioritise
Prove
Rate
Recommend
Rule on
Select
Support
Test
Useful
Validate
Value
Why
Evaluation
To justify. Presenting and defending opinions by making judgements about information, validity of ideas or quality of work based on a set of criteria.
Evaluation
Key words:
Adapt
Add to
Build
Change
Choose
Combine
Compile
Compose
Construct
Convert
Create
Delete
Design
Develop
Devise
Discover
Discuss
Elaborate
Estimate
Experiment
Extend
Formulate
Happen
Hypothesise
Imagine
Improve
Innovate
Integrate
Invent
Make up
Maximise
Minimise
Model
Modify
Original
Originate
Plan
Predict
Produce
Propose
Reframe
Revise
Rewrite
Simplify
Solve
Speculate
Substitute
Suppose
Tabulate
Test
Theorise
Think
Transform
Visualise
Synthesis
To change or create into something new. Compiling information together in a different way by combining elements in a new pattern or proposing alternative solutions.
Synthesis
Key words:
Analyse
Appraise
Arrange
Assumption
Breakdown
Categorise
Cause and effect
Choose
Classify
Differences
Discover
Discriminate
Dissect
Distinction
Distinguish
Divide
Establish
Examine
Find
Focus
Function
Group
Highlight
In-depth discussion
Inference
Inspect
Investigate
Isolate
List
Motive
Omit
Order
Organise
Point out
Prioritize
Question
Rank
Reason
Relationships
Reorganise
Research
See
Select
Separate
Similar to
Simplify
Survey
Take part in
Test for
Theme
Comparing
Analysis
