Week 4 Flashcards

Question

What comprises a test item?

Answer 1

1. Stimulus 2. Response Format - Conditions governing responses 3. Scoring Procedures

Answer 2

The introductory statement or question that sets the context for a specific question or problem. - Can also be a picture (TAT) or apparatus (coloured blocks) with instructions for their use.

Answer 3

The ways in which respondents can answer questions, which can be structured or unstructured. Includes factors as whether the item on the test will be multiple-choice/SAQ etc.

Answer 4

Any factors that could influence responses. Factors such as: - Whether there is a time limit for responding - Whether the administrator can probe ambiguous responses - Exactly how the responses will be recorded (e.g., answer sheet, test booklet)

Answer 5

How an item is scored. The scoring procedure for each item is clearly specified and easily understood.

Answer 6

Selected-Response Format Or Constructed-Response Format

Answer 7

A test structure where test-takers are to select a response from a set of alternative [given] responses. Include: Multiple-Choice Items, Rating Scales (Likert, Comparative, Guttman)

Answer 8

1. Stem 2. Correct Alternative 3. Distractors or Foils

Answer 9

Rating scales use a system of ordered numerical, verbal and/or pictorial descriptors judgements. Likert, Comparative, Guttman.

Answer 10

In the comparative scaling approach judgements are made using either sentences, printed cards, drawings, photographs or objects. Is an ordinal or rank order scale that can also be referred to as a non-metric scale. Respondents evaluate two or more objects at one time and objects are directly compared with one another as part of the measuring process. For example you could ask someone if they prefer listening to MP3s through a Zune or an iPod. You could take it a step further and add some other MP3 player brands to the comparison. - Do you prefer A or B or C

Answer 11

The intent of this survey is that the respondent will agree to a point and their score is measured to the point where they stop agreeing. For this reason questions are often formatted in dichotomous yes or no responses. The survey may start out with a question that is easy to agree with and then get increasingly sensitive to the point where the respondent starts to disagree. You may start out with a question that asks if you like music at which point you mark yes. Four questions later it may ask if you like music without a soul and which is produced by shady record labels only out to make money at which point you may say no. A series of increasingly extreme one-dimensional questions to see where the test-taker stops Endorsing them. E.g., 1. Do you want immigrants in your Country? 2. In your community? 3. Your neighbourhood? 4. Next door? 5. Live with an immigrant?

Answer 12

A test structure where test-takers are to create or construct a response, not merely select it.

Answer 13

Fill-in-the-Blank SAQ Essay Questions (LAQ) Projective Tests use CRF e.g., TATs, Rorscach.

Answer 14

Straight-Forward Numbers are assigned to diff responses and final score is by summing the numbers across all items (Summative Score)

Answer 15

Challenging because responses are diverse. Requires considerable judgement - Time Consuming - Exxy - Susceptible to low inter-rater reliability.

Answer 16

Once items have been written they are often subject to review from several perspectives prior to the formal test tryout. 1. Conformity with various item-writing rules 2. Content Relevance 3. Sensitivity Review 4. Informal Tryouts

Answer 17

An item that asks about two separate topics or issues within a single question, but only allows for one response.

Answer 18

1. Use Plausible Distractors 2. Use Question Format over incomplete sentences 3. Balance/randomise placement of correct answer across items. 4. Make options are mutually exclusive w only ONE correct answer. 5. Only use MCQ if there are no other appropriate formats.

Answer 19

Reading and Comprehension lvls of test-taker/s Impact or influence of Ethnic/Cultural factors Use the simplest possible language and give clear instructions.

Answer 20

Expert Panel Review Sensitivity Review Informal Tryout

Answer 21

A group of experts may be called on to review items for content relevance or correctness. - For example, if the construct is “anxiety” a group of clinical psychologists might be called in to review test items. - In recent years, more emphasis has also been placed on involving end-users in the review of items.

Answer 22

A review of all items for possible gender, racial, or ethnic bias. - Illustration: the developers of the Stanford Achievement Test employed an advisory panel of 12 minority group members. - The panel identified several potential forms of content bias that might find its way into achievement tests: 1. Status 2. Stereotyping 3. Familiarity 4. Offensive choice of words

Answer 23

An informal tryout is often used to ensure that test items are “working” as intended. - Test-takers completing informal tryouts asked to comment on the items and test directions. - Test-takers may be asked to “think aloud” while answering the items. Informal tryouts help the test developer to identify ambiguous wording, unexpected interpretations of an item, confusion about methods for responding and so on.

Answer 24

1. Administration of the item pool to a representative sample of test-takers. 2. Conducted under identical conditions under which the standardised test will be administered. - Identical Time limits, Instructions, environment etc. 3. Administered to a large sample size - Generally several hundred when using Classical Item Analysis Procedures

Week 4 Flashcards

Test Bias vs Test Fairness