Psychological Measurement Exam 4

Question 1

a statistic indicating how many test takers responded correctly to an item.

Answer 1

item-difficulty index

Question 2

If 80% got item correct then item-difficulty index is ____.

Answer 2

.8

Question 3

The larger item-difficulty index, the _____the item.

Answer 3

easier

Question 4

You want an item difficulty to be halfway between that and 1, ex. for a 5 option multiple choice item, the probability of guessing correctly is .20, so the optimal item difficulty is therefore ____.

Answer 4

.60
.20+1.00=1.20
1.20/2=.60

Question 5

provides an indication of the internal consistency of a test

Answer 5

item-reliability index

Question 6

the higher the _____, the greater the tests internal consistency.

Answer 6

item-reliability index

Question 7

What are the 3 test construction approaches?

Answer 7

1. Rational approach
2. Empirical approach
3. Rational with empirical refinement approach

Question 8

Making up statements about traits of a personality to tap every aspect

Ex. I am depressed once a month
I am depressed a couple times a month
I am depressed once a week
I am depressed a couple times a week
I am depressed everyday

This is easy to construct, easy to fake.

Answer 8

Rational approach to test construction

Question 9

Uses 2 criterion groups, 1 normal and 1 exhibits trait that you want to tap into. Come up with an item pool of questions, give to both groups. Determine which questions are answered statistically significantly different.

Hard to fake because the questions are random and they don’t know what they’re being tested for.

Limitation - p

Answer 9

Empirical approach within test construction

Question 10

Use rationality to come up with questions then run tests with 2 criterion groups and distinguish.

Answer 10

Rational with empirical refinement approach within test construction

Question 11

What are the 5 steps in test development?

Answer 11

1. Test conceptualization
2. Test construction
3. Test tryout
4. Item analysis
5. Test revision

Question 12

Coming up with an idea that a test ought to be designed to measure [fill in the blank ] in [such and such ] way.

Answer 12

Test conceptualization

Question 13

Preliminary research surrounding the creation of a prototype of the test.

Answer 13

Pilot work

Question 14

Process of setting rules for assigning numbers in measurement.

Answer 14

Scaling

Question 15

Grouping of words, statements, or symbols on which judgements of the strength of a particular trait, attitude, or emotion are indicated by the test taker.

Answer 15

Rating scale

Question 16

Summative rating scale.
Presents test taker with five alternative responses.
Ex. Never, rarely, sometimes, usually, always
(each assigned a value [agreement])

Answer 16

Likert scale

Question 17

Test takers are presented with pairs of stimuli (2 photos, 2 objects, 2 statements) which they are asked to compare. Must select one stimuli according to some rule (that they agree more with, etc.)

Answer 17

Method of paired comparisons

Question 18

Printed cards, drawings, photos, objects, etc. are presented for evaluation.

Answer 18

Sorting tasks

Question 19

Compare stimulus with others (ex. Rank)

Answer 19

Comparative scaling

Question 20

Stimuli are placed in 1 of 2 or more categories.

Answer 20

Categorical scaling

Question 21

Yields ordinal level measures.
Ex.
1. All people should have the right to decide whether they wish to end their lives.
2. People who are terminally ill and in pain should have the option to have a doctor assist them in ending their lives.
3. People should have the option to sign away the use of artificial life support equipment before they become seriously ill.
4. People have the right to a comfortable life.
All who agree with 1, also agree with 2, 3, 4…etc.

Answer 21

Guttman scale

Question 22

Reservoir or well from which items will or will not be drawn from for the final version of the test.

Answer 22

Item pool

Question 23

Form, plan, structure, arrangement, and layout of individual test items.

Answer 23

Item format

Question 24

Requires test takers to select a response from a set of alternative responses.

Answer 24

Selected-response format

Question 25

Requires test takers to supply or to create the correct answer, not select it.

Answer 25

Constructed response format

Question 26

test taker is presented with two columns: premises on the left and responses on the right, determine which response is best associated with which premise.

Answer 26

Matching Item

Question 27

Multiple choice that contains only 2 possible responses. Ex: True/False

Answer 27

Binary Choice Item

Question 28

Requires the examinee to provide a word or phrase that completes a sentence.

Answer 28

Completion Item

Question 29

word, term, sentence, or paragraph

Answer 29

Short Answer Item

Question 30

test item that requires the test taker to respond to a question by writing a composition, typically one that demonstrates recall of facts, understanding, analysis, and/or interpretation.

Answer 30

Essay Item

Question 31

large and easily accessible collection of test questions often used by teachers.

Answer 31

Item Bank

Question 32

interactive, computer-administered test taking program process wherein items presented to the test taker are based in part on the test takers performance on previous items.

Answer 32

Computerized Adaptive Testing (CAT)

Question 33

diminished utility of an assessment tool for distinguishing test takers at the low end of the ability, trait, or other attribute being measured.

Answer 33

Floor Effect

Question 34

diminished utility of an assessment tool for distinguished test takers at the high end of the ability, trait, or other attribute being measured.

Answer 34

Ceiling Effect

Question 35

ability of the computer to tailor the content and order of presentation of test items on the basis of responses to previous items.

Answer 35

Item Branching

Question 36

Most commonly cumulative.
-Cumulative-Higher score=higher test taker is on the ability, trait, or other characteristic that the test purports to measure.

Answer 36

Scoring Items

Question 37

test taker responses earn credit toward placement in a particular class or category with other test takers whose pattern of responses is presumably similar in some way.

Answer 37

Class/Category Scoring

Question 38

comparing a test takers score on one scale within a test to another scale within that same test.

Answer 38

Ipsative Scoring

Question 39

tried on people who are similar in critical respects to the people for whom the test was designed.

Answer 39

Test Tryout

Question 40

different types of statistical scrutiny that the test data can potentially undergo at this point.

Answer 40

Item Analysis

Question 41

statistic indicating how many test takers responded correctly to an item.

Example:
If 80% got item correct then item-difficulty index is .8
Larger item-difficulty index=easier the item.
For a true/false item, the optimal item difficulty is .75
.50+1.00=1.5
1.5/2=.75

For 5 option multiple choice item, optimal is .6
.20+1.00=1.2
1.2/2=.6

Answer 41

Item-Difficulty Index

Question 42

provides an indication of the internal consistency of a test; the higher the index-the greater the tests internal consistency.

Item-Reliability= Sx (rxt) Sx being the item score standard deviation
rxt being the correlation between item score and the total test score

Answer 42

Item-Reliability Index

Question 43

statistic designed to provide an indication of the degree to which a test is measuring what it purports to measure.
Higher validity index=greater the test’s criterion-related validity.
Item-Validity= Sx (rxc) Sx being the item score standard deviation
rxc being the correlation between item score and the criterion score

Answer 43

Item-Validity Index

Question 44

a measure of discrimination, symbolized by a lowercase italic (d). Compares performance on a particular item with performance in the upper and lower regions of a distribution of continuous test scores.
Measure of the difference between the proportion of high scorers answering an item correctly and the proportion of low scorers answering the item correctly.
Ranges from -1 to +1
Higher d, the more it discriminates.

Answer 44

Item-Discrimination Index

Question 45

Optimal boundaries of lower and upper _________ are top 27% and bottom 27% of the distribution of scores.-If it’s normal.

Answer 45

Item-Discrimination Index

Question 46

Formula:

Passing scores in top - passing scores in bottom
N

Answer 46

Item-Discrimination Index

Question 47

a graphic representation of item difficulty and discrimination. Extent to which an item discriminates high from low scoring examinees is apparent from the slope of the curve.
The steeper the slope=the greater the item discrimination.

Answer 47

Item Characteristic Curves

Question 48

throw out items that are too easy or too hard and don’t discriminate. Keep those that do discriminate low from high scores and demonstrate reliability. If more questions are needed, go through items that were tossed out and pick out those that are repairable, revise and rewrite them. Redo process, revise standard conditions as well

Answer 48

Test Revision

Question 49

the revalidation of a test on a sample of test takers other than those on whom test performance was originally found to be a valid predictor of some criterion.

Answer 49

Cross validation

Question 50

the decrease in item validity that inevitably occurs after cross validation of findings (most likely due to chance).

Answer 50

Validity Shrinkage

Question 51

as soon as test is taken out of original general context, validity goes down.

Answer 51

Generaliability