Lecture 3: Essentials of Test score interpretation

Question 1

Frames of References

Answer 1

specific standard used to judge an individual test score

Question 2

Raw scores

Answer 2

• summarizes a person’s test performance (dichotomous items, polytomous items)
• conveys no meaning beyond expressing how many items were solved

Question 3

Two sources of information to derive from frames of reference for test scores

Answer 3

1. Norms

2. Performance criteria

Question 4

Norms

Answer 4

• norm-referenced test interpretation uses standards based on the performance of speicific groups
• test performance ot typical behavior of one or more reference groups
• Typical question: performance of test taker compared to that of others?

Question 5

Performance Criteria

Answer 5

• test scores are interpreted against the demonstration of “performance”
• assess whether and to what extent the desired levels of mastery or performance criteria have been met (educational and clinal setting)

Question 6

Norm- Referenced Test interpretation

developmental norms

Answer 6

• ordinal scales based on behavioral sequences
• still in use today with young children
• age vs. theory
• mental scores
• grade equivalent scores

Question 7

Within-group norms

Answer 7

provide a way of evaluating a person’s performance against the performance of a reference group (place test takers performance within a normal distribution)

Question 8

The normative sample

Answer 8

• should be representative of the intended population
• needs to be sufficient large
• important: recency (re-norming)

Question 9

Standardization sample

Answer 9

group of individuals on whom the test is originally standardized in terms of administration and scoring procedures, and developing test’s norms

Question 10

The normative sample

Answer 10

often used as synonymous with the standardization sample, but can refer to any group from which norms are gathered

Question 11

Reference group

Answer 11

any group of people against which test scores are compared

Question 12

Information needed to evaluate the application of a normative sample

Answer 12

• how large is the normative sample?
• when and where was the sample gathered?
• how were individuals identified and selected in this sample?
• who tested the sample? And how did the examniner’s qulaify to test the sample?
• what was the composition of this normative sample? (age, sex, ethinicty, etc.)

Question 13

Variants (of norms)

Answer 13

• subgroup norms
• local norms
• convenience norms

Question 14

Subgroup norms

Answer 14

• when large samples are gathered, norms can be seperated into subgroup norms
• formed in terms of any variable that may have a significant impact on test scores or yield comparisons of interest

Question 15

Local norms

Answer 15

• based on respondents from a specific geographical or institutional setting

Question 16

Convenience norms

Answer 16

test developers use norms based on a group of people who simply happen to be availabe at the time (for financial constraints?)

Question 17

Scores used for expressing within-group norms

Answer 17

1. Percentiles

2. Standard scores

Question 18

Percentiles

Answer 18

indicates the percentage of person’s in the reference group who scored at or below a given raw score
- 50th percentile (the median)

Question 19

Percentiles vs. Percentages

Answer 19

• percentiles reflect the rank or position of an individuals’s performance on a test in comparison to a reference group
• percentage scores reflect the number of correct responses that an individual obtain out of a total number of correct responses

Question 20

Units are unequal across the range?

Answer 20

• differences are exaggerated in the middle section of the raw score distribution and compressed at the extremes (percentag of poeple who score near the middle is greater)

Question 21

Test ceiling

Answer 21

test taker reaches the highest score attainable on an already standardized test, maximum difficulty level is insufficient

Question 22

Test floor

Answer 22

person fails all the items presented in a test or scores lower than any of the poeple in the normative sample (insufficient test floor)

Question 23

Standard scores

Answer 23

• linear trasnformation of raw scores

- then transformation of z scores into other derived standard scores

Question 24

Linear transformation of raw scores

Answer 24

chnages the units in which scores are expressed while leaving the interrelationships among them unaltered
- normally distributed scores of tests with different means (etc) can be fully compared (as long as the same reference group is used)

Question 25

trasnformation into z-scores

Answer 25

X - reference group mean/ standard deviation of the reference group - indicate the direction in which the orginal score deviated from the mean of a group

Question 26

Pearsons prodcut-moment correlation

Answer 26

correlation between variable x and y | - mean of the cross-products of z-scores of the correlated variables

Question 27

Deriving standard scores

Answer 27

New standard score = z score x New SD + New Mean | - z scores involve negative values, therefore z scores undergo additional linear transformations

Question 28

non-linear transformations

Answer 28

convert a raw score int a distribution that has a different shape than the original - normalized standard scores - stanines

Question 29

Normalized test scores

Answer 29

usen when a score distribution approximates but does not quite match the normal distribution raw score -> cumulative percent -> cumulative proportion -> normalized z score

Question 30

Raw score -> cumulative percent (CP)-> cumulative proportion (cP)-> normalized z score

Answer 30

1. raw score and CP are located in the score distribution 2. cp is CP/100 3. normalized z score is obtained from the tables of areas under the normal curve - > can then be trasnformed into various other stnadrd score systems

Question 31

Stanine scores

Answer 31

- > transforms all scores in a distribution into single-digit numbers from 1-9 - > makes use of cumulative frequency and cumulative percentage distribution

Question 32

Intertest comparisons | - comparability is also compromised by...

Answer 32

1. difference in scale units 2. contruct meaning 3. additional factors affecting the test results of individuals

Question 33

Test scores cannot be meaninfully compared, when...

Answer 33

- the test or test versions are different - the reference groups are different - the score scales differ

Question 34

Test equating

Answer 34

designed to achieve (some) comparability of scores across tests (place test scores in a common frame of reference)

Question 35

Different forms of test equating

Answer 35

1. Alternate forms (Parallel tests) 2. Anchor tests 3. Fixed reference groups 4. Simultaneous norming

Question 36

Alternate forms

Answer 36

cosists of two (more) versions of a test meant to be used interchangeably, intended for the same purpose, and administered in identical fashion

Question 37

Parallel tests

Answer 37

forms are equated not only in content coverage and procedures. But also, in some of their statistical charcateristics

Question 38

Anchor tests

Answer 38

consists of common set of items adminstered to different groups of examinees in the context of two or more tests (different skill areas - uniform test)

Question 39

Fixed reference group

Answer 39

achieve some comparability and continuity of test scores over time

Question 40

Simultaneous norming

Answer 40

two or more tests on the same standardization sample are normed (compare the performance of individuals on more than one test, using the same standard)

Question 41

Item Response Theory (IRT)

Answer 41

model seeks to estimate the level of unobservable abilities, traits or psychological constructs (latent variables), produce item parameter estimates that are invariant across populations

Question 42

computerized adaptive testing

Answer 42

individuals ability level can be estimated based on their response to test items during testing process

Question 43

Small test revisions

Answer 43

small changes in the wording of items | - comparability of the old and revised versions can be established easily

Question 44

Major test revisions

Answer 44

reorganization in content, scoring and adminstrative procedures - revision of norm-referenced tests, require restandardization of the test with a new normative sample

Question 45

Need for updated norms

Answer 45

- score norms tend to drift into a direction due to changes in the population at different time periods - Lynn-Flynn effect: long-term upward trend in the level of performance required to obtain any given IQ score (increase pace of restandardization of the Wechsler's scale)

Question 46

Criterion-referenced Test interpretation

Answer 46

- specific test purpose - needs a determined criterion or standard against which performance/response behavior is to be measured - use of cutoff scores

Question 47

Specific test purposes (two types)

Answer 47

- to ascertain whether a person has reached a certain level of competence - to assess to what degree a person meets certain requirements

Question 48

Validity of criteria? (common characteristics)

Answer 48

1. are meant to assess the extent to which test takers are proficient in certain skills or knowledge domains 2. are scored in such way that one perons's performance does not affect the relative standing of others

Question 49

Norm-referenced test interpretation

Answer 49

seek to locate performance with regard to the contruct the tests asses (frame reference - other poeple)

Question 50

criterion-referenced test interpretaion

Answer 50

seek to evaluate performance in relation to standard related to the construct itself - frame reference: knowledge of a domain or level of competence displayed