Week 2

Question 1

What is a raw score?

Answer 1

The score on an individual test -

not individually meaningful
needs to be compared to a standard (norm referencing)

Question 2

What is norm referencing?

Answer 2

Relating a raw score to a standard to give it meaning.
Eg: a parent giving meaning to their child’s spelling test score. If the child scores 55, that may not seem good, but if the majority of children scored less than 30 or more than 70.

Question 3

Criterion referencing

Answer 3

reference to a standard

a way of giving meaning to a test score by specifying the standard that needs to be reached in relation to a limited set of behaviours

Question 4

Non-linear transformation

Answer 4

a transformation that preserves the order but not the equivalence of distance of the original scores

Most common form of non-linear transformation is the percentile

Question 5

Derived score

Answer 5

A derived score is a numerical description of an individual’s performance in terms of norms

Question 6

Standard score (z-score)

Answer 6

the distance of a score in a normal distribution from the mean expressed as a ratio of the SD of the distribution

The z-score is a standard score, which transforms the score into SD units. If a z-score is -3.2, we know it sits within the third SD away from the mean.

Question 7

Percentile

Answer 7

an expression of the position (rank) of a score in a distribution of all scores by dividing the distribution into 100 equal parts; also known as ‘centile’

Calculation: number of values below the raw score, divided by the number of all raw scores, multiplied by 100.

Question 8

Z-score

Answer 8

a linear transformation of test scores that expresses the distance of each score from the mean of the distribution of scores in units of the SD of the distribution

To calculate, take the mean from the raw score and divide by SD.

Typically good to use because it retains all the features of the raw score

Question 9

Linear transformation

Answer 9

a transformation that original set of scores preserves the order and equivalence of distance of the original set of scores. Another way to think about this is adding a constant to all the raw scores. Any operation can be performed , so long as the straight line relationship is preserved i.e. +100 then /2 to all raw scores

Most common form of linear transformation is the z-score (SD)

Question 10

Norm referencing

Answer 10

compared to a representative sample

a way of giving meaning to a test score by relating it to the performance of an appropriate reference group for the person e.g. class results

Question 11

Deviation IQ

Answer 11

a method that allows an individual’s score to be compared with same-age peers; the score is reported as distance from the M in SD units.

Used in the WAIS-IV.

Question 12

Sten score

Answer 12

a point on a scale that has 5 units above and 5 units below the mean, which is set at 5.5 with a standard deviation of 2

Cattel used this in the 16PF

Question 13

T-score

Answer 13

a score standardised to a distribution with a mean of 50 and a standard deviation of 10

Used in MMPI where there are no wrong answers - answers either indicate the trait or they do not

Question 14

Derived score (or norms)

Answer 14

Allows us to ascertain an individual’s position relative to a standardisation (or normative) sample

Provide comparable measures that permit a comparison across different tests.

Question 15

Percentile scores

Answer 15

Percentage of people in the standardisation sample who fall below a particular raw score.

Advantages: easy to compute, readily understood, universally applicable

Disadvantages: inequality of units

Question 16

Standard scores

Answer 16

To compare individuals who have taken the same test

To compare scores across different tests with different distributions

Question 17

Normalised standard scores

Answer 17

Linear standard scores will be comparable only if they come from similar distributions

To ensure comparability > normalise

Question 18

Norms

Answer 18

Specific to the population from which they are derived

Often tests come from samples of WEIRD populations

The responsibility is to the psychologist to understand the sample and interpret test scores in light of any limitations.

Question 19

What are the three factors that determine objectivity of a test?

Answer 19

Administration (same materials, instructions, time limits)

Scoring (templates provided, computerised etc.)

Interpretation (directions about how to interpret scores)

A test will manual will typically specify the directions to ensure uniformity in administration.

Question 20

What are the two criteria of a good test?

Answer 20

Reliability (consistency and/or dependability) and

Validity (accuracy or that the test will measure what it says it will)

consistency on its own is not sufficient to ensure accuracy

Question 21

Correlation

Answer 21

The relationship between two variables.

co-efficient is the degree of linear relationship between -1 and +1

Question 22

Correlation Co-efficient

Answer 22

Degree of linear relationship between two variables

rxy = Σ(zx zy)/N

Question 23

Reliability co-efficient

Answer 23

The correlation between scores (rxx) on two administrations of a test

The extent to which scores on one test administration will generalise to other administrations of the same test.

The reliability co-efficient shows the proportion of raw score variance (total variance) explained by true score variance i.e. if rxx = .8, then 80% of the raw score variance is due to true score variance and 20% is due to error variance

Question 24

True Score Theory (Spearman, 1904)

Answer 24

If the same test is given to an individual an infinite number of times, the obtained scores will be normally distributed.

the mean of that distribution is the true score

the SD of that distribution is the standard error of measurement.

Question 25

True score reliability

Answer 25

Obtained (raw) score = true score + error variance

Question 26

What are the types of reliability?

Answer 26

Test-retest

Alternate forms

Internal consistency > split half > Kuder-Richardson and Co-efficient-Alpha

Scorer (or interviewer)

Question 27

Test-retest reliability

Answer 27

The same test is given to the same group of people on two different occasions

The rxx is the correlation between the scores on the two different occasions (coefficient of stability)

error variance (1-rxx) is attributable to changes in testing conditions and test-takers between the two occasions (sampling)

The longer the time between the two tests, the lower the reliability (allows more room for uncontrollable changes

Question 28

Alternate forms

Answer 28

Can be immediate or delayed.

Two versions of the test, constructed in an identical way with different content, completed by the same group of people.

rxx is the correlation between the scores on the two forms of the test (co-efficient of equivalence)

error variance (1-rxx) is attributable to differences in content and time (when delayed).

Question 29

Internal consistency and split half reliability

Answer 29

a single test, split into two halves is administered

rxx is the correlation between the person’s scores on each half of the test (co-efficient of consistency).

error variance (1 - rxx) is due to content.

Limitations -
Deciding the split
Speeded tests
Correlating the two halves (shortening the test)

Question 30

Spearman-Brown Formula

Answer 30

Looks at the effect of reliability of lengthening or shortening the test

Longer tests tend to be more reliable.

Estimates the length of a test needs to be to obtain desired level of reliability

Question 31

Kuder Richardson 20

Answer 31

Uses only dichotomously scored items i.e. true/false

Question 32

Coefficient Alpha

Answer 32

Used for multiple choice items and items rated on scale with more than two options

Question 33

Inter-rater reliability

Answer 33

Two scorers (raters) provide scores for an individual on a test

Used when the scores on the test require subjectivity on the part of the scorer