Chapter 6: Properties of Good Measures

Question 1

Properties of Good Measures (2)

Answer 1

(1) Reliability
(2) Validity

Question 2

Reliability is a necessary condition for …

Answer 2

validity
-> At a certain point, your score is mostly made up of error, so it can’t be validly measuring what you want it to

Question 3

Less reliable = more …

Answer 3

error
-> The less reliable, the more error there is (+ measurement error): Not measuring real variability between people

Question 4

Diff types of reliability (5)

Answer 4

(1) Internal consistency
(2) Test-retest reliability
(3) Inter-rater reliability
(4) Parallel-form reliability
(5) Split-half reliability

Question 5

Internal consistency

Answer 5

High correlation between items. Cronbach’s alpha.

Question 6

Test-retest reliability

Answer 6

If ADHD symptoms today, scores same in same weeks (highly correlated with each others)
-> Be careful! Some constructs should change over time, even over very short intervals
-> E.g. ADHD symptoms at 8 vs early 20s: wouldn’t expect to be correlated. Some symptoms go down, some hyperactivity might just be ‘be a young kid’. Test-retest wouldn’t make sense.

Question 7

Inter-rater reliability

Answer 7

Agreement between two people judging whether something is present or occurring

Question 8

Parallel-form reliability

Answer 8

The reliability coefficient obtained by two comparable sets of measures
-> See this a lot more in educational testing settings/IQ testing/academic testing

Question 9

Split-half reliability

Answer 9

Reflects the correlations between two halves of an instrument
-> E.g. Super long measure (100 items)
-> Are the scores that pple get on the first 50 items associated with scores that pple get on the second 50 items.

Question 10

We are creating new ADHD rating scales. We create 2 10-item scales that are meant to be similar in the symptoms they measure. In a very large sample of adolescents, scores between these 2 measures are correlated @ .92. What type of reliability is measured?

Answer 10

Parallel form

Question 11

Types of Validity (3)

Answer 11

(1) Convergent validity
(2) Discriminant validity
(3) Face validity

Question 12

Convergent validity

Answer 12

Are scores on the measure related to other measures or indicators of the SAME construct

Question 13

Discriminant validity

Answer 13

Are scores on the measure DIFFERENT from scores of other constructs
-> Our measure should be uncorrelated with unrelated/random constructs

Question 14

Face validity

Answer 14

Does this APPEAR to measure what it is supposed to measure
-> Maybe low face validity if i don’t want the person to know what i’m testing

Question 15

Measurement Invariance

Answer 15

‘Fairness’ of a measure
-> Has to do with all of the argument around potential cultural bias in diff tasks (e.g. GRE, LSATS, SAT…)
-> People have a lot of concern over whether scales are INVARIANT (= function similarly across diff groups)
-> If there’s a lot of measurement bias, certain types of pple will perform systematically better than others (even if not better on the trait)