Chapter 5: Measurement*

Question 1

How do you evaluate the accuracy of your measurement tool?

Answer 1

Reliability and construct validity

Question 2

Reliability

Answer 2

the degree to which a measure (of behavior/personality/intelligence/psychological construct) is consistent, providing a stable form of measurement

Question 3

Construct validity

Answer 3

the degree to which your measure actually measures what you want it to measure

Question 4

True score theory

Answer 4

If only measuring a variable, true score is a person’s real score. If conducting an experiment, true score is a person’s score affected by the condition they are in.

Question 5

Errors

Answer 5

sources of variability in your measure caused by things other than your IV (if there is an IV)

Question 6

Types of errors

Answer 6

random and systematic

Question 7

True score

Answer 7

an individual’s actual level of the variable being measured, not the score they get on the measure of that variable

Question 8

Measurement error

Answer 8

any contributor to the measure’s score that is not based on the actual level of the variable of interest (i.e. not the true score); responsible for the degree to which a measure’s score deviates from the true score

Question 9

Random error

Answer 9

has no pattern, unavoidable, unpredictable, and can’t be replicated by repeating the experiment e.g. misreading or misunderstanding questions, time of day

Question 10

Systematic error

Answer 10

has a pattern, produces consistent errors, and affects a participant’s scores in all conditions e.g. response biases, individual differences, incorrectly calibrated measuring instruments

Question 11

Why is low reliability a problem?

Answer 11

Difference between conditions can be misleadingly inflated or deflated due to unreliable measurements

Question 12

Types of reliability for a measure using the correlation coefficient

Answer 12

Test-retest reliability, internal consistency reliability, inter-rater reliability

Question 13

Test-retest reliability

Answer 13

degree of reliability assessed by administering the same measure on two different occasions, then calculating the correlation between the two different scores obtained

Question 14

Alternate forms reliability (solution for practice effects in test-retest)

Answer 14

two different forms of the same test are administered on two separate occasions

Question 15

Challenges of test-retest reliability

Answer 15

practice effects and demand characteristics

Question 16

Internal consistency reliability

Answer 16

form of reliability assessing the degree to which items in a scale are consistent in measuring the same construct or variable

Question 17

Cronbach’s alpha

Answer 17

indicator of internal consistency reliability assessed by examining the average correlation of each item in a measure (inter-item correlations) with every other question; higher alpha = more reliable (max is 1)

Question 18

Challenge of internal consistency reliability

Answer 18

makes the assumption that items actually measure the same construct

Question 19

Interrater reliability

Answer 19

an indicator of reliability that examines the degree to which two or more raters agree on an observation (score), having the same or similar judgements for a set of stimuli

Question 20

Intraclass correlation coefficient (ICC)

Answer 20

higher ICC = greater inter-rater reliability (max of 1)

Question 21

Challenge of inter-rater reliability

Answer 21

judges need to be trained and independent, and can be expensive

Question 22

Uses of inter-rater reliability

Answer 22

behavioral coding, personality measures, thematic/content coding

Question 23

Construct validity

Answer 23

the degree to which a measure accurately measures the theoretical construct it is designed to measure; the quality of operationalization

Question 24

Construct or conceptual variable

Answer 24

abstract variable that, in its natural form, can’t be quantified; needs an operational definition

Question 25

Face validity

Answer 25

the degree to which a measure appears to measure the intended variable; a subjective process

Question 26

Content validity

Answer 26

form of construct validity evaluated by comparing the content of the measure to the theoretical definition of the construct, ensuring that all aspects of the construct are measured and no other extraneous elements

Question 27

Predictive validity

Answer 27

aspect of construct validity that involves examining if a measure can predict a theoretically relevant FUTURE behavior or criterion

Question 28

Concurrent validity

Answer 28

type of construct validity that examines whether the measure can predict a criterion measured at the same time the measure is administered

Question 29

Convergent validity

Answer 29

aspect of construct validity assessed by examining the extent to which scores on the measure are related to other measures of the same or similar constructs

Question 30

Discriminant/divergent validity

Answer 30

aspect of construct validity in which scores on a measure are not related to scores on conceptually unrelated measures

Question 31

Indicators of construct validity of a measure

Answer 31

face validity, content validity, predictive validity, concurrent validity, convergent validity, discriminant validity

Question 32

When are reliability and validity necessary?

Answer 32

(1) reliability alone is necessary (but not sufficient) to establish validity (2) construct validity is not necessary to establish reliability (3) reliability and indicators of construct validity are both necessary to establish construct validity

Question 33

Reactivity

Answer 33

when reacting to the act of measuring or observing something changes a person’s behavior; minimized by using nonreactive or unobtrusive operationalizations

Question 34

Scales of measurement

Answer 34

nominal, ordinal, interval, ratio

Question 35

Nominal scale

Answer 35

scale of measurement with 2 or more categories that have no numerical properties; a.k.a. categorical variables

Question 36

Ordinal scale

Answer 36

scale of measurement in which the measurement categories form a rank order along a continuum but the distance is unknown

Question 37

Interval scale

Answer 37

a scale of measurement in which the intervals between numbers on the scale are all equal and zero is arbitrary (i.e. does not indicate a complete absence of quantity) e.g. temperature in fahrenheit and celsius

Question 38

Ratio scale

Answer 38

numeric scale of measurement with equal intervals and has a meaningful zero indicating total absence of the variable measured e.g. temperature in Kelvin