Chapter 5: Identifying Good Measurement

Question 1

Operationalization

Answer 1

Process of turning a construct of interest into a measured or manipulated variable
Start with defining construct
Create operational definition
E.g. happiness operationalized through self reports

Question 2

Conceptual Definition

Answer 2

Researchers definition of the variable in question at a theoretical level

Question 3

Self Report Measures

Answer 3

Operationalizes variables by recording people’s answers to questions about themselves in questionnaires or interviews
In children self reports may be replaced by parent or teacher reports
Seeing how people think about themselves (internal processes not observable)
More cost effective
Easiest way to find out the answer (e.g. age or gender)

Question 4

Observational Measures

Answer 4

Sometimes called behavioural measure
Operationalizes variables by recording observable behaviours or physical traces of behaviours
Participant less likely to be influenced by researcher
Better for morality related questions (people more inclined to give incorrect answers)
Bypasses people’s biases about themselves (self serving bias, memory bias, etc)
Studying younger children (can’t report as well)\
E.g. observing how many times a person smiles, intelligence tests
E.g. bobo doll studies with adults and children
E.g. observing rigged monopoly game

Question 5

Physiological Measures

Answer 5

Operationalizes variables by recording biological data
Usually require use of equipment to amplify, record, and analyze biological data
E.g. eye tracking, brain activity, hormone levels, heart rate

Question 6

Categorical/Nominal Variables

Answer 6

Categories
Can assign numbers to categories (but numbers are meaningless)
E.g. sex, species, nationality, university
E.g. male is 1, female is 2

Question 7

Quantitative Variable

Answer 7

Coded with meaningful numbers
E.g. height, weight, etc

Question 8

Three types of quantitative variables

Answer 8

Ordinal
Interval
Ratio

Question 9

Ordinal Scale

Answer 9

Applies when numerals of a quantitative variable represent a ranked order
Sets of rankings
Intervals are not equal
E.g. top 10 best selling books, fast to slow
E.g. order exam papers were turned in

Question 10

Interval Scale

Answer 10

Applies to numerals of a quantitative variable that meet two conditions; numeral represent equal intervals between levels and there is no true zero
Intervals are equal between points
No true zero
E.g. IQ test, shoe size, degree of agreement, temperature (0 does not mean absence of temperature)
Can’t say things are 2x something else

Question 11

Ratio Scale

Answer 11

When numerals of quantitative variable have equal intervals and when the value of 0 truly means none of the variable being measured
E.g. knowledge test, number of eye blinks
Can apply math to these variables (e.g. one person blinked 2x as much as another)

Question 12

Three types of reliability

Answer 12

Test retest
Interater
Internal

Question 13

Test Retest Reliability

Answer 13

Participant will get pretty much same score each time they are measured with it
Assess how good your measure is of a construct
E.g. if people took IQ test today results would be similar in one month
If increases occur, they should all occur about the same amount (e.g. not just one person changes)
Can apply whether operationalization is self report, observational, or physiological
Most relevant when measuring constructs that are theoretically stable (e.g. happiness tends to fluctuate more than intelligence)

Question 14

Interrater Reliability

Answer 14

Consistent scores are obtained no matter who measures the variable
Two or more independent observers come up with consistent finding
Most relevant for observational measures
E.g. recording a child’s smiles at a playground, both researchers should have consistent number

Question 15

Internal Reliability/Consistency

Answer 15

Participant gives consistent pattern of answers no matter how question is phrased
Consistency of people’s responses across multi item measure
Applies to measures that combine multiple items
If the items are measuring the same construct they should correlate
E.g. subjective well being scale
Each item on scale is worded differently but intends to measure the same construct
People who agree with first item should also agree with subsequent items (internal reliability)

Question 16

Cronbach’s a

Answer 16

Reflects average of the inter item correlation
Low cronbach’s alpha suggests that measure is not consistent (at least one item does not belong with others)

Question 17

Correlation Coefficient (r)

Answer 17

Single number to indicate how close the points on a scatter plot are to a line drawn through them
Indicates strength and direction of relationship (-1 to 1)

Question 18

Validity

Answer 18

Does the measure actually measure what it intends to
Especially important when a construct is directly observable

Question 19

Face Validity

Answer 19

Subjectively considered to be a plausible operationalization of the conceptual variable
Align well with conceptual definition of a construct
Can check face validity by consulting experts

Question 20

Content Validity

Answer 20

Measure must capture all parts of the defined construct
Requires knowledge of conceptual definition
Measure is contaminated if it measures something else

Question 21

Criterion Validity

Answer 21

Evaluates whether the measure under consideration is associated with a concrete behavioural outcome that it should be associated with, according to the conceptual definition
Objective measure
Assessed via correlation
E.g. people who score high on a measure associated with work level become better employees in following months

Question 22

Known Groups Paradigm

Answer 22

Researchers see whether scores on the measure can discriminate among two or more groups whose behaviour is already confirmed
E.g. measuring salivary cortisol in people in audience vs before giving speech (speeches are known to be stressful so cortisol should be higher in speech condition)

Question 23

Convergent Validity

Answer 23

Measures of one concept should correlate across other known measures
E.g. on self report measure of depression should correlate with results from other known measures
Measures of similar constructs should correlate
E.g. depression self report correlated with scores of psychological well being

Question 24

Discriminant Validity

Answer 24

Concepts that are very different should not be correlated
Measure should not correlate with things measuring opposite concepts
E.g. no strong correlation between depression self report and physical well being report
E.g. a scale supposed to diagnose learning disabilities should not be correlated with IQ because learning disabilities are not related to general intelligence
Researchers tend to measure discriminant validity for similar concepts
E.g. does depression self report measure depression or perceived health problems

Question 25

Relationship Between Reliability and Validity

Answer 25

Measure can be reliable but not valid
Cannot be more valid than it is reliable