Class 6 Notes - Identifying Good Measurement

Question 1

in psych…

Answer 1

often interested in latent concepts, that cannot be directly observed [happiness, stress]

Question 2

ways to measure variables

Answer 2

• self report measure
• observational measure
• physiological measure

Question 3

self-report measure

Answer 3

• Self-efficacy: general belief that I can confident in a particular domain - subjective/personal interpretation
• Recording peoples answers about themselves
• Questionnaire/interview: subjective thoughts/feelings/desires
• Limitations
   People might not want to be honest about a certain question (racial bias/prejudice — uncomfortable with saying true thoughts on that topic)… social desirability might influence our answer [measuring the way in which one wishes to be perceived]
   How much fast-food do you eat a month?
   Often are not consciously aware of many of our true feelings/beliefs
   Unconscious
   We often don’t know, and underestimate, the things that motivate us, why we like things, or why we ascribe to certain beliefs
   Self-report on a process that they are unaware — cannot possibly be described

Question 4

observational/behavioural measures

Answer 4

~ Pain: societal masculinity means pain tolerance will be inflated when a man is among a group of women
 Self-report might not be the most accurate
 Micro-facial expression: eyebrows down-inside

~ Recording observable behaviours; pressing keys on a keyboard; rate what emotion do u think this person is feeling

~ Can train observers to just look at participants and judge

~Important when ppl have desire to self-represent — gets around the limitations of self-report method

~Limitations
 If you have to train observers, to recognized a certain behaviour: requires money, resources, time
 Best practice to have multiple observers observing at the same time
 Trained observers still disagree — further training required to establish a communal conception

Question 5

physiological measures

Answer 5

~ Craving: measure of brain waves to quantitatively express how much craving is this person experiencing
 Self-report difficult to determine: do they have perfect conscious interpretation of their cravings? — people have different scale
 More accurately quantify — linked to standardized processes that we assume to be the same across different people

~Recording biological data

~How much of a molecule is in the blood stream; how much sweat; how much are their pupils dilating — that we know are associated with certain traits

~Limitations
 Do not give any insight into subjective experiences
 Expensive: fMRI (300-400$); magnetic tools

Question 6

quantitative variables are …

Answer 6

coded with MEANINGFUL NUMBERS

Question 7

categorical (nominal) variables have

Answer 7

discrete categories as levels

Question 8

quantitative scales

Answer 8

• ordinal : ordered
• interval : ordered, equidistant, no meaningful zero
• ratio: ordered, equidistant, meaningful zero

Question 9

examples of interval and ratio scales

Answer 9

Interval: temperature, GPA, IQ — no meaningful zero

Ratio: meaningful zero — reaction time, Kelvin scale

Question 10

reliability of measurement

Answer 10

test retest

interrater

internal

Question 11

 Test-retest

Answer 11

~ Construct to be stable over time

   * Measure today, and measure tomorrow: good measure should be the same 
   * Ex. satisfaction with life

Question 12

 Interrater

Answer 12

~ Trained 2 observers to quantify some behaviour that they see in the outside world

   * How much do each of these observers agree: did they say the same thing
   * If so, measures are reliable/consistent
   * If not, measure are not very reliable

Question 13

 Internal

Answer 13

~ Specifically when asking participants questions; each of those questions being a test of the concept

* Each of those questions seems to be assessing something similar to the other questions
* Correlation between all of the different items; do they point to the same measure of the concept?

Question 14

Evidence for reliability is a special example of an association claim

Answer 14

• These two things are linked
o Test with retest
o Quantity of 1st rater with quantity of 2nd rater
o Internal reliability: is item 1 associated with item 2

Question 15

Scatterplots + correlations can be used to evaluate reliability

Answer 15

• Any deviants from the line indicative of disagreement between results

Question 16

Minimum levels of reliability

Answer 16

Test-retest
 r = 0.5 +
 medium-strong positive relation
 assuming that the trait that were measuring stay pretty stable

Interrater
 r = 0.7 +
o minimum level of reliability
o time is not changing; more than one rater/observer
o 1-10 how happy is each child
 0.7 +
o quantify the rate of agreement between raters
o for categorical judgements: which these 4 emotions do these school-children feel? [nominal variable] happiness, sadness, shame, anger

Internal
 Cronbach’s alpha considers the correlation between all items on a scale: individual relationships between each item on the scale — average / mean
 0.7 +

Question 17

construct validity

Answer 17

subjective assessments

empirical assessments

Question 18

subjective assessments

Answer 18

face validity

content validity

Question 19

empirical validity

Answer 19

criterion validity

convergent validity

discriminant validity

Question 20

face validity

Answer 20

 Does the measure “look good”?

~Rorschach image: what does the person see

* Psychodynamic measurement: supposed to be a projection of what we are experiencing — fears, hopes
* Fascinating: doesn’t do as bad of a job as you might think
* Some evidence of validity
* Much better measures of anxieties/dreams/wishes

Question 21

content validity

Answer 21

 The extent to which a measure represents all facets of a given construct
~Difference ways of conceptualizing personality
~Personality
* Traits; long lasting/durable within an individual

Question 22

criterion predictive validity

Answer 22

does the measure related to concrete outcomes?

 If the Rorschach human movement variable measures empathy, then it should correlate with mirror neuron activity

~ The score from the Rorschach measurement does have some criterion validity with mirror neuron activity

Question 23

convergent validity

Answer 23

 Two measures, cause they measure the same construct — should correlate strongly

 Empathy should correlate with other measures of the same construct — already validated in the scientific literature

 - MMPI
     * Bunch of true/false statements: aspects of a persons personality
     * Many sub-scales
     * The scores received generally correlated with the scores seen on Rorschach — convergent validity (measuring personality)

Question 24

discriminant validity

Answer 24

 Two measures, hypothesis that our construct should not be related to other constructs — isn’t being contaminated by measuring other constructs
~ Intelligence
* WAIS-R Intelligence Scale
* Sex, eye colour, height, ice cream cravings, etc.
* If there is not a correlation with unrelated constructs, not related to intelligence