Lecture 27: Critical thinking about psychological constructs

Question 1

neuroimaging wordt vaak gezien als een meer scientific manier, terwijl dit eigenlijk alleen maar…

Answer 1

meer technologically advanced is

Question 2

wat is er met 10% en het brein

Answer 2

we only use 10% of our mental capacity (not 10% of our brain!)

Question 3

interpretation bias=

Answer 3

a bias towards interpretations that favour a researchers theory, not that their assumptions may be wrong

dus dit is die bias:
significant results: theory is goed
non significant results: measurement was niet goed, theorie wel

Question 4

quine-duhem problem

Answer 4

P1 Substantive assumptions & assumptions
about measurement & assumptions about
research etc.
P2 If P1, then these predicted results
P3 Not these predicted results
C ?

Then which part of P1 did we falsify, exactly?

dus we weten basically niet welk deel van P1 we hebben gefalsificeerd: gaat dit over de meting of de theorie?

Question 4

2 types of beliefs

Answer 4

1. theory relevant beliefs: Theoretical mechanisms that produce observable behavior
2. method relevant beliefs: Procedures with which we produce and analyse data

Question 5

dus hoe leg je de interpretation bias uit aan de hand van deze 2 types of beliefs

Answer 5

is the tendency to interpret the failure to confirm predicted outcomes in terms of method relevant beliefs, but confirmed predictions in terms of theory relevant beliefs

Question 6

wat zeiden LeBel en Peters over deze twee types beliefs

Answer 6

• Method-relevant beliefs are too peripheral
• Theory-relevant beliefs are too central

Question 7

waar zorgt de interpretatie bias voor

Answer 7

dat de theorie unfalsifiable wordt

Question 8

hoe zijn deze twee beliefs in physics

Answer 8

als je geen resultaten krijgt bij een thermometer, moet je heel veel uitleggen en veel bewijs leveren om deze methode te scrunizen.

dus hier methode = centraal, theory = peripheral

Question 9

bij psychologie is het dus veel makkelijker om kritiek te leveren op de methode, dan op de theorie. zo maak je het dus moeilijker om een theorie te falsificeren, omdat je altijd wel kan zeggen dat er iets fout was gegaan in de methode (en er dus helemaal niks mis was met de theorie!!)

Answer 9

oke

Question 10

wat betekent het als een type belief centraal staat

Answer 10

= a lot of other stuff hinges on it: we have a lot of explaining to do if we scrutinize this!

Question 11

dus physics vs psychologie: welke belief meer centraal?

Answer 11

physics= methode centraal
psychologie = theory centraal

Question 12

voor welke type of belief heb je minder evidence nodig

Answer 12

peripheral, want die kan je makkelijker criticeren.

Question 13

wat was er met de goldilocks coffee

Answer 13

ze zeiden dat de resultaten niet gerepliceerd konden worden omdat de andere experimenters vast mega hete koffie gebruikten ipv lauwe koffie zoals in originele experiment -> scrutinize the methods, instead of thinking the theory may not be right.

Question 14

wat zou dus ideaal zijn bij deze two types of belief in psychology

Answer 14

more central method relevant beliefs! -> more conservative about accepting our theories, meer bewijs leveren voor een theorie

Question 15

theory relevant beliefs are too …. in psychology

Answer 15

central

Question 16

hoezo zijn theory relevant beliefs te centraal in psy?

Answer 16

omdat er een hele grote overlap is tussen wat een specifieke theory zou voorspellen, en wat hele general assumptions zouden zeggen over dat fenomeen -> hard to falsify!

Question 17

waarom is dat met general assumptions too hard to falsify

Answer 17

because we would need to falsify the general assumption as well!

Question 18

in an ideal world…

Answer 18

research procedures and measuring instruments are unambiguously defined and validated

Question 19

dus meer centrale method-relevant beliefs leiden tot…

Answer 19

meer conservativiteit in de interpretatie van een studie

Question 20

The value of methodological rigor is precisely that, by leveraging conservatism, it becomes
more difficult to blame negative results on flawed methodology; this constrains the field of
alternative explanations and so makes empirical tests more diagnostic.

Answer 20

oke

Question 21

L&P are convinced that theory-relevant beliefs are often too central to psychology.
* By this they mean that empirical predictions are often indistinguishable from very general assumptions
about human behaviour.
* The more general (central) these assumptions are, the less stringent the theory can be tested.

Answer 21

oke

Question 22

degree of corroboration depends on….

Answer 22

how strict your test is/to what extent you expose the theory to falsification.

Question 23

But the stringency of our tests ….

Answer 23

is debatable, due to the aforementioned reliance on conceptual replication as well as the reliance on the Null ritual

Question 24

3 common issues with modal research practices in psychology further exacerbate this situation:

Answer 24

1. Overemphasis on conceptual replication
2. Problematic implementation of NHST
3. Insufficient attention to verifying the integrity of measurement instruments and experimental procedures

Question 25

wat was feeling the future een voorbeeld van

Answer 25

overreliance on conceptual replication: ze hadden een hoop pilot testing etc gedaan, en toen kwam er een ding uit wat significant was.

Question 26

An over-reliance on conceptual replication: wat bedoelen ze hiermee en voorbeeld

Answer 26

Failure to produce significant results constitute failed pilot studies that end op in the file drawer.

E.g. failure to show predicted increases in aggression in a certain context on a hot-sauce task do not cause any concern in terms of method relevant beliefs. The study can be considered a pilot, and in a subsequent study a different DV or procedure or both are used.

Question 27

voorbeeld van problematic implementation of nhst

Answer 27

For example, Bem (2011) seems to be testing a specific theory about PSI/pre-cognition: primes from the future have an influence on the present.
* But if we look at the statistical hypothesis, the theory/prediction is actually a lot more general: any deviation from 50% would do.
* Straw target: Setting up an H0 as zero difference/association is a rather weak test of a theory
* There is a fairly good chance that some differences will always be found.
* Given enough power, finding a significant difference is virtually guaranteed- when setting up H0 as zero difference

Question 28

Insufficient attention to verifying the integrity of measurement instruments and experimental procedures: voorbeeld

Answer 28

• In Bem’s research and arguably in MRP, insufficient effort is taken to verify the integrity of
  measurement
• Reliability of DV and personality measurements
• Use of ad-hoc instead of validated measures

Question 29

Psychological processes are context sensitive
which makes validation of psychological
measurement very difficult

Answer 29

oke

Question 30

wat zijn karakteristieken van surveys

Answer 30

• subjective: interpretation and phrasing might differ per researcher
• context sensitive: social desirability, test-retest reliability is rarely checked.

Question 31

welke 4 dingen horen bij psychometrics

Answer 31

• Test-retest reliability
• Internal consistency
• Predictive validity
• Construct validity

Question 32

wat is conflation=

Answer 32

het meten van dezelfde dingen! het is gewoon anders verwoord, maar komt neer op hetzelfde construct

Question 33

conflation voorbeeld

Answer 33

Haapala et al (2014) have found a positive correlation between self-reported physical activity and literacy and numeracy (derived from school tests)
Children aged 6-8 years old had to indicate, together with their parents, the amount of exercise in minutes per day, for multiple separate forms of activity.

= meten van hoe goed iemand kan lezen (door self-report) en hoe goed iemand kan lezen

Question 34

2e voorbeeld van conflation

Answer 34

correlatie meten tussen life satisfaction and positivity

Question 35

3e voorbeeld van conflation

Answer 35

correlatie van theory of planned behaviour: perceived behavioural control (how easy will it be for you to avoid sugar) and intention (how likely is it that you will avoid sugar in the future)

Question 36

context sensitivity=

Answer 36

did the researcher create the response, or does the response reflect an actual attitude?

Question 37

dus conflation wat is hier mis mee

Answer 37

• The fact that we can ask different questions does not mean that we’re actually measuring the same thing
• The fact that A & B are correlated might just reflect that we’re measuring the same thing twice

Question 38

welke twee dingen heb je nodig om wel een goede reflectie te krijgen van response on survey question and attitude

Answer 38

-Validity and reliability
-Clearly defined concepts

Question 39

implicit association test: 3 verwchtingen van mensen

Answer 39

• Designed to bypass socially desirable answers
• Popular in research
• A basis for popular explanations of persisting bias and discrimination

Question 40

waarom hadden ze de IAT opgesteld

Answer 40

omdat explicit racism decreases, but behaviour still reflects some bias. therefore there was a disconnect between behaviour and self-reported belief

Question 41

2 vragen die je kan stellen bij IAT

Answer 41

1. can you really say that reaction time is a good measurement?
2. can you pronounce this in strengths?

Question 42

construct validity kritiek op IAT

Answer 42

How should we interpret the absence of a
consistent relationship with explicit measures of
prejudice? Does the IAT measure an implicit part of the same attitude, or something else altogether? Does the IAT reflect culturally transmitted knowledge/associations? Endorsement of predjudice and stereotypes?

If an IAT measures implicit attitudes, why do most
men not exhibit “implicit sexism”? Is it consistent that IAT measurement does seem in line with one form of bias and prejudice but not with another?

dus komt niet overeen met explicit measures of prejudice, en mannen laten geen implicit sexism zien bij de test.

Question 43

predictive validity van IAT kritiek

Answer 43

“Automatic white preference” would predict (e.g):
* More favorable ratings of white (vs black) job applicants
* More optimal treatment decisions for white (vs black) patients, given
the same symptoms
* More laughter in response to anti-black racial humour

maar….

IAT scores are weak to moderate predictors of
discriminatory “behaviour”

Question 44

test-retest reliability kritiek

Answer 44

IAT scores turn out to be a poor
predictor of future scores by the same
individuals on the same test

• So do IAT scores reflect individual implicit
  bias? The IAT is ill suited for diagnostic purposes. Misschien dan toch meer voor verschillen tussen mensen????

Question 45

wat is arbitrary metrics kritiek op IAT

Answer 45

you should always be able to compare the numbers, anders kan je het niet goed interpreteren. we hebben context nodig!
maar.. de IAT geeft deze context niet! the mean of IAT corresponds with behaviour neutrality. dus je kan niet denken dat 0 = neutral, maar 0.48 = neutral!!!

dit gaat vaak fout. -> moet expliciet zijn!!

Question 46

wat zou een score van 0 dus aangeven op de IAT

Answer 46

een preference for black faces

Question 47

depending on how you rate that midpoint, the bars move more to outgroup bias.

Answer 47

oke

Question 48

wat is het verschil tussen theory testing en diagnostics

Answer 48

theory testing = how are two variables related?
diagnostics = what is the standing of this individual/group on the construct dimension of interest

Question 49

meter reading =

Answer 49

when you think that your scale exactly maps onto the attribute, a score of 0 means neutral, a score at the extreme positive end of the scale is also seen as being at the extreme positive end of the attribute

Question 50

• It’s not self-evident that the midpoint of the “IAT scale” corresponds with behavioural neutrality
• There might be reason to believe that the emperical zero-point (of behaviour) is higher on the IAT scale
• It’s not self-evident that more extreme scores actually reflect more extreme forms of “racial preference”

Answer 50

oke

Question 51

Xu et al (2011) find that men and women ‘in the early stages of a romantic relationship’ show heightened activation in parts of the brain associated with reward, when looking at pictures of their lovers. Specifically, activation in the superior frontal gyrus, was associated with higher relationship satisfaction 1,5 year later.

wat is hier mis mee

Answer 51

= conflation!!!

Question 52

Beware of these ‘implicit fallacies’ or ‘brain heuristics’:

Answer 52

• Implicit measures (like the IAT) do not necessarily reflect our ‘true’ preferences.
• Measuring blood oxygenation isn’t strictly speaking a more ‘direct’ measurement of psychological states, then reaction time measures or even survey questions

Question 53

Psychologists are trying to run with
idealistic all-answering paradigms, rather
than walk with mundane incremental
methodologies. In an effort to scramble
towards useful findings and humanly
meaningful conclusions, they end up cutting
corners around parsimony, accuracy &
falsifiability“ (Hughes, 2016, p178).

Answer 53

we want to avoid overconfidence!

Question 54

hypothetico deductivism =

Answer 54

According to it, scientific inquiry proceeds by formulating a hypothesis in a form that can be falsifiable, using a test on observable data where the outcome is not yet known.

Question 55

A researcher predicts on the basis of existing theory that relationship satisfaction and equity will be related, and sets out to test this hypothesis using quasi-experimental research. This would be most consistent with which approach?

Answer 55

hypothetico deductivism