Critically Thinking about Constructs Flashcards
Name some key features when distinguishing science from other fields
Falsifiability, parsimony, coherence, connectivity, replicable/ openness, systematic evidence
Substantive hypothesis > ___________ hypothesis > ____________ hypothesis
Research; statistical
What is meant by interpretation bias?
A bias towards interpretations that favour a researcher’s theory, both when getting significant results and when not
the tendency to interpret the failure to confirm predicted outcomes in terms of method-relevant beliefs but confirmed predictions in terms of theory-relevant beliefs
What is the Quine-Duhem problem
it is impossible to test a specific hypothesis because it requires multiple assumptions. If the conclusion is not true, it is not clear which part of the premise we can refute, because it entails many assumptions.
How does the Quine-Duhem problem apply to the hypotheses?
If you have falsified the statistical hypothesis, or found evidence for it then it is an assumption that this also applies to the substantive bias.
What are these theory relevant belief/ assumptions known as?
Substantive assumptions
What assumptions might fall under method relevant beliefs? (2)
Assumptions about measurement & research etc
Why is interpretation bias more likely to occur in psychology than the natural sciences?
In psychology our method related beliefs are peripheral an our theory-relevant beliefs are too central according to L&B
What does L&B mean by saying that our theory-relevant beliefs are too central to psychology?
Our empirical predictions are often indistinguishable from very general assumptions about human behaviour and are difficult to falsify
The more central these assumptions are, the less stringent the theory can be tested
What does the degree of corroboration depend on?
How strict the test is/ to what extent you expose the theory to falsification
What three common issues with modal reasearch practices in psychology further exacerbate the problem with interpretation bias?
- Overemphasis on conceptual replication
- Problematic implementation of NHST (Null hypothesis significance testing; strawman etc)
- Insufficient attention to verifying the integrity of measurement instruments and experimental procedures
Give two criticisms of surveys
Inherently subjective; interpretation and phrasing, language is often vague
Survey measurement is context sensitive; influenced by social desirability and test-retest desirability is rarely checked in general surveys
What is meant by conflation with constructs?
When the operalisations are actually measuring the same thing
What is IAT?
Implicit association test (IAT) = a test that is designed to bypass socially desirable answers and supposedly measures implicit associations. The scores on an IAT reflect differences in average reaction times.
What is the problem with drawing inferences from IAT
It is not self evident that the midpoint or extreme areas of the “IAT scale” corresponds with behavioural neutrality
Give three criticisms of the IAT
• it is not sure what the test really measures (construct validity), because sexism, for example, is
not shown while it is a related construct;
• results are highly unstable (test-retest reliability), making the test ill-suited for diagnostic
purposes, and;
• IAT scores are weak to moderate predictors of discriminatory behavior (predictive validity).
What did Bem’s research show?
Bem’s publication was consistent with the common way of doing research and publishing results (modal research practices). Using MRP he ‘proved’ that the future influences the past (pre-cognition). Thus, we either accept pre-cognition or reject MRP.
What does it mean to say that method-relevant beliefs are central?
When method-relevant beliefs are central, research procedures and measuring instruments are unambiguously defined and validated.
What is meant by conservatism in theory choice?
the preference for the interpretation that keeps established beliefs intact as much as possible. The more central method-relevant beliefs are, the more conservative we have to be.
What is the problem with overemphasising conceptual replication?
when the replication fails, it is unclear if the underlying theory is false or if there were methodological flaws introduced by changes in the conceptual replication. On the basis of conceptual replication we can’t disprove previous research, this can only be done with direct replication.
What three recommendations are given for strengthening method-relevant beliefs?
- stronger emphasis on close replication;
- routinely verifying the integrity of measurement instruments and experimental procedures, and;
- using stronger, more diagnostic form of NHST.
Why is predictive validity important for IAT?
Predictive validity is crucial because the IAT lacks face validity (= the degree to which a measure appears effective in measuring what it intends to measure).
What is meant by the implicit sexism puzzle?
the puzzling finding that men usually don’t exhibit implicit sexism and women show pro-female attitudes, while on the IAT the common finding is that both groups show sexism towards women.
What two score interpretation problems are associated with the IAT?
Reaction times: because of the attitude categories of the IAT, it is possible for persons with
different associations and association strengths for the opposing attitude objects to receive similar IAT scores. There is no evidence linking scores to observable behaviors or showing that persons in the same bias category reliably show the same behavioral patterns. Thus, it is impossible to give meaning and practical significance to IAT scores.
Relative association strength: implicit attitudes can be defined as nothing more than evaluative associations of varying strengths with attitude objects. The IAT measures relative association strength with reaction times. A faster reaction is interpreted as a higher association strength. This way, implicit prejudice can be found whenever associations with different groups aren’t equal in strength while it may just be that one has less experience with some attitude object and thus, weaker associations.
