Critically Thinking about Constructs

Question 1

Name some key features when distinguishing science from other fields

Answer 1

Falsifiability, parsimony, coherence, connectivity, replicable/ openness, systematic evidence

Question 2

Substantive hypothesis > ___________ hypothesis > ____________ hypothesis

Answer 2

Research; statistical

Question 3

What is meant by interpretation bias?

Answer 3

A bias towards interpretations that favour a researcher’s theory, both when getting significant results and when not

the tendency to interpret the failure to confirm predicted outcomes in terms of method-relevant beliefs but confirmed predictions in terms of theory-relevant beliefs

Question 4

What is the Quine-Duhem problem

Answer 4

it is impossible to test a specific hypothesis because it requires multiple assumptions. If the conclusion is not true, it is not clear which part of the premise we can refute, because it entails many assumptions.

Question 5

How does the Quine-Duhem problem apply to the hypotheses?

Answer 5

If you have falsified the statistical hypothesis, or found evidence for it then it is an assumption that this also applies to the substantive bias.

Question 6

What are these theory relevant belief/ assumptions known as?

Answer 6

Substantive assumptions

Question 7

What assumptions might fall under method relevant beliefs? (2)

Answer 7

Assumptions about measurement & research etc

Question 8

Why is interpretation bias more likely to occur in psychology than the natural sciences?

Answer 8

In psychology our method related beliefs are peripheral an our theory-relevant beliefs are too central according to L&B

Question 9

What does L&B mean by saying that our theory-relevant beliefs are too central to psychology?

Answer 9

Our empirical predictions are often indistinguishable from very general assumptions about human behaviour and are difficult to falsify

The more central these assumptions are, the less stringent the theory can be tested

Question 10

What does the degree of corroboration depend on?

Answer 10

How strict the test is/ to what extent you expose the theory to falsification

Question 11

What three common issues with modal reasearch practices in psychology further exacerbate the problem with interpretation bias?

Answer 11

1. Overemphasis on conceptual replication
2. Problematic implementation of NHST (Null hypothesis significance testing; strawman etc)
3. Insufficient attention to verifying the integrity of measurement instruments and experimental procedures

Question 12

Give two criticisms of surveys

Answer 12

Inherently subjective; interpretation and phrasing, language is often vague

Survey measurement is context sensitive; influenced by social desirability and test-retest desirability is rarely checked in general surveys

Question 13

What is meant by conflation with constructs?

Answer 13

When the operalisations are actually measuring the same thing

Question 14

What is IAT?

Answer 14

Implicit association test (IAT) = a test that is designed to bypass socially desirable answers and supposedly measures implicit associations. The scores on an IAT reflect differences in average reaction times.

Question 15

What is the problem with drawing inferences from IAT

Answer 15

It is not self evident that the midpoint or extreme areas of the “IAT scale” corresponds with behavioural neutrality

Question 16

Give three criticisms of the IAT

Answer 16

• it is not sure what the test really measures (construct validity), because sexism, for example, is
not shown while it is a related construct;
• results are highly unstable (test-retest reliability), making the test ill-suited for diagnostic
purposes, and;
• IAT scores are weak to moderate predictors of discriminatory behavior (predictive validity).

Question 17

What did Bem’s research show?

Answer 17

Bem’s publication was consistent with the common way of doing research and publishing results (modal research practices). Using MRP he ‘proved’ that the future influences the past (pre-cognition). Thus, we either accept pre-cognition or reject MRP.

Question 18

What does it mean to say that method-relevant beliefs are central?

Answer 18

When method-relevant beliefs are central, research procedures and measuring instruments are unambiguously defined and validated.

Question 19

What is meant by conservatism in theory choice?

Answer 19

the preference for the interpretation that keeps established beliefs intact as much as possible. The more central method-relevant beliefs are, the more conservative we have to be.

Question 20

What is the problem with overemphasising conceptual replication?

Answer 20

when the replication fails, it is unclear if the underlying theory is false or if there were methodological flaws introduced by changes in the conceptual replication. On the basis of conceptual replication we can’t disprove previous research, this can only be done with direct replication.

Question 21

What three recommendations are given for strengthening method-relevant beliefs?

Answer 21

• stronger emphasis on close replication;
• routinely verifying the integrity of measurement instruments and experimental procedures, and;
• using stronger, more diagnostic form of NHST.

Question 22

Why is predictive validity important for IAT?

Answer 22

Predictive validity is crucial because the IAT lacks face validity (= the degree to which a measure appears effective in measuring what it intends to measure).

Question 23

What is meant by the implicit sexism puzzle?

Answer 23

the puzzling finding that men usually don’t exhibit implicit sexism and women show pro-female attitudes, while on the IAT the common finding is that both groups show sexism towards women.

Question 24

What two score interpretation problems are associated with the IAT?

Answer 24

Reaction times: because of the attitude categories of the IAT, it is possible for persons with
different associations and association strengths for the opposing attitude objects to receive similar IAT scores. There is no evidence linking scores to observable behaviors or showing that persons in the same bias category reliably show the same behavioral patterns. Thus, it is impossible to give meaning and practical significance to IAT scores.

Relative association strength: implicit attitudes can be defined as nothing more than evaluative associations of varying strengths with attitude objects. The IAT measures relative association strength with reaction times. A faster reaction is interpreted as a higher association strength. This way, implicit prejudice can be found whenever associations with different groups aren’t equal in strength while it may just be that one has less experience with some attitude object and thus, weaker associations.