Foundations of Design

Question 1

Non-experimental design

Answer 1

Descriptive

Correlational

Researcher gathers data without making any kind of intervention

Question 2

Experimental designs

Answer 2

Non-randomized (quasi experimental)

Randomised

Aim to examine associations in order to make predictions or explore causal linkages

Question 3

Non-experimental: descriptive

Answer 3

Used to assess prevalence, incidence rates

Question 4

Non-experimental correlational

Answer 4

Examine the relationship between two or more variables to see whether they covary, correlation or are associated with each other

Question 5

Two types of correlational design

Answer 5

Cross-sectional (all observations made only once at a single time)

Longitudinal (measurements made at two or more time points)

Question 6

Inferring causality from correlational research

Answer 6

1. Covariation (variables must occur together)
2. Precedence (the hypothesized causal variable must reliably precede the outcome variable)
3. Exclusion of alternative explanations
4. Logical mechanism (there must be a plausible account/THEORY for they hypothesized causation)

Question 7

Problems with causation

Answer 7

1. Bidirectionality (two-way causality; X –> Y OR Y –> X)
2. Spurious association (no relationship between X and Y, but by a third variable)
3. Mediation or Moderation

Question 8

Non - experimental: Quasi/non-randomized experimental design

Answer 8

One group posttest only design (X O; intervene and then observe)

One group pretest posttest design (O X O)

Question 9

RCT

Answer 9

Gold standard

Pretest-posttest design with randomised groups

Question 10

No-treatment control

Answer 10

Control group gets zero treatment

Question 11

Wait-list controls

Answer 11

Delay before treatment

Question 12

Placebo control

Answer 12

No real treatment given

Question 13

Comparative treatment groups

Answer 13

Alternative treatment used which is also effective (‘treatment as usual’)

Question 14

Dismantling studies

Answer 14

Break apart treatment into components, and use each component in isolation

Question 15

Good experimental feature designs

Answer 15

• Patient homogeneity
• Randomised assignment
• Specific intervention (manualisation)
• Control
• Low attrition
• Groups treated equivalently (except intervention)
• Double/triple blind
• Independent replication

Question 16

Three types of experimental validity

Answer 16

1. External validity
2. Internal validity
3. Statistical conclusion validity

Question 17

External validity

Answer 17

To what extent can the study results be generalized to other samples with different characteristics than the study sample?

Question 18

Internal validity

Answer 18

Degree to which causality can be inferred from a study

To what extent the intervention/manipulation and not chance can account for the study results

Question 19

Spontaneous remission (threat to IV)

Answer 19

Recovery from a disorder without reason or intervention

Question 20

Interfering events (threat to IV)

Answer 20

Significant events that occur between pre and posttest measurements (natural disaster, finds a partner, wins lotto etc.)

Question 21

Secular drift (threat to IV)

Answer 21

Long term social trends taking place over time (e.g. smoking, premarital sex, etc.)

Question 22

Maturational trends (threat to IV)

Answer 22

Growth or maturation of the person that may account for some of the findings

Question 23

Regression to the mean

Answer 23

Extreme scores revert towards the mean of a distribution when a measurement is redistributed

Extreme scores can only get better, even if the treatment isn’t working

Question 24

Attrition (threat to IV)

Answer 24

Loss of participants over time

Only a problem if its different between conditions OR there are different reasons

Question 25

Diffusion (threat to IV)

Answer 25

Participants in the control group may receive aspects of the intervention (drug sharing, talking amongst participants)

e.g. workplace studies

Question 26

Special treatment/reaction of controls

Answer 26

Controls may be aware that they are not receiving treatment and may be motivated to out perform the experimental group

May feel demoralized or lose interest

Question 27

Poor adherence to treatment (threat to IV)

Answer 27

Doesn’t attend all treatment sessions or take medication (50% of schizophrenics)

Question 28

Confounded manipulation (threat to construct validity)

Answer 28

More than one thing can be accidentally manipulated (e.g. therapy and therapist)

Question 29

Expectancy effects (threat to construct validity)

Answer 29

Sometimes people get better simply by thinking the experiment will work (placebo)

Question 30

Hawthorne (threat to construct validity)

Answer 30

Sometimes people appear to get better simply because they’re being observed (i.e., like the attention)

Question 31

Statistical conclusion validity

Answer 31

The extent to which the analyses performed enables one to draw correct/true inferences about the phenomena of interest

Question 32

Threats to statistical conclusion validity

Answer 32

Lower power (small n/effect size)

Improper data analysis (reporting only the “best” results)

Confounding variables

Poor reliability/validity of measures

Question 33

The dead salmon study

Answer 33

Brain activity “found” in a dead fish

If you do a lot of statistical tests some will give you result just by chance

Question 34

Systemic pressure on statistics

Answer 34

Best journals favour novel and statistically significant results, thus researchers are motivated to keep running statistics until they find something that fits these criteria