Week 3: Internal and External Validity

Question 1

Why is it important to inference causally and give an example?

Answer 1

The ultimate aim of research questions is to infer the consequences of a change in the world (treatment/policy), e.g. the effect of an advertising campaign (X) to cut smoking (Y)

Question 2

What does association mean?

Answer 2

Means hat two variables move together

Question 3

What are two other words for association?

Answer 3

• Covariation
• Correlation

Question 4

What does causation mean?

Answer 4

That one of the variables (X) is causing at least part of the movement in the other variable (Y).

Question 5

Does association imply causation?

Answer 5

No

Question 6

How would we go about testing if X causes Y?

Answer 6

Randomised experiment where the researcher both controls and randomly assigns values of the independent variable (X) to the participants.

Question 7

What is internal validity?

Answer 7

The extent to which we are confident that results of a study establish causality between X and Y.

Question 8

Which two things should a design eliminate to be classed as internal validity?

Answer 8

• If the design eliminates reverse causality
• If the design eliminates all or most confounders (Z)

Question 9

What is external validity?

Answer 9

The extent to which you can translate your knowledge to other settings, e.g. whether the results can be generalised to other situations, people, countries, etc.

Question 10

List 5 threats to internal validity.

Answer 10

• Selection
• Confounding
• Design contamination (diffusion)
• Experimenter bias
• Mortality (differential drop-out/attrition)

Question 11

How is selection a threat to internal validity?

Answer 11

Selection bias can reduced the extent to which samples are representative of the populations they are drawn from.

Question 12

How is confounding a threat to internal validity?

Answer 12

There may be another variable Z that can influence both X and Y, so it’s hard to tell whether the predictor caused Y or if it was the confounding variable.

Question 13

How is design contamination a threat to internal validity?

Answer 13

The control group may have found out or known about the treatment group, skewing their feedback.

Question 14

How is experimenter bias a threat to internal validity?

Answer 14

Experimenter may behave in different ways to control and treatment groups.

Question 15

How is mortality - differential drop out - a threat to internal validity?

Answer 15

Differential rates of attrition between treatment and control groups can skew results.

Question 16

List the disadvantages of using experiments to solve the threats towards internal validity.

Answer 16

• Low external validity
• Not every cause can be manipulated
• Experiments can be ethically problematic
• A causal effect established by an experiment may not be the most important one

Question 17

What part of the randomised experiment is external validity related to?

Answer 17

The random sampling part

Question 18

What does external validity assure?

Answer 18

Assures that if we replicate the same experiment
on a different sample the results are the same.

Question 19

What is convenience sampling?

Answer 19

Collecting samples by taking samples that are conveniently located around a location or internet service.

Question 20

List two threats to external validity.

Answer 20

• Differences in population (population validity)
• Differences in settings (ecological validity)

Question 21

What is population validity?

Answer 21

Describes whether you can reasonably generalise the findings from your sample to a larger group of people (the population), revolving around how representative of the population the sample is.

Question 22

What is ecological validity?

Answer 22

Measures how generalisable experimental findings are to the real world, such as situations or settings typical of everyday life.