Quantitative- threats to internal validity

Question 1

7 threats to internal validity

Answer 1

1. history
2. maturation
3. testing
4. instrumentation
5. selection bias
6. attrition (morality)
7. regression to the mean

Question 2

History

Answer 2

• NOT what happens to the participants before they have participated in the study (not personal histories)
• it is an event that happens between the pretest and posttest that will mess with results of posttest

Question 3

Maturation

Answer 3

• people change over time, participants mature

- eg. test anxiety in first year vs third year is going to be affected by maturation

Question 4

Testing

Answer 4

• the mere fact of taking the pretest changes things for posttest
• eg. you ask how people relax and go through methods- this influences people to change their behaviour

Question 5

Instrumentation

Answer 5

• change in the way you measure from pretest to posttest
• eg. different IQ tests at pretest and posttest
• 2 different measures of intelligence we don’t know if they are exactly equivalent. We wont know if this has to do with change of test. Do research into the difference between the two tests- assess before you do the experiment
• If instrumentation changes you introduce a confound- wont know if due to change in instrumentation or genuine change in participants

Question 6

Selection bias

Answer 6

• is when there is an inequality between the pretest and posttest in groups.
• a control group only works if they are like the experimental group
• Eg. Test anxiety treatment programme if we have all the anxious people in experimental group, there will be a difference between control and experiment group- will look like experiment didn’t work. -Randomisation is NB to get away from selection bias.

Question 7

Attrition

Answer 7

-also called study mortality (people drop out of the study)
- It is an issue if dealing with drugs in study. Some will die, some will just drop out. -Problem with attrition: if all attrition all occurs in experimental group- for example: if testing anxiety in test takers the experimental group will maybe take lots of tests to try reduce anxiety. BUT for very anxious people this is unappealing so they drop out. The experimental group then only contains the less anxious people.
=You can bias your results

Question 8

Regression to the mean

Answer 8

• scores are distributed around the mean
• you can catch people in an unusual point when scores are unusually low or high and when you next test them they regress to the mean

Question 9

How to control threats to validity: history

Answer 9

if both control group and experimental group exposed to same historical event. then they will both change in same sort of way. History is controlled for automatically by having a control group.

Question 10

How to control threats to validity: maturation

Answer 10

if your control group and experimental group come from the same population group (eg. First years) all exposed to same environment. The only difference between them is the treatment. A control group controls for maturation.

Question 11

How to control threats to validity: testing

Answer 11

if pretest changes behaviour then the control group will also change. Then maybe a test is all that is needed to change behaviour and the treatment isn’t necessary (cheaper)

Question 12

How to control threats to validity: instrumentation

Answer 12

-Best way to control is to not change your measure- some circumstances you do have to though. Eg. You may use research assistants and they will each interact with people in a different way which may influence results. Way to manage it: research assistant interviews participant 1 for pretest and again for posttest and research assistant 2 interviews participant 2 for pretest and again for posttest.
When you do need to change measure: using a particular iq test- practice effect. You need to do a study to see if one results differ from another

Question 13

How to control threats to validity: selection bias

Answer 13

randomise them using a large sample. No chance for bias. Controls for things you might not think to ask about effects of people’s subjectivity

Question 14

How to control threats to validity: attrition

Answer 14

cope by saying we have 100 people at pretest and 6 drop out, you compute data for them – make up data as if they did posttest- statistical way to predict what their answers were likely to be. – called an attempt treat analysis

-when people drop out they create selection bias
-If they died you assume the worst case scenario- that the drug didn’t help
=So if they died because of the drug you cant ignore them

Question 15

How to control threats to validity: regression to the mean

Answer 15

solve by randomisation

Question 16

Solomon four-group design

Answer 16

exp group 1: pretest, treatment, posttest
exp group 2: treatment, posttest
control 1: pretest, posttest
control 2: posttest

Pretest asks about mindfulness etc. and that the pretest makes people think about the treatment- and they go home and practice it. This is a threat to external validity because your treatment only works in presence of pretest and so it only works in context of your experiment- this is how we control for it (solomon 4 sqr design).
Only use it for threats to external ability. Or test acting in treatment in some way. If pretest works alone then cont 1 will also show change.

If your pretest has an effect then exp 1 and control 1 will show reduction in anxiety at posttest. And exp 2 and control 2 will have higher levels of anxiety at posttest.

Need a very big sample bc you have 4 arms to your study

Question 17

Control problems in experimental research: 2 basic designs

Answer 17

1. between-subjects

2. within-subjects

Question 18

Between-subjects designs

Answer 18

• Used when the IV is a subject variable (e.g., extrovert/introvert; marital status)
• When experience gained in one level would make it impossible to participate in another level

Two ways to cope with having equivalent groups:
Random assignment
Matching

Question 19

Random assignment

Answer 19

goal: to take individual factors that could bias the study, and spread them evenly throughout the different groups of subjects
- not a guarantee
- works best with large numbers

Question 20

Matching

Answer 20

• Useful when you only have a small number of participants available
• Or the matching variable correlates with the DV (i.e., is expected to affect the outcome in some way)
• Or there is some reasonable way of measuring participants on the matching variable

MAIN USE:
1. WHEN YOU CAN’T RANDOMLY ASSIGN
SMALLE SAMPLE GROUP AVAILABLE

Question 21

Within-subjects designs advantages

Answer 21

• need fewer people
• no problems with equivalent groups
• Reduces error variance, since you have no between-condition individual difference – so it gives more statistical power to find an effect if there is one

Question 22

Within-subjects designs: problems

Answer 22

1. practice effects:IQ improves if you carry on taking tests.
2. Fatigue effects- performance declines over the day. Need to keep questionnaires as short as possible.
   3.Carry over effects/order effects- if you drink OJ before brushing teeth= fine, swap= gross. The order matters.
   fix this by randomly assigning them to an order= called COUNTER BALANCING