Chapter 11: More on Experiments: Confounding and Obscuring variables

Question 1

One-group, pretest/post test design

Answer 1

An experiment in which a researcher recruits one group of participants; measures them on a pretest; exposes them to a treatment, intervention, or change; and then measures them on post test. AKA really bad experiment!!

Question 2

Maturation threat

Answer 2

A threat to internal validity that occurs when an observed change in an experimental group could’ve emerged more or less spontaneously over time. Using a pre-test/post test design (comparison group) helps control for maturation threats.

Question 3

History threat

Answer 3

A threat to internal validity that occurs when it is unclear whether a change in the treatment group is caused by the treatment itself or by an external or historical factor that affects most members of the group. Using a pretest/posttest design (comparison group) helps control for history threats.

Question 4

Regression threat

Answer 4

A threat to internal validity related to regression to the mean a phenomenon in which any extreme finding is likely to be closer to its own typical, or mean, level the next time it is measured (with or without the experimental treatment or intervention). Only occurs when a group is measured twice and has an extreme score at pre-test. Using a pretest/posttest design (comparison group) and carefully inspecting pattern of results can help control for regression threats.

Question 5

Regression to the mean

Answer 5

A phenomenon in which an extreme finding is likely to be closer to its own typical, or mean, level the next time it is measured because the same combination of chance factors that made the finding extreme are not present the second time.

Question 6

Attrition threat

Answer 6

In a pretest/post test, repeated-measures, or quasi-experimental study, a threat to internal validity that occurs when a systematic type of participant drops out of the study before it ends. Removing dropout scores from pre-test data can help control for attrition threats. Researchers can check pretest scores of dropout participants to determine their level of extremity and if they need to be removed (more extreme scores are a higher threat to internal validity).

Question 7

Testing threat

Answer 7

In a repeated-measures experiment or quasi-experiment, a kind of order effect in which scores change over time just because participants have taken the test more than once; includes practice effects. Using a posttest-only design, a pretest/post-test design with alternative measurements, and/or having a comparison group helps control for testing threats.

Question 8

Instrumentation threat

Answer 8

A threat to internal that occurs when a measuring instrument changes over time. Can occur with observational research when coders change standards, and when different measurements for pretest and posttest are not sufficiently equivalent. Using a posttest-only design or ensuring pretest and posttest measures are equivalent can help control for instrumentation threats. Retraining behavioral coders throughout experiment and having clear coating manuals can help control for instrumentation threats in observational research.

Question 9

Instrumentation versus testing threats

Answer 9

An instrumentation threat means the measuring instrument has changed from Time 1 to Time 2, whereas a testing threat means the participants change over time from having been tested before.

Question 10

Selection-history threat

Answer 10

A threat to internal validity in which a historical or seasonal event systematically affects only the participants in a treatment group or those in the comparison group, not both.

Question 11

Selection-attrition threat

Answer 11

A threat to internal validity in which participants are likely to drop out of either the treatment group or the comparison group, not both.

Question 12

Observer bias

Answer 12

Observe her expectations influence the interpretation of participant behaviors or the outcome of the study. A threat to both internal validity and construct validity of the dependent variable. Using a double-blind study can control for observer bias.

Question 13

Demand characteristic

Answer 13

A cue that leads participants to guess a study’s hypothesis or goals; a threat to internal validity. Using a double-blind study can control for demand characteristics.

Question 14

Double-blind study

Answer 14

A study in which neither the participants nor the researchers who evaluate them know who is in the treatment group and who is in the comparison group.

Question 15

Masked design

Answer 15

A study design in which the observers are unaware of the experimental conditions to which the participants have been assigned (but the participants are aware). AKA blind design. Alternative option when a double blind study is not possible.

Question 16

Placebo effect

Answer 16

A response or effect that occurs when people receiving an experimental treatment experience a change only because they believe they are receiving a valid treatment. Using a double-blind placebo control study can control for placebo effects.

Question 17

Double-blind placebo control study

Answer 17

A study that uses a treatment group and a placebo group and in which neither the researchers nor the participants know who is in which group.

Question 18

Null effect

Answer 18

I’m finding that an independent variable did not make a difference in the dependent variable; there is no significant variance between the two. Also called null result.

Question 19

Ceiling effect

Answer 19

An experimental design problem in which independent variable groups score almost the same on a dependent variable, such that all scores fall at the high end of their possible distribution. Can be detected using a manipulation check.

Question 20

Floor effect

Answer 20

An experimental design problem in which independent variable groups score almost the same on a dependent variable such that all scores fall at the low end of their possible distribution. Can be detected using a manipulation check.

Question 21

Noise

Answer 21

Unsystematic variability among the members of a group in an experiment, which might be caused by situation noise, individual differences, or measurement error. AKA error variance.

Question 22

Measurement error

Answer 22

The degree to which the recorded measure for a participant on some variable differs from the true value of the variable for the participant. Measurement errors may be random, such that scores are too high and too low to cancel each other out; or may be systematic, such that most scores are biased too high or too low. Creates high within-groups variability.
Solutions: Use reliable, precise tools and measure more instances.

Question 23

Solutions for individual differences

Answer 23

Change the design or add more participants.

Question 24

Situation noise

Answer 24

Unrelated events or distractions in the external environment that create unsystematic variability within groups in an experiment.

Question 25

Power

Answer 25

The likelihood that a study will show a statistically significant result when the independent variable truly has an effect in the population; the probability of not making a type II error. Can be increased by adding more participants.