Chapter 13 - Experiments and Observational Studies

Question 1

observational study

Answer 1

A study based on data in which no manipulation of factors has been employed. In this study, researchers do not assign choices; they simply observe them.

Question 2

retrospective study

Answer 2

An observational study in which subjects are selected and then their previous conditions or behaviors are determined. Need not be based on random samples and they usually focus on estimating differences between groups or associations between variables.

Question 3

prospective study

Answer 3

An observational study in which subjects are followed to observe future outcomes. No treatments are deliberately applied, so not an experiment. Typically focus on estimating differences among groups that might appear as the groups are followed during the course of the study.

Question 4

What is it not possible for observational studies to do?

Answer 4

To demonstrate a causal relationship.

Question 5

An experiment:

Answer 5

MANIPULATES factor levels to create treatments, randomly ASSIGNS subjects to
these treatment levels, and then COMPARES the responses of the subject groups across treatment levels.

Question 6

What must an experimenter identify, manipulate, and measure?

Answer 6

IDENTIFY at least one explanatory variable, called a factor to MANIPULATE. IDENTIFY at least one response variable to MEASURE.

Question 7

What distinguishes an experiment from other types of investigations?

Answer 7

The experimenter actively and deliberately
MANIPULATES the factors to CONTROL the details of the possible treatments, and ASSIGNS
the subjects to those treatments AT RANDOM.

Question 8

Humans who are experimented on are commonly called:

Answer 8

subjects or participants

Question 9

Generic term for individuals experimented on:

Answer 9

experimental unit

Question 10

What are the levels of a factor?

Answer 10

The specific values that the experimenter chooses for a factor.

Question 11

What is a treatment?

Answer 11

The process, intervention, or other controlled circumstance applied to randomly assigned experimental
units. Treatments are the different levels of a single factor or are made up of combinations
of levels of two or more factors.

Question 12

What are the four principles of experimental design?

Answer 12

1. Control
2. Randomize
3. Replicate
4. Block (not required in experimental design)

Question 13

How do we control sources of variation other than the factors we are testing?

Answer 13

By making conditions as similar as possible for all treatment groups.

Question 14

What does randomization allow us to do?

Answer 14

It allows us to EQUALIZE the effects of unknown or uncontrollable sources of variation.

Question 15

The outcome of an experiment on a single subject is what?

Answer 15

An anecdote (NOT data)

Question 16

What are the kinds of replication?

Answer 16

1. Apply each treatment to a number of subjects. Only with such replication can we estimate the variability of responses.
2. When the experimental units are
   not a representative sample from the population of interest. (Then you would want to replicate the entire study with a different group of subjects, preferably
   from a different part of the population.)

Question 17

When should randomization be used to equalize variation across treatment groups?

Answer 17

In the long run.

Question 18

What is a factor?

Answer 18

A variable whose levels are manipulated by the experimenter. Experiments attempt to discover
the effects that differences in factor levels may have on the responses of the experimental units.

Question 19

What is a response?

Answer 19

A variable whose values are compared across different treatments. In a randomized experiment, large response differences can be attributed to the effect of differences in treatment level.

Question 20

Why block?

Answer 20

To reduce the effects of identifiable attributes of the subjects that cannot be controlled. Blocking is an important compromise between randomization and control.

Question 21

When do we consider the difference to be statistically significant?

Answer 21

When an observed difference is too large for us to believe that it is likely to have occurred naturally.

Question 22

What is the control group?

Answer 22

The experimental units assigned to a baseline treatment level, typically either the “well understood” default
treatment, or a null, placebo treatment. Their responses provide a basis for comparison.

Question 23

Any individual associated with an experiment who is not aware of how subjects have been
allocated to treatment groups is said to be _________.

Answer 23

blinded

Question 24

What are the two main classes who can effect the outcome of an experiment?

Answer 24

• those who could influence the results (the subjects, treatment administrators, or technicians).
• those who evaluate the results (judges, treating physicians, etc.).

(blinding one entire class is a single-blind, blinding both would be a double-blind)

Question 25

What is a placebo?

Answer 25

A treatment known to have no effect, administered so that all groups experience the same conditions. Only
by comparing with a placebo can we be sure that the observed effect of a treatment is not due sim-
ply to the placebo effect.

Question 26

What is the placebo effect?

Answer 26

The tendency of many human subjects (often 20% or more of experiment subjects) to show a response even when administered a placebo.

Question 27

What should one do when groups of experimental units are similar? Why?

Answer 27

It is often a good idea to gather them together
into blocks (blocking) using a blocking variable. This isolates the variability attributable to the differences between the blocks so that we can see the differences caused by the treatments more clearly.

Question 28

What is matching?

Answer 28

In a retrospective or prospective study, subjects who are SIMILAR in ways NOT under study may be matched and then compared with each other on the variables of interest.

Question 29

What are the designs for experiments?

Answer 29

A COMPLETELY RANDOMIZED DESIGN, in which all experimental units have an equal chance of
receiving any treatment. A RANDOMIZED BLOCK DESIGN, in which the randomization occurs only within blocks.

Question 30

What happens when we say factors are confounded?

Answer 30

The levels of one factor are associated with the levels of another factor in such a way that their effects cannot be separated.

Question 31

What is the recommended way to start comparing results for treatment groups?

Answer 31

Boxplots.