Chapter 11: Experiments and Observational Studies

Question 1

What are the elements of a designed randomized experiment?

Answer 1

1. Experimental units (sometimes called subjects or participants) are assigned at random to treatments.
2. The experimenter manipulates factors, setting them to specified levels to establish the treatments.
3. A quantitative response variable is measured or observed for each experimental unit.
4. We can attribute statistically significant differences in the response to the differences among the treatments

Question 2

What are the four principles of experimental design?

Answer 2

1. Control sources of variation other than the factors being tested. Make the conditions as similar as possible for all treatment groups except for differences among the treatments.
2. Randomize the assignment of participants to treatments.
3. Replicate by applying each treatment to more than one participant.
4. Block the experiment by grouping together participants who are similar in important ways that you cannot control.

Question 3

What are the differences between experiments and surveys?

Answer 3

1. Surveys try to estimate facts (parameter) about a population, so they require a representative random sample from that population.
2. Experiments try to estimate the differences in the effects of treatments. They randomize a group of experimental units to treatments, but there is no need for the experimental units to be a representative sample from the population.
3. Be alert for possible confounding due to a variable that is not under control affecting the responses differentially.

Question 4

Define ‘Observational study’.

Answer 4

A study based on data in which no manipulation of factors has been employed.

Question 5

Define ‘Retrospective study”.

Answer 5

An observational study in which subjects are selected and then their previous conditions or behaviors are determined. Because retrospective studies are not based on random samples, they usually focus on estimating differences between groups or associations between variables.

Question 6

Define ‘Prospective study’.

Answer 6

An observational study in which subjects are followed to observe future outcomes. Because no treatments are deliberately applied, a prospective study is not an experiment. Nevertheless, prospective studies typically focus on estimating differences among groups that might appear as the groups are followed during the course of the study.

Question 7

Define ‘Experiment’.

Answer 7

An experiment manipulates factor levels to create treatments, randomly assigns subjects to these treatment levels, and then compares the responses of the subject groups across treatment levels.

Question 8

Define ‘Random assignment’.

Answer 8

To be valid, an experiment must assign experimental units to treatment groups at random. This is called random assignment.

Question 9

Define ‘Factor’.

Answer 9

A variable whose levels are controlled by the experimenter. Experiments attempt to discover the effects that differences in factor levels may have on the response of the experimental units.

Question 10

Define ‘Response variable’.

Answer 10

A variable whose values are compared across different treatments. In a randomized experiment, large response differences can be attributed to the effect of differences in treatment level.

Question 11

Define ‘Experimental units’.

Answer 11

Individuals on whom an experiment id performed. Usually called subjects or participants when they are human.

Question 12

Define ‘Level’.

Answer 12

The specific values that the experimenter chooses for a factor are called the levels of the factor.

Question 13

Define ‘Treatment’.

Answer 13

The process, intervention, or other controlled circumstance applied to randomly assigned experimental units. Treatments are the different levels of a single factor or are made up of combinations of levels of two or more factors.

Question 14

Define ‘Statistically significant’.

Answer 14

When an observed difference is too large for us to believe that it is likely to have occurred naturally, we consider the differences to be statistically significant. Subsequent chapters will show specific calculations and give rules, but the principle remains the same.

Question 15

Define ‘Control group’.

Answer 15

The experimental units assigned to a baseline treatment level, typically wither the default treatment, which is well understood, or a null, placebo treatment. Their responses provide a basis for comparison.

Question 16

Define ‘Blinding’.

Answer 16

Any individual associated with an experiment who is not aware of how subjects have been allocated to treatment groups is said to be blinded.

Question 17

Define ‘Single-blind/Double-blind’.

Answer 17

There are two main classes of individuals who can affect the outcome of an experiment:
1. Those who could influence the results (the subjects, treatment administrators, or technicians).
2. Those who evaluate the results (judged, treating physicians, etc.).
When every individual in either of these classes is blinded, an experiment is said to be single-blind. When everyone in both classes is blinded, we call the experiment double-blind.

Question 18

Define ‘Placebo’.

Answer 18

A treatment known to have no effect, administered so that all groups experience the same conditions. Many subjects respond to such a treatment (a response known as the placebo effect). Only by comparing with a placebo can we be sure that the observed effect of a treatment is not due simply to the placebo effect.

Question 19

Define ‘Placebo effect’.

Answer 19

The tendency of many human subjects (often 20% or more of experimental subjects) to show a response when administered a placebo.

Question 20

Define ‘Blocking’.

Answer 20

When groups of experimental units are similar, it is often a good idea to gather them together into blocks. By blocking, we isolate the variability attributable to the differences between the blocks so that we can see the differences caused by the treatments more clearly.

Question 21

What are 2 types of designs?

Answer 21

1. In a completely randomized design, all experimental units have an equal chance of receiving any treatment.
2. In a randomized block design, the randomization occurs only within blocks.

Question 22

Define ‘Matching’.

Answer 22

In a retrospective or prospective study, subjects who are similar in ways not under study may be matched and then compared with each other on the variables of interest. Matching, like blocking, reduces unwanted variation.

Question 23

Define ‘Completely randomized two-factor experiment’.

Answer 23

Experimental units assigned at random to any of the treatments when two factors are involved.

Question 24

Define ‘Confounding’.

Answer 24

When the levels of one factor are associated with the levels of another factor in such a way that their effects cannot be separated.