Part 2 - Experimental Design

Question 1

What is Within-Subjects-Design?

Answer 1

different levels of the focus variable(s) (treatment(s)) are used for each subject (=no different treatment groups)

Question 2

What is Between-Subjects-Design?

Answer 2

levels or steps of focus variable(s) vary across subjects (=different treatment groups)

Question 3

What are advantages of Within-Subjects-Design?

Answer 3

you need less respondents and you can directly compare the reactions of each individual to the different conditions

Question 4

What are disadvantages of Within-Subjects-Design?

Answer 4

effects of certain conditions may not be independent of the other condition(s) being also presented

Question 5

What are advantages of Between-Subjects-Design?

Answer 5

• treatments are independently measured from each other
• blinding participants
• comparing different treatments

Question 6

What is statistical power?

Answer 6

The power to reject the Null-Hypothesis

Question 7

What is direct experimental control?

Answer 7

–holding a certain variable constant at some convenient level throughout the experiment, or
–controlling it at two or more levels, so called treatment variables

Question 8

What is factorial design?

Answer 8

checking interactions between several treatment variables
•Factor: a specific treatment dimension (i.e. probability level)
•Step: a certain level of the treatment dimension (i.e. chances of 1 in 10 versus 1 in 20)
•2 x 2 or „two times two“ design means that we have two factors with two steps of each

Question 9

What does “having a confounded treatment” mean?

Answer 9

the treatment manipulation varies too many things at once - therefore, one cannot separate the effects and actually does not know anything

Question 10

what is the difference between interaction and correlation?

Answer 10

• correlation: no IV, DV, just two correlated variables

- interaction: needs three variables

Question 11

difference between interaction and linear regression?

Answer 11

• linear regression only “needs” 2 variables

- interaction “needs” 3 variables

Question 12

What is an interaction effect?

Answer 12

one variable influences the relationship between the independent and dependent variable

Question 13

What is the “main effect of A”?

Answer 13

The general effect that variable A has on the outcome

Question 14

What do we mean by “indirect control via randomization”?

Answer 14

• Some variables are difficult or impossible to control

* The solution lies in making uncontrolled variables independent of treatment variables by randomization

Question 15

What is “random block”?

Answer 15

–One or more nuisance variables are controlled as treatments rather than randomized
–Nuisance variables are often labeled blocking variables, held constant within a block (or subset of trials) and varied across blocks

Question 16

What is a one-shot case study?

Answer 16

X O
•Total absence of control —almost no scientific value
•Often implicit comparison with general expectations of what the data would have looked like had the X not occurred; so-called “common-knowledge” comparisons

Question 17

What is a One-Group Pretest-Posttest Design and what are potential problems?

Answer 17

O₁ X O₂
•Problem: variables jeopardizing internal validity:
–History (time lapse relevant for the importance of the problem)
–Maturation (of respondents)
–Testing (itself)
–Instrumentation (calibration of measurement)
–Statistical regression (to the mean); a severe problem especially if respondents are selected because of particularly poor or well performance in O1

Question 18

What is a Static-Group Comparison and what are potential problems?

Answer 18

    O₂

•Problems/factors needing control:
– Selection (non-homogeneity of the groups)
– Mortality (respondents not participating anymore dependent of treatment)

Question 19

What is a Pretest-Posttest Control Group Design and what are potential problems?

Answer 19

R O₁ X O₂
R O₃ O₄
•Introducing control group advantageous, but there are still potential problems remaining
–History is controlled only insofar as general historical events that might have produced an O1–02 difference would also an O3–04 difference
–Maturation and testing are controlled since they should affect both experimental and control groups
–Instrumentation is controlled if there is a standardized test
–Regression is controlled if mean differences are concerned and if both experimental and control groups are randomly assigned from the same pool (no matter how extreme it is)
•Selection is ruled out as explanation of difference if randomization has in fact assured group equality at time R
•Mortality is always difficult to handle: if experimental condition requires attendance at certain sessions while the control condition does not, sample biases may be introduced

Question 20

What is a Solomon Four-Group Design and what are potential problems?

Answer 20

R O₁ X O₂
R O₃ O₄
R X O₅
R O₆
Explicit consideration of external validity factors; with one experimental and one control group lacking the pretest, effects of testing as well as interaction of testing and X can be measured
downside: very much effort! might not be necessary

Question 21

What is a Posttest-Only Control Group Design and what are potential problems?

Answer 21

• Pretests are not essential to true experimental designs; the most adequate all-purpose assurance of lack of initial biases between groups is randomization
• Can be considered the last two groups of the Solomon Four-Group-Design, controls for testing as main effect and interaction, but does not measure them —what is in fact not crucial

Question 22

What is a normative vs. behavioral hypothesis?

Answer 22

• normative = rational, economic

- behavioral = what to test

Question 23

What do you need to discuss when describing an experimental design?

Answer 23

• treatment
• observations
• within/between subject design and exact setup