Hypothesis Testing, Validity, and threats to Validity

Question 1

Please name the four types of validity!

Answer 1

1. statistical validity
2. construct validity
3. External Validity
4. Internal Validity

Question 2

1. statistical validity

Answer 2

focusses on the question whether or not the conclusions are reasonable given the statistical laws. Important are validated instruments and large sample sizes.

Question 3

2. construct validity

Answer 3

focusses on how well an instrument measures the intended construct, often investigated by comparing results of an instrument with previous and validated research.

Question 4

3. External Validity

Answer 4

refers to the degree of generalizability of the results to other context.

Question 5

4. Internal Validity

Answer 5

the extent to which what you measure is really due to the manipulation.

Question 6

Could you name the nine major confounding variables?

Answer 6

Maturation, History, Testing, Instrumentation, Regression to the mean, Selection, Experimental mortality, design contamination, order effects.

Good Job!

Question 7

What’s the sequence effect? (Maturation)

Answer 7

changes in the dependent variable due to normal developmental changes, such as a declining rate of good answers in a long exam due to fatigue
Solution:
Counterbalancing of sequences (half: AB; rest: BA)
Counterbalancing of sequences for more than two conditions.
- complete counterbalancing
- Latin Square

Question 8

Whats the History confounder?

Answer 8

changes in the dependent variable due to unexpected wituational changes during the study, but not related to the study.

Question 9

Whats the carry-over effect (confounder)?

Answer 9

changes in the dependent variable scores due to pre-testing, for example pre-testing pre-testing can adjust people to the way of questioning, they learn.
Solution:
Often asymmetrical: A  B ≠ B  A
This “asymmetrical transfer” is unsolvable in a within-subjects design
Solution: between-subjects design

Question 10

The instrumentation confounder

Answer 10

a change in the way a dependent variable was measured during the study can influence the results.

Question 11

Regression to the mean

Answer 11

when participants are selected because of their extreme scores, they tend to have less extreme scores in the future because they get back to their mean, this does not mean that the treatment was succesful, its just their mean

Question 12

What do we call the subject or participant effect?

Answer 12

changes in the participants behavior that are more likely due to participating than to the variables under investigation.

Question 13

placebo effect

Answer 13

occurs when participants have a strong expectation about the treatmentand project this expectation on themself, so its either a false sense of effect or the effect is due to the participnts expectations and not due to the treatment itself.

Question 14

experimenter effects:

Answer 14

biasing effects cause by the actions ofthe researcher.

Question 15

What are the advatages of between-subject designs?

Answer 15

advantage: no sequence/carryover effects
disadvantage: potential group differences
 random assignment of subjects to conditions
 random assignment of matched pairs to conditions

Question 16

What are the (dis)advantages of within-subject designs?

Answer 16

advantage: no group differences; subject is its own control
disadvantage: chance of sequence effects
Randomization of sequences
Counterbalancing (complete or Latin Square)
do not use in case of (asymmetrical) carryover effects