Ch11

Question 1

Six threats to internal validity in one-group pretest/posttest designs

Answer 1

• Maturation threats
• History threats
• Regression threats
• Attrition threats
• Testing threats
• Instrumentation threats

(also combined threats)

Question 2

Maturation threats + prevention

Answer 2

change in behavior that happens spontaneously over time

To prevent:
add comparison group-
if treatment group improves significantly more than comparison group, subtract the effect of maturation to see results of the treatment alone

Question 3

History threats + prevention

Answer 3

when an external, “historical,” event affects members of the treatment group at the same time as the treatment

To prevent: comparison group-
if both groups experienced the same external circumstances, the treatment group should still improve more than the comparison group

Question 4

Regression threat (regression to the mean) + prevention

Answer 4

• statistical threat in which an extreme mean(avg) at time one regresses back to a more avg mean at time 2
• Only occur when there’s an extreme pretest score
• To prevent: use comparison groups-
  regression might be present if one group starts off with extreme scores, or if both start with extreme scores and end with moderate scores

Question 5

Attrition threat + prevention

Answer 5

Attrition threat: when there’s a reduction in # of participants between the pretest and posttest (people drop out)

• Only a problem if systematic- depends on what the scores are- closer to the mean is better

Prevent by:
- removing the participants who dropped out from pretest avg

• Look at pretest score of dropouts- if their scores are extreme, more likely to threaten internal validity

Question 6

Testing threat + prevention

Answer 6

type of order effect- change in participants as a result of experiencing the dependent variable ( a test) more than once

To prevent:
- Posttest only design

• Different versions of test for pretest and posttest
• Comparison group- if larger change in treatment group, then testing threat can be ruled out

Question 7

Instrumentation threats (instrumentation decay) + prevention

Answer 7

Instrumentation threats (instrumentation decay): when a measuring instrument changes over time
- can be observer/coder

Prevent:
- Posttest only

• Make sure pretest and posttest forms are equivalent
  Retrain observers throughout study (use clear coding manuals)
• Counterbalance pretest and posttest forms

Question 8

combined threats

Answer 8

Selection-history threats: an outside event or factor systematically affects participants at one level of the IV

Selection-attrition threat: participants in one experimental group experience attrition

Question 9

3 Potential Internal validity threats to any study-

Answer 9

Observer bias
Demand characteristics
Placebo effect

Question 10

observer bias + prevention

Answer 10

researcher’s expectations influence their interpretation of results

Prevent by: double blind study or masked design

Question 11

demand characteristics

Answer 11

participants change their behavior after figuring out the study’s purpose

Prevent by: double blind study or masked design

Question 12

Placebo effects + prevention

Answer 12

when people receive a treatment (inert or real) and improve only because they believe it’s working

• Double-blind placebo control study: patients and those treating don’t know who receives the real treatment

prevention:
Introduce third group (comparison group) that receives no treatment or placebo, if there’s a placebo effect, the placebo group should improve more than the no treatment group

Question 13

Null effects

Answer 13

independent variable didn’t change the dependent variable, no covariance

Question 14

types of null effects

Answer 14

• Not enough btwn groups difference
• Within-groups variability obscured group differences
• No difference

Question 15

null effect: Not enough btwn-groups difference types

Answer 15

• Weak manipulations
• insensitive measures
• Ceiling and floor effects
• Reverse design confound

Question 16

Weak manipulations

Answer 16

type of null effect: Not enough btwn-groups difference

not enough of an increase in levels of IV to make a difference (in the dependent variable)

Question 17

Insensitive measures

Answer 17

type of null effect: Not enough btwn-groups difference

Insensitive measures: dependent variable not operationalized with enough sensitivity

Needs detailed, quantitative increments instead of 2-3 levels

Question 18

Ceiling and Floor effects:

Answer 18

When participants scores on the dependent variable are clustered on either the high or low end

-Can be result of problematic independent variable (weak manipulation), or a problematic dependent variable (insensitive measures)

Question 19

Design Confounds acting in reverse

Answer 19

• Want to be sure if there aren’t changes, it’s really because there isn’t an effect of the IV on the dependent variable

-Can counteract true effects of an IV

Question 20

noise (error variance, unsystematic variance)

Answer 20

• unsystematic variability within each group
• obscures differences btwn each group b/c there’s more overlap btwn the groups
• A Statistical validity problem- the more overlap btwn groups, the smaller the effect size, less likely to be statistically significant

Question 21

Types of within-group differences

Answer 21

• Measurement error
• Individual differences
• Situation noise

Question 22

Measurement error + prevention

Answer 22

Measurement error: a human or instrument factor can inflate or deflate a person’s true score on the dependent variable

• DV score = participants true score +/- random error of measurement

To reduce:
- Use reliable (internal, interrater, test-retest) , precise measurement
- Measure more instances
(Measure larger sample- random errors will cancel each other out)

Question 23

Individual differences + prevention

Answer 23

spread out scores within each group, obscuring btwn-group differences

• Can be a problem in independent-groups design

To reduce:
- Change to within-groups (or matched groups)
-Add more participants

Question 24

Situation noise + prevention

Answer 24

any kind of external distraction that causes variability within-groups that obscure btwn-group differences

Minimize by controlling surrounding of an experiment

Question 25

power + what kind of validity

Answer 25

likelihood that a study has statistically significant results when the IV actually has an effect on the DV

• aspect of statistical validity

Question 26

What kind of studies are more likely to detect true differences

Answer 26

Studies with a lot of power are more likely to detect true differences
- Can detect even small effects

-Strong manipulation is similar to having large effects