Chap 4: Validity & Control

Question 1

What is validity defined as in terms of research?

Answer 1

An indication of accuracy in terms of the extent to which a research conclusion corresponds with reality.

Question 2

What are the 4 main types of validity that are considered when designing and evaluating a piece of research?

Answer 2

Internal validity

Construct validity

External validity

Statistical conclusion validity

Question 3

What is internal validity?

Answer 3

The extent to which a study provides evidence of a cause-effect relationship between the IV and the DV.

An experiment with high internal validity would show that the IV caused the DV to change.

Question 4

What is construct validity in terms of research?

Answer 4

The extent to which the results support the theory behind the research.

An experiment with high construct validity would rule out all possible theoretical explanations of the results.

Question 5

What is external validity?

Answer 5

The extent to which the results or findings of an experiment generalises to other situations or populations. Usually in terms of time.

Question 6

What is statistical conclusion validity?

Answer 6

The extent to which the results are shown to be due to the cause-effect relationship or were they accidental or caused by pure chance.

Question 7

What are the 7 threats to internal validity?

Answer 7

Ambiguous temporal precedence (Unclear which variable is the cause or effect, chicken or the egg effect)

History/events outside the laboratory (External events outside may influence the results in different conditions and timings)

Maturation (For longitudinal studies, related to the amount of time during measurements)

Practice effects (Performance on the second test is influenced by simply having taken the first test)

Regression effect (Regression towards the mean, the tendency of subjects with extreme scores in the first measure to score closer to the mean on the second)

Selection (Bias in group assignment and selection of members of groups)

Mortality (Attrition, dropping out of subjects before an experiment is completed)

Question 8

What are the threats to construct validity in terms of research?

Answer 8

Loose connection between theory and method (Poor operational definition of theoretical concepts)

Ambiguous effects of IV (Participants perceive the situation differently than the experimenter, IV is affected differently in different participants)

Question 9

What are the threats to external validity?

Answer 9

Other subjects (Participants should be chosen with equal attention to their representativeness relative to some larger population, do not assume that one person or animal can be substituted for any other)

Other times (Would the same experiment conducted at one time produce the same results when conducted at another time?)

Other settings (Laboratory effect, results may not survive transplantation to the world outside of the laboratory)

Question 10

What are the threats to statistical conclusion validity?

Answer 10

Problem of power (Low sample sizes and too few observations may lead to an erroneous conclusion and lacks in statistical power)

Inaccurate effect size estimation (IV causing a very minute effect on the DV, due to the size of the relationship being measured too poorly)

Question 11

What is control in terms of research?

Answer 11

Means used to rule out threats to the validity of research.

Question 12

What is a control group?

Answer 12

A group who does not receive the experimental treatment, it is usually the standard to which comparisons are made in an experiment.

Question 13

What is a between-subjects experiment design?

Answer 13

A design where each subject experiences only ONE condition or treatment.

Question 14

What is a within-subjects experiment design?

Answer 14

A design where each subject experiences EVERY condition or treatment.

Question 15

What are the control methods for a between-subjects experiment design?

Answer 15

Random assignment (Each subject has an equal and independent chance of being placed in every condition)

Matching (Experimental and control groups are equated on one or more variables before the experiment)

Question 16

What are the control methods for a within-subjects experiment design, what are its advantages and disadvantages?

Answer 16

Subject as own control (Subjects also go through the control condition during the experiment, reduces random error, however prone to practice effects)

Question 17

What is replication with regards to experiments?

Answer 17

Method of control where the experiment is repeated to see if the same results are found the second time.

Question 18

A survey found that individuals who regularly listen to classical music tend to report lower levels of stress and improved productivity than those who did not listen to classical music. However, because the study was not a true experiment, it is unclear whether listening to classical music leads to lower negative emotions and better productivity.

1) Identify an alternative explanation (a confounding variable) for this association.

2) Describe the design for (a) between-subjects and (b) within-subjects experiments that would determine whether listening to classical music would result in lower negative emotions and higher productivity.

Answer 18

1) Temporal precedence (e.g., individuals with less stress may listen to classical music)

2) Between-subject: Randomly assign all participants to classical music vs control groups

Within-subject: Each participant is subjected to all experimental conditions

Question 19

What are the advantages and disadvantages of a within-subject design?

Answer 19

Advantages:
Fewer participants required and higher statistical power
Reduce errors due to individual differences as each participant is their own baseline

Disadvantages:
Carryover effects (e.g., performance in one condition may affect performance in the subsequent condition)
Practice effects (e.g., participants may get better at taking the test)