P&B Chapter 10: Rigor and Validity in Quant. Research

Question 1

Extent to which appropriate inferences can be made; how well a test measures what it is purported to measure

Answer 1

Validity

Question 2

Reasons that an inference could be wrong

Answer 2

threats to validity

Question 3

Validity of inferences that there truly is an empirical relationship, or correlation between the presumed cause and the effect

Answer 3

statistical conclusion validity

Question 4

Validity that shows that it is the independent variable, rather than something else, that caused the outcome

Answer 4

internal validity

Question 5

Validity that ensures that the measure is actually measuring what it is intended to measure, and not other variables of inferences; “from the observed persons, settings, and cause-and-effect operations included in the study to the constructs these these instances might represent; degree to which an intervention is a good representation of the underlying construct that was theorized as having the potential to cause beneficial outcomes

Answer 5

construct validity

Question 6

Validity that concerns whether inferences and observed relationships will hold over variations in persons, setting, time or measures of the outcomes; generalizability of causal inferences

Answer 6

external validity

Question 7

Involves using information about people’s characteristics to create comparable groups

Answer 7

Matching (also called pair matching)

Question 8

Refers to the ability to detect true relationships among variables

Answer 8

Statistical power

Question 9

Extent to which the implementation of an intervention is faithful to its plan

Answer 9

Intervention Fidelity (or treatment fidelity)

Question 10

This threat to internal validity refers to proving that the cause proceeded the effect

Answer 10

Temporal ambiguity

Question 11

This threat to internal validity concerns bias resulting from preexisting differences between groups. When participants aren’t randomly assigned to groups, the groups being compared could be non-equivalent. This can be reduced by collecting participant characteristics prior to the occurrence of the independent variable and then designing study around that

Answer 11

Selection bias

Question 12

Threat to internal validity: The threat of _____ refers to the occurrence of external events that take place concurrently with the independent variable, and that can affect the outcomes

Answer 12

History

Question 13

Threat to internal validity: Refers to processes occurring within participants during the course of the study as a result of the passage of time rather than as a result of the independent variable

Answer 13

Maturation

Question 14

Threat to internal validity: Threat that arises from attrition (the ‘wearing away’ or progressive loss of data in research. Attrition occurs when cases are lost from a sample over time or over a series of sequential processes) in groups being compared

Answer 14

Mortality/attrition

Question 15

Threat to internal validity: Effects of taking a pretest on people’s performance on a posttest (just the act of taking data from people changes them)

Answer 15

Testing

Question 16

Threat to internal validity: bias that reflects changes in measuring instruments or methods of measurement between two points of data collection

Answer 16

instrumentation

Question 17

Threats to construct validity: participants may behave in a particular manner because they are aware of their role in a study

Answer 17

reactivity to the study situation (Hawthorne effect)

Question 18

Threats to construct validity: researchers influence on participant responses through subtle communication about desired outcomes

Answer 18

researcher expectancies

Question 19

Threats to construct validity: when a treatment is new, participants and research agents alike might alter their behavior

Answer 19

novelty effects

Question 20

Threats to construct validity: examples are compensatory equalization where health-care or staff members try to compensate for the control group members failure to receive a perceived beneficial treatment; compensatory rivalry where control group members have a desire to demonstrate that they can do as well as those receiving treatment

Answer 20

compensatory effects

Question 21

Threats to construct validity: when participants in control group receive similar services to those available in the treatment condition

Answer 21

treatment diffusion or contamination

Question 22

Threats to external validity: effect observed with certain types of people but not others

Answer 22

interaction between relationship and people

Question 23

Threats to external validity: innovative treatment might be effective because it is paired with other elements (ex. an enthusiastic and dedicated project director)

Answer 23

interaction between causal effects and treatment variation

Question 24

A solution to the conflict between external and internal validity is to use a phased series of studies: this first phase uses tight controls, strict intervention protocols, and stringent criteria for including people in the RCT

Answer 24

efficacy study

Question 25

A solution to the conflict between external and internal validity is to use a phased series of studies: the second phase is used once the intervention has been deemed effective under tight conditions, it is then tested with larger samples in multiple sites under less restrictive conditions

Answer 25

effectiveness studies