Construct and statistical concl validity

Question 1

Construct validity

Answer 1

What is the intervention and why did it produce the effect?
Addresses the presumed cause or explanation of the causal relation b/w the intervention and outcome
Discuss after the internal validity is established

Question 2

Key question for construct validity

Answer 2

Is the reason for the relation b/w the intervention and bx change due to the construct (explanation, interpretation) given by the investigator?

Question 3

Features within the experiment that can interfere w interpretation of the results

Answer 3

Confounds
Possibility that a specific factor varied or co-varied w the intervention
Could be responsible for the results

Question 4

Questions concerning construct validity

Answer 4

What is the intervention?

Does the intervention incl other components than those discussed by the investigator?

Question 5

Features associated w the intervention that interfere w drawing inferences about the basis for the difference b/w groups referred to as..

Answer 5

Threats to construct validity

Question 6

Types of threats to construct validity

Answer 6

Attention and contact w clients
Single operations and narrow stimulus sampling
Experimenter expectancies
Cues of the experimental situation

Question 7

Attention and contact w the clients

Answer 7

Attention and contact accorded to the experimental grp, or differential attention across experimental and control grps may be the basis for differences observered
Intervention assoc w all aspects incl administration
Threat when attention, contact w clients, and their expectations might plausibly account for findings and were not controlled for by the design

Question 8

Placebo effect and construct validity

Answer 8

Expectancies for improvement generated by placebos must be controlled if an investigator wishes to draw conclusions about specific effects of the interventions
To examine basis for effects (construct validity) - must incl a third grp that received a placebo on the same schedule of administration

Question 9

Attn is parsimonious bc..

Answer 9

construct provides and explanation of the effects of many studies in which tx is better than a control grp

Question 10

Single operations and narrow stimulus sampling

Answer 10

Features that the investigator considers irrelevant to the study, but these features may introduce ambiguity in interpreting the findings

Question 11

Single operations and narrow stimulus sampling - key question for construct validity

Answer 11

Whether the intervention is responsible for the outcome
OR was it some seemingly irrelevant feature with which the intervention was associated

Key = being unable to separate the constructs of interest from the conditions of its delivery

Question 12

Experimenter expectancies

Answer 12

When experimental expectancies provide a plausible rival interpretation of the effects otherwise attributed to the experimental manipulation or intervention

Question 13

Statistical concl validity

Answer 13

Refers to facets of the quantitative evaluation that infl the concls we reach about the experimental condition and its effect

Question 14

Statistical evaluation viewed from 2 standpoints

Answer 14

1. Understanding the tests themselves and their bases - what the tests accomplish and the formulae and derivations of the tests
2. (Complementary) Computational aspects of the tests - application of the tests to the data sets, use of software, and interpretations of the findings

Question 15

Threats to statistical concl validity refer to…

Answer 15

Facets of the results and statistical evaluation that can obscure interpretation of the experiment

Question 16

Ho specifies…

Answer 16

That there are “no” differences b/w groups

Question 17

Reject the null hypothesis when…

Answer 17

There is a statistically significant difference found

Question 18

Accept a null hypothesis when…

Answer 18

No statistical difference is found

Question 19

Threats to statistical concl validity - types

Answer 19

Low statistical power 
Variability in procedures 
Subj heterogeneity
Unreliability of measures 
Multiple comparisons and error rates

Question 20

Low statistical power

Answer 20

Threat to statistical concl validity
Weak power or low probability of detecting a difference if one exists
May concl that no difference exists when there is a difference

Question 21

How to increase power?

Answer 21

Increase sample size

Question 22

Variability in procedures

Answer 22

Threat to statistical concl validity
Sometimes can tell by the design that there will be relatively high levels of variability and great difficulty in demonstrating a difference b/w conditions
Higher variability = lower effect size

Question 23

Subj heterogeneity

Answer 23

Threat to statistical concl validity
Greater heterogeneity = less likelihood of detecting a difference b/w conditions
Greater variability in reactions to measures

Question 24

Addressing subj heterogeneity

Answer 24

1. Chose homogenous samples

2. Chose heterogenous samples but ensure that the impact of characteristics can be evaluated by the design

Question 25

Unreliability of measures

Answer 25

Likely to obtain lower effect sizes
Reliability = matter of degree and refers to the extent of the variability in responding
Unreliable measure = greater portion of subj’s score due to unsystematic, random variation

Question 26

Multiple comparisons and error rates

Answer 26

The more tests that are perform = the more likely a difference will be found even if no true differences exist
TYPE I ERROR
Risk = specified by alpha
Multiple comparisons increase alpha

Question 27

Experiment-wise error rate

Answer 27

Risk across several statistical tests