Construct and statistical concl validity Flashcards

1
Q

Construct validity

A

What is the intervention and why did it produce the effect?
Addresses the presumed cause or explanation of the causal relation b/w the intervention and outcome
Discuss after the internal validity is established

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Key question for construct validity

A

Is the reason for the relation b/w the intervention and bx change due to the construct (explanation, interpretation) given by the investigator?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Features within the experiment that can interfere w interpretation of the results

A

Confounds
Possibility that a specific factor varied or co-varied w the intervention
Could be responsible for the results

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Questions concerning construct validity

A

What is the intervention?

Does the intervention incl other components than those discussed by the investigator?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Features associated w the intervention that interfere w drawing inferences about the basis for the difference b/w groups referred to as..

A

Threats to construct validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Types of threats to construct validity

A

Attention and contact w clients
Single operations and narrow stimulus sampling
Experimenter expectancies
Cues of the experimental situation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Attention and contact w the clients

A

Attention and contact accorded to the experimental grp, or differential attention across experimental and control grps may be the basis for differences observered
Intervention assoc w all aspects incl administration
Threat when attention, contact w clients, and their expectations might plausibly account for findings and were not controlled for by the design

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Placebo effect and construct validity

A

Expectancies for improvement generated by placebos must be controlled if an investigator wishes to draw conclusions about specific effects of the interventions
To examine basis for effects (construct validity) - must incl a third grp that received a placebo on the same schedule of administration

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Attn is parsimonious bc..

A

construct provides and explanation of the effects of many studies in which tx is better than a control grp

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Single operations and narrow stimulus sampling

A

Features that the investigator considers irrelevant to the study, but these features may introduce ambiguity in interpreting the findings

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Single operations and narrow stimulus sampling - key question for construct validity

A

Whether the intervention is responsible for the outcome
OR was it some seemingly irrelevant feature with which the intervention was associated

Key = being unable to separate the constructs of interest from the conditions of its delivery

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Experimenter expectancies

A

When experimental expectancies provide a plausible rival interpretation of the effects otherwise attributed to the experimental manipulation or intervention

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Statistical concl validity

A

Refers to facets of the quantitative evaluation that infl the concls we reach about the experimental condition and its effect

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Statistical evaluation viewed from 2 standpoints

A
  1. Understanding the tests themselves and their bases - what the tests accomplish and the formulae and derivations of the tests
  2. (Complementary) Computational aspects of the tests - application of the tests to the data sets, use of software, and interpretations of the findings
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Threats to statistical concl validity refer to…

A

Facets of the results and statistical evaluation that can obscure interpretation of the experiment

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Ho specifies…

A

That there are “no” differences b/w groups

17
Q

Reject the null hypothesis when…

A

There is a statistically significant difference found

18
Q

Accept a null hypothesis when…

A

No statistical difference is found

19
Q

Threats to statistical concl validity - types

A
Low statistical power 
Variability in procedures 
Subj heterogeneity
Unreliability of measures 
Multiple comparisons and error rates
20
Q

Low statistical power

A

Threat to statistical concl validity
Weak power or low probability of detecting a difference if one exists
May concl that no difference exists when there is a difference

21
Q

How to increase power?

A

Increase sample size

22
Q

Variability in procedures

A

Threat to statistical concl validity
Sometimes can tell by the design that there will be relatively high levels of variability and great difficulty in demonstrating a difference b/w conditions
Higher variability = lower effect size

23
Q

Subj heterogeneity

A

Threat to statistical concl validity
Greater heterogeneity = less likelihood of detecting a difference b/w conditions
Greater variability in reactions to measures

24
Q

Addressing subj heterogeneity

A
  1. Chose homogenous samples

2. Chose heterogenous samples but ensure that the impact of characteristics can be evaluated by the design

25
Unreliability of measures
Likely to obtain lower effect sizes Reliability = matter of degree and refers to the extent of the variability in responding Unreliable measure = greater portion of subj's score due to unsystematic, random variation
26
Multiple comparisons and error rates
The more tests that are perform = the more likely a difference will be found even if no true differences exist TYPE I ERROR Risk = specified by alpha Multiple comparisons increase alpha
27
Experiment-wise error rate
Risk across several statistical tests