Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Stats > Validity Threats > Flashcards

Validity Threats Flashcards

1
Q

What are the different kinds of validity

A

– Internal
– External
– Statistical

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

An experiment is the most important tool to…

A

identify cause-effect relationships:

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is a cause and effect relationship

A

 In an experiment we control the cause and
observe the effect.
 Changes in the DV are caused by the
manipulation of the IV.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is internal validity

A

The extent to which the results obtained are a function of the variables that were systematically manipulated.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Examples of internal validity

A
  • Are changes in the IV responsible for the observed variation in the DV?
  • Might the variation in the DV be attributable to other causes? (confounds)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Why is internal validity important

A

We often conduct research in order to determine cause-and-effect relationships.

Can we conclude that changes in the independent variable caused the observed changes in the dependent variable?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

High internal validity =

A

Strong evidence of causality.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Low internal validity

A

Little or no evidence of causality.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

To maximise internal validity:

A

●Must be able to rule out the possibility of other factors producing the change (confounds).

●Must control everything and eliminate possible extraneous influences.

●Easiest in highly controlled, laboratory settings.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Threats to internal validity compromise our confidence in saying that

A

a relationship exists between the IV and DV.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Threats to internal validity compromise our confidence in saying that a relationship exists between the IV and DV. They include:

A 
●History effects
●Maturation effects
●Mortality
●Instrument decay
●Participant selection
●Statistical regression to the mean
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

History effects:

A

Events occurring during the experiment that are not part of the treatment.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

History effect example and solution

A

During the experiment, a fire alarm goes off which affects one group.

Solution: Hold experiences constant except for IV; randomly assign conditions to time.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Maturation effects

A

Biological or psychological processes within participants that may change simply due to the passing of time.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Maturation examples

A

– Aging
– Fatigue
– Hunger

These changes occur naturally over time, and influence the results of a research study.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Mortality effects

A

The differential loss of individuals from treatment and/or control groups due to nonrandom reasons.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Mortality problem and solution

A

Problem: Those who drop out of your study could be qualitatively different from those who remain.

Solution: Try to stop people dropping out, e.g., provide an inconvenience allowance.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Instrument Decay:

A

Measuring devices change over time.
Equipment becomes inaccurate with age.
Experimenters become more skilled, or bored.

19
Q

Instrument decay solution

A

Randomise condition to time, check reliability of instruments, train staff.

20
Q

Participant Selection:

A

Different types of participants placed at the different levels of the IV.

21
Q

Participant selection and solution

A

–Example: Only males in control group and only females in experimental group.
–Example: If you test different groups at different times, then people will be different.
Solution: Random assignment, matching.

22
Q

When participants communicate…

this effects…

A

●Effects equalising groups

–Diffusion of treatment effects

–Compensatory rivalry

–Compensatory equalisation

●Effect separating groups

–Resentful demoralisation

23
Q

Diffusion of Treatment:

A

Occurs if the control group learns about the manipulation.
Can occur when participants talk about an ongoing experiment.
Do not debrief any participants until ALL participants have finished.

24
Q

Compensatory Rivalry:

A

When participants in different conditions start competing.
Example:
If participants become aware that they are in the control group, they may work harder to overcome the ‘disadvantage’ of being in the control group.
Saretsky (1972) named this the “John Henry effect”

25
Q

Compensatory equalisation:

A

What if experimenters know which condition participants are in?

Experimenters in charge of the control group might feel bad that their group is not receiving treatment, and attempt to make up for this by providing enhanced services that go beyond the routine treatment regimen.

26
Q

Resentful Demoralisation:

A

If the control group learns that they are in the control group, they may become resentful and not try as hard.
This would increase the size of the observed difference between experimental and control groups.

27
Q

External validity: generalisation

A

●Does the IV represent the concept we intend?
–A measure is externally valid if it truly measures the hypothetical construct intended.
–An experiment is externally valid if it is similar to phenomenon in the real world.

Having high external validity (as field experiments might) often means having a lack of control of confounds.

28
Q

Types of external validity

A

Population validity

Ecological validity

29
Q

Population Validity

A

The extent to which the results can be generalised from the experimental sample to a defined population.

30
Q

Ecological Validity

A

The extent to which the results can be generalised from the set of environmental conditions in the experiment to other environmental conditions.

31
Q

Threats to external validity compromise our confidence in:

A

stating whether the study’s results are generalisable.

32
Q

Threats to external validity compromise our confidence in stating whether the study’s results are generalisable. They include:

A

● Reactive effects of testing
● Reactive effects of experimental setting
● Selection-treatment interaction
● Multiple-treatment interference

33
Q

Reactive effects of testing:

A

Occurs whenever a pre-test increases or decreases the respondents’ sensitivity to the treatment.
Example: Studies involving self-report measures of attitude and interest are very susceptible to this threat.

34
Q

Reactive effects of experimental arrangements:

A

These can occur when the conditions of the study are such that the results are not likely to be replicated in non-experimental situations:

–Hawthorn effects
–Novelty effects
–Experimenter effects
–Rosenthal effect

35
Q

Selection-treatment interaction:

A

The possibility that some characteristic of the participants selected for the study interacts with some aspect of the treatment.

Examples: May include prior experiences, learning, personality factors, or any traits that might interact with the effect of the treatment.

36
Q

Multiple-treatment interference:

A

When participants receive more than one treatment, the effects of previous treatments may influence subsequent ones.

Examples: This has a likelihood of occurring whenever the same research participants are exposed to multiple treatments.
–Sequence effects
–Carry-over effects

37
Q

Improving external validity

2

A

Replication

Replication with extension

38
Q

Replication

A

An additional scientific study that is conducted in exactly the same manner as the original research project.
–When we replicate an experimental finding, we are able to place more confidence in that result.

39
Q

Replication with extension

A

An experiment that seeks to replicate a previous finding but does so in a different setting or with different participants or under different conditions.

40
Q

Threats to statistical validity

A

● Making a Type 1 Error

Rejecting the null hypothesis when the null hypothesis is true (i.e., false positive).

Possible causes:

–Fishing

41
Q

Threats to statistical validity

A

● Making a Type 2 Error
Failing to reject the null hypothesis when the null hypothesis is false (i.e., false negative).

Possible causes:

–Power
–Reliability of measures, treatments
–Random irrelevance
–Random heterogeneity of respondents

42
Q

Internal validity:

A

History effects, maturation effects, mortality, instrument decay, participant selection, statistical regression to the mean.

43
Q

External validity:

A

Reactive effects of testing, reactive effects of experimental setting, selection-treatment interaction, multiple-treatment interference.

44
Q

Statistical validity:

A

Fishing (Type 1 error), power, reliability of measures, random irrelevance, random heterogeneity of respondents (Type 2 error).

Stats (19 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions