Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Research Methods > Chapter 10: More on Experiments- Confounding and Obscuring Variables > Flashcards

Chapter 10: More on Experiments- Confounding and Obscuring Variables Flashcards

1
Q

one group, pretest/posttest design?

A
  • one group of participants, measured on a pretest, exposure to treatment, intervention or change and than measured on a posttest. (bad experiment)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What are six threats to internal validity that the one group, pretest/posttest design introduces?

A
  1. maturation threats
  2. history threats
  3. regression threats
  4. attrition threats
  5. testing threats
  6. instrumentation threats
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Maturation threats ?

A

change in behaviour merges more or press spontaneously over time…people slowly adapt to strange environments.
L> spontaneous remission…symptoms get better for an unknown reason.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Prevention of maturation threats?

A

comparison group

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

History threats?

A

threats to internal validity that occur when a historical or external event occurs to everyone in the treatment group at the same time as the treatment so it is unrelieved ….to be one it must effect everyone or almost everyone

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Prevention of History Threats?

A

a comparison group

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Regression threat??

A

aka regression toward the mean
L> when a performance is extreme at time 1 the next time that performance is measured it is likely to be less extreme than that is closer to a typical of avg performance.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Regress works at??

A

both extremeness

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Regression and internal validity?

A

when measured in a pretest condition are extreme on DV….most likely to be a threat
L> if a group is unusually high pretest we can expect their scores to go towards the number

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Preventing regression?

A

comparison

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Regression is a big threat when?

A

one group in exactly this situation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Attrition??

A

occurs when people drop out of the study before it ends.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

How does Attrition usually happen in pret-test and post-test designs and when does it become a threat to internal validity?

A
  • when the two tests are administered on separate days…..and its a threat to internal validity when it is systematic…aka when only a certain kind of participant drops out
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Is a comparison group a cure all for attrition?

L> if not why?

A

NO
L> in a two group experiment like a prettest or post-test design if both groups experience the same pattern of dropouts then attrition is not an internal validity threat. But it can be a threat when only one group experiences it.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

How do we prevent attrition threats?(2)

A
  • remove the original scores of the participants that drop out so only the scores of those that fully complete the study are included.
  • another way is to check the pretest scores of dropouts if they have extreme scores on the pretest their attrition is more threat to internal validity than if it was closer to the group avg.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Testing threat?
L> what type of effect are they?
L> they lead to what five things?

A
  • kind of order effect
  • scores have changed over time just because participants have taken the test more than once……they become practiced….bored….jitters for the first testing, sensitization (pretest may sensitize the participant to certain things that influence their post-test answers)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

How do we prevent testing threats?

A

-abandon a pretest all together and use only post test design
- if a pretest is used.they might opt to use alt forms of the test for the two measurements.
- comparison group
L> it takes both the pre test and post test …if a larger effect is seen in the treatment group = testing threats are not harming internal validity.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Instrumentation threats?

A
  • aka instrumental decay

- occur when a measuring instrument changes over time from having been used before.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

How do testing threats differ from instrumentation threats?

A

they occur when a participant changes over time from having been tested before.
L> instrumentation = instruments used to measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

How do we prevent instrumentation threats?

A

-use only a post-test only design
- if a pre-test, post-test is used….one should make sure the pretest and post-test measurements are equivalent
L> collect data from each instrument and be sure they are calibrated the same
- avoid shifting standards of behavioural coders….via training the coders.. multiple times throughout the experiment establishing reliability and validity at both pretest and post test.
- use clear coding manuals
- counterbalance the versions of the test giving some version A at pretest and version B at post test and giving other participants version B and than version A .

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Many threats to one group pretest/post-test design are corrected by adding a comparison group When one has a two group pretest/post-test design what are three more threats to internal validity ?

A
  • observer bias, demand characteristics and placebo effects
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Observer bias?

A

when researchers expectations influence their interpretation of the results or even influence the outcome of the study.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

Observer biases can threaten what two validities?

A
  • internal validity because an alt explanation exists for the results….and construct validity of the DV because it means that the results do not represent the true levels
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Comparison groups always or do not always control for observer bias?

A
  • not
  • if the experimenter knows what participants are in each group their expectations could lead them to see more improvement in the therapy groups vs the comparison group
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

Demand characteristics?

A

when participants guess the studies purpose and change their behaviours in the expected direction.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

What can control for both demand characteristics and observer bias?

A
  • double blind study in which neither the participants nor the researchers who evaluate them know who is in the treatment group and who is in the comparison group
27
Q

When trying to control for both demand characteristics and observer bias what can be used when double blind studies are not possible?

A
  • a variation of it can be done

L> ex: in some studies participants know which group they are in but the observers do not…single blind study?

28
Q

How does only participating in one condition of an experiment control for demand characteristics and observer biases?

A

participants were blind to the reason

29
Q

Keeping observers blind to the conditions is especially important when?

A

they re rating behaviours that are more difficult to code

30
Q

Placebo effect?

A

when people receive a treatment and really improve but only because they believe they are receiving a valid treatment.
L> inert pill, injection or therapy = placebo

31
Q

Are placebos imaginary?

A

no

32
Q

How do researchers rule out the placebo effects in a study?

A
  • using a double blind placebo control study
    L> one group receives the real drug or therapy, the second group receives the placebo drug or therapy. Neither the person treating the patients nor the patients themselves know whether they are in the real group or the placebo group.
33
Q

If the treatment group shows a larger increase/improvement in comparison to the placebo group what does this mean?

A
  • placebo effects plus the effects of the real drug
34
Q

If someone wants to show a placebo effect specifically you would need to include what?

A
  • a no treatment comparison group one that receives neither drug nor placebo.
35
Q

Null effect??

A

when researchers find that the IV did not make a difference in the DV

36
Q

Why do we not hear much about null effects?

A
  • journals, newspapers and webistes are more likely to report the results of a study in which the IV does have an effect
37
Q

Do null effects occur only in post-test only designs?

A
  • no

- they also occur in within groups design or a pretest/posttest design and even correlational studies

38
Q

What is a very simple explanation that a null effect is occurring?

A
  • that the IV really does not affect the DV
39
Q

What is a second possible explanation for a null effect?

A
  • the study was not designed well enough …aka the IV does effect the DV but some obscuring factor in the study prevented the researchers from detecting the true difference.
40
Q

When a study returns a null result sometimes the culprit is the design of the study. What are three explanations for this?

A
  • weak manipulation, insensitive measures and reverse confounds
41
Q

Weak manipulations?

A
  • how the researchers operationalized the IV…aka ask about construct validity
    ex: someone gave each group money 0, 0.25 and 1.00.
    vs
    giving each group 0, 5 and 150 dollars
    L> each group will yield very different patterns results
42
Q

Insensitive measures?

A
  • researchers have not used an operationalization of the DV with enough sensitivity
  • when it comes to dependent measures it is smart to use dependent measures that have detailed, quantitative increments not just two or three levels.
43
Q

What are ceiling and floor effects?

A
  • special cases of weak manipulations and insensitive measures.
  • cause IV groups to score almost the same on the dependent variable.
  • all scores are squeezed together either at the high end (ceiling) or the low end ( floor)
44
Q

Ceiling and floor effects can be the result of what?

A
  • a problematic IV

L> as a result the various levels of the IV would appear to make no difference

45
Q

Other than a problematic IV what can also lead to ceiling and floor effects?

A
  • poorly designed DV
    L> the measure for the DV results in low scores in all groups
    L> no room for between group variability
46
Q

Manipulation checks can hep with what?

A
  • detect weak manipulations, ceilings and floors

L> help identify potential problems with the IV….a separate DV is included to make sure the manipulation works

47
Q

Explain confounds acting in reverse?

A
  • when a study is designed in such a way that a confound actually counteracts some true effect of an IV
48
Q

What is noise?/noisy data?

A
  • too much unsystematic variability within each group can rust in a null effect
    L> it gets in the way of detecting a true difference between groups. aka error variance
49
Q

Noisy data can come from three sources what are they?

A
  • measurement error
  • individual differences
  • situation noise
50
Q

How does more variability within groups obscure the difference between the groups?

A

there is more overlap between the members of the two groups…the greater the overlap the smaller the effect size and the less likely it is that the two group means will be stat sig

51
Q

Why do most researchers prefer to keep within group variability to a minimum?

A

so that they can more easily detect between group differences

52
Q

Measurement error?

A

factors that can inflate or deflate a person’s true score on a DV
- this can be a reason for high within group variability

53
Q

Do all measurements involve a certain amount of measurement error?

A

yes but researchers need to keep those errors as small as possible

54
Q

When measurement errors are random what happens?

A

they cancel each other out across a sample of people and will not effect the group’s average or mean. BUT an operationalization with a lot of measurement error will result in a set of scores that are more spread out around the group mean.

55
Q

With measurement erros the more sources of random error there are in a DV’s measurement that more what exists?

A

more variability within each group in the experiment

56
Q

The more precisely and carefully a DV is measured the less what exists?

A

variably within each group which makes it easier to detect a difference between IV groups

57
Q

What are two solutions to measurement error?

A
  1. reliable, precise measurements
    L> use measurement tools that have excellent reliability (internal, interrupter and test retest)…to reduce errors…..when a tool has good construct validity there will be less error in a measurement
    L> the more precise an experimenter measures the DV the less chance there is that that measurement error will obscure results.
  2. Measure more instances
    L> measure larger sample….aka take more measurements
    L> the more participants there are the better the chances of having a full representation of all possible measurement errors so they cancel each other out and we get a better estimate of the true average for the group
    * also when a measurement tool has little measurement error…researchers will get away with having a small sample
58
Q

Individual differences???

A
  • problem in independent group designs.
  • effect of spreading out the scores of the people within each group
  • data is mixed and far from consistent …a lot of overlap
59
Q

Two solutions to individual differences?

A
  1. use a within groups design instead of a independent groups design
    L> it compares each participant with themselves and controls for individual differences
    L> requires less participants and controls for individual differences in comparison to independent variables.
    L> similar effects are seen in matched groups design
  2. Add more participants
    L> when a great deal of variability exists because of the individual differences, one simple thing to do is to measure more people…the more you measure the less impact any one extreme person will have on the avg.
    L> reduces impact of individual differences within groups and will increase a study’s ability to detect differences between groups. ( think of t-test formula )
60
Q

Situation noise??

A

third factor that could cause variability within groups and obscure true group differences
L> variability in the external situation can great variability within each group in an experiment

61
Q

How do researchers control for situation noise?

A

controlling the external situations
L> sometimes these controls need to be extreme
- they typically try to control the potential distractions that might affect the DV

62
Q

If measurement error, individual differences , and situation noise were all investigated and the study was found to be conducted appropriately one may conclude what??

A

that the IV truly does not affect the DV

63
Q

Publication bias??

A
  • differences seem more interesting than null effects so scientific journals magazines and newspapers that pick up story ideas from them are more interested in IVs that matter than those that do not.

Research Methods (10 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions