Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

HESP724 research design > Exam 1: Experimental Designs > Flashcards

Exam 1: Experimental Designs Flashcards

1
Q

Treatments

A

Levels of the independent variable that you control

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Denotations of variables/designs:

A

X: independent variable
O: dependent variable
R: random assignment

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is an independent variable

A

One or more factor that you manipulate or control

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

True Experiment

A
  • IV AND participants are randomly assigned (R)

* Strong evidence for causal conclusions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Quasi-Experiment

A

• IV and participants are not randomly assigned
• Weaker evidence for causal conclusions
Ex: degrees of HL, age of people

  • Manipulate the IV, but can’t/don’t randomly assign participants to groups or conditions (e.g., intact groups)
  • More susceptible to threats to internal and external validity- weaker evidence for cause-effect conclusions

Threats to internal validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Factorial Design

A
  • May include both true and quasi-experimental components

* Consider a true experiment if even one IV meets criteria, but strong causal evidence only for the true manipulation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Posttest Only

Randomized Treatment Groups

A 
• No-Treatment Control
R X O
R     O
• Alternative Treatment Control
R   X1   O
R   X2  O

• What are other possibilities for post-test only designs? Why would (or wouldn’t) you use this design?

Part of the random assignment means that differences between age groups should be naturally “washed out”… education level, sex, etc. but their may actually be differences between these sub-groups- you can’t always do pre-testing based on the situations

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Pretest-Posttest

Randomized Control Group Design

A 
• Also known as a mixed model
randomly assign, make a measurement, apply treatment and re-measure
• Within-subjects
• Between-subjects
R  O  X  O
R  O       O

Why would (or wouldn’t) you use this design? learning affects from repeated exposure, if learning effect is big in the control this may swamp the effects of the treatment

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Solomon Randomized Four-Group Design

A

need double the participants and the time

Pretest-Posttest
R . O X O
R . O . O

Posttest Only
R . X O
R . O

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Switching Replications Design

A 
R\_\_O\_\_X\_\_O\_\_\_\_O
R\_\_O\_\_\_\_\_O_X\_\_O
Similar to cross over study design
each person is their own control
we hope our interventions/therapies carry over; can’t be sure the no treatment epochs are the same. Differences in exposure for the groups.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Factorial

A

• Multiple IVs examined in one design
• Examine main effects and interactions
The # of #s indicates how many IVs there are
The # itself tells you have many levels of that IV there are

For example: 2 X 3
group (younger vs. older) x treatment type (new1, new2, standard)

a 2X2X2 has 8 conditions

dependency- if the effect of one IV is determined by the other HAs work well when the patient is young but not old

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Nonequivalent

Control Group Design

A

• Compare intact groups
- N (or omitted) denotes nonrandom assignment
• Typically pretest-posttest NOXO
NOO
• What are some other options? What design option might strengthen cause-effect conclusions?

if you found that the treatment is really effective, what do you tell the people that you denied treatment?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Double Pretest

A

N O O X O

N O O O

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Repeated Measures

A

• AKA within-subjects design
• Two or more measures from the same individual
• May be experimental (e.g., group) or non-experimental (e.g., time)
• Measurements may either occur within a particular session (e.g., task conditions) or across multiple sessions
X1 O X2 O
• Why is this a quasi-experimental design? What are the benefits/limitations? not random b/c everyone gets everything

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Counterbalancing

A

• Design to avoid order effects
• Randomly assign people to a given order
R X1 O X2 O
R X2 O X1 O

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Single-Subject Design (concepts)

A

• AKA single case
• Not necessarily conducted on a single participant, but the data are presented in terms of individuals rather than groups
Single subject does not = case study
• Repeated measures: many measures of the target behavior taken over many sessions
• Lacks random assignment
need a lot of samples to know whats going on

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Single-Subject Design Notations

A

Notation
• A = baseline (no treatment)
• B = first treatment
• C (etc.) = second treatment
• Simplest A-B sequence is weak, seldom used
• Many other explanations for changes from pre to posttest
• Might be used as part of documentation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Treatment Withdrawal Design

A

• A1-B-A2
• Also called reversal design
On-Off see if there’s an affect
stronger - if i stop providing the therapy you should go back to baseline
* stronger causal claims if you can show changes when treatment is both added and taken away

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Treatment Replication - withdrawal

A

• A1-B1-A2-B2
• What if the person doesn’t return to baseline?
if A2 doesn’t return totally to baseline, then B2 should be even better

20
Q

Multiple Treatment

A

• A1-B1-A2-C-A3
• Draw conclusions about B, but note that C always
appears after B (serial order confound)
• How might you deal with this if you wanted to draw conclusions about C?

get another person and change the order

21
Q

Multiple Baselines

A

• Across Behaviors
• Observe 2+ behaviors
-One you want to treat first
-One you will treat
differently
• e.g., lists that test perception/ production of speech sounds
• Similar in difficulty, but not overlap in content
• What sorts of treatments is this design good for?

• Across Participants
-Balance treatment orders
across more than one
individual
-Deals with carryover effects
-Example here used with a
treatment replication design
22
Q

Levels of Evidence

A
  • Strength of the evidence
  • How well can you draw causal conclusions?
  • True experiments are sometimes called RCTs

There is not one agreed on hierarchy.

Background information; case-control/reports; cohort studies; RCTs; CATS; systematic reviews

23
Q

what is depth in terms of levels of evidence

A

amount of converging evidence

24
Q

internal validity

A

History, maturation, statistical regression, instrumentation, selection, mortality

  • Extent to which conclusions about cause-effect relationships are accurate
  • Might there be other explanations (covert variables) for the patterns you observed?
25
Q

History

A

Threat to internal validity

• Outside events influence participants in the course of the experiment or between repeated measures of the DV

26
Q

Maturation

A

Threat to internal validity
• Participants may change in the course of the experiment or between repeated measures of the DV due to the passage of time

  • Permanent changes: e.g., biological growth
  • Temporary changes: e.g., fatigue

address this with a control group

maturation tends to be biological growth or fatigue

27
Q

Statistical Regression

A

Threat to internal validity
• AKA Regression to the Mean
• You might select to study participants who perform
• However, participants with extreme scores on a first measure of the DV tend to have scores closer to the mean on a second measure
• Extreme scores are more likely to occur via chance

eliminate the possibility that the response was an outlier (eliminate random noise)

• Example:
• A person scored 750 out of 800 on the quantitative portion of the
• Assuming the same test difficulty and no learning/practice effect, what would you expect the person to score on the second test?
• Ways someone could score 750
1. “true” score is 750 & they had exactly average luck
2. “true” score is < 750 & they had better than average luck
3. “true” score is > 750 & they had worse than average luck
• Few people have “true” scores >750 (~6 in 1,000); more people have true scores 700-750 (~17 in 1,000)
• Thus, more likely that someone scoring 750 is from group 2 than group 3

28
Q

Instrumentation

A

Threat to internal validity
• The reliability of the instrument used to gauge the DV or manipulate the IV may change over the course of an experiment
- Changes in calibration of a measuring device
- Changes in the proficiency of an observer/ interviewer (human “instruments”)
• Possible in any pretest-posttest design
• Importance of recruiting study groups in parallel

29
Q

Selection

A

Threat to internal validity
• Groups differ in a systematic, non-random way prior to a study
• Intact groups (e.g., two groups of children in classrooms)
• Only weaker cause-effect conclusions are possible
• May co-occur with maturation differences (Different learning rate prior to the study)
• May co-occur with history differences
• Differences in teacher style in the two classrooms

30
Q

Mortality

A
  • In the course of an experiment, some subjects may drop out before it is completed (attrition)
  • Particularly problematic if the reason for dropping out is non-random (examples?)
  • How might you deal with this problem?
  • Pre-test comparisons
  • Intention-to-treat vs. as-treated analysis

non-compliance = people only showed up for 3/5 sessions

31
Q

Experimenter/Participant Bias

Threats

A

Rosenthal effect, blinding, placebo effect, hawthorne effect,

32
Q

Rosenthal effect

A

• AKA Pygmalion effect or self-
fulfilling prophecy
• Set up expectations (intentionally or unintentionally)
• Researcher Robert Rosenthal told
some teachers that children in their classes would undergo a jump in intellectual ability
• Differences arose likely just due to different expectations/interactions

33
Q

Blinding

A

• Single Blind: Participant doesn’t know what
group they’re in

• Double Blind: Participant AND researcher
don’t know

• Triple Blind: Participant AND researcher
AND person scoring the data don’t know

34
Q

Placebo Effect

A

Positive effect simply due to believing you’ve received treatment

35
Q

Nocebo Effect

A

Negative effect simply due to believing you’ve received treatment

36
Q

Hawthorne effect

A
  • AKA Reactivity
  • Named for a study of worker productivity at the Hawthorne plant of an electric company
  • Productivity improved regardless of what was changed
  • Behavior changes just because individuals know they are being studied
37
Q

Case Example:
• There were an equal number of boys and girls in the class, so for convenience the boys were assigned to the Control Group and the girls to the Experimental Group.
• One day, the boys were told to go to one room and the girls to another room, where they were given their respective videos. Two days later, the Generalization Probe was conducted.
• The mean score for the Control Group was 1.2 and for the Experimental Group was 3.4. We conclude the video improved turn-taking behavior for children with autism.

What is this type of design called? What is the problem (i.e., is there a threat to internal validity)?

A

post test only non-randomized control group

selection issue

38
Q

Case Example:
• On Day 1, all children viewed the 20-minute cartoon; On Day 2, the Generalization Probe for all students was conducted; On Day 3, all children viewed the 20-minute video; On Day 4, a second Generalization Probe was conducted for all students.
• The mean score for the Control video was 1.2 and for the Experimental video was 3.4. We conclude the video improved turn-taking behavior for children with autism.

What is the design type?
Is there a problem?

A

repeated measures

repeated testing threat

history threats

39
Q 
Case Example:
There were an equal number of boys and girls, so for convenience the boys were assigned to the Control Group and the girls to the Experimental Group. During a class early in the school year, a Generalization Probe was conducted for all children. The experimenter fell ill soon afterwards, and so it wasn't until a class late in the school year that the children were separated into groups, with the control children viewing the 20-minute cartoon and the experimental children viewing the 20-minute video. Two days after, a second Generalization Probe was conducted. We conclude that the 20-minute interactive video improved the children's turn-taking skills.

What is the design type? Is there a problem?

A

non random assignment

pre test and post test design

maturation

40
Q

Case Example:
• The name of each child in the classes was written on a separate slip of paper. All the slips were put in a bowl and mixed up thoroughly. Students were assigned to the Experimental and Control Groups alternately as their names were pulled out of the bowl.
• One day at school, the children in the Control Group were told to go to one room and children in the Experimental Group to another room, where they were exposed to their respective conditions.
• Some of the children in the Experimental Group appeared bored by the interactive video, became disruptive, and were removed from the room. Two days later, the Generalization Probe was conducted.
• The mean score for children in the Control Group was 1.2 and the mean score for the remaining children in the Experimental Group was 3.4. We conclude that the 20-minute interactive video improved the children’s turn-taking skills.

What is the design type? Is there a problem?

A

Random Assignment

Post test

mortality - loss of the kids who were disruptive- systematically losing kids from one group then the others

history - one group had a more disruptive atmosphere

41
Q

Case Example:

  • Andy swore a lot, a bad habit he hoped to kick. His wife, Betty suggested that Andy record every time that he used profanity during the day by transferring a bead from one pocket to another, and counting the beads transferred at the end of the day.
  • “Do this for the first week,” she said, “and we’ll see how often you typically swear. Then, the next week continue counting, but we’ll add a new twist: I will only cook dinner on days when you have swore 50% less than your average daily count in the first week.”
A

single subject design - kind of pre/post experiment

pre-test, treatment

instrumentation- measuring himself

self fulfilling prophecy

experimenter bias

42
Q

Writing research methods:

PARTICIPANTS

A

• Who was enrolled in your experiment
• Include major demographics that have an impact on the results of the experiment (i.e. if race is a factor, provide a breakdown by race).
• The accepted term for describing a person who participates in research studies is
a participant not a subject; When talking about a design, usually still say “within-subjects” or “between-subjects.”

43
Q

Writing research methods:

Equipment

A
  • AKA apparatus
  • Any specialized equipment used for data collection (Eye trackers, Audio systems)
  • Again, think about what information someone would need to replicate the study in a new lab
44
Q

Writing research methods:

Materials

A
  • Can include scripts, surveys, or software used for data collection
  • Type and number of stimuli (e.g., 200 NU-6 words), how they were processed (e.g., low pass filtered at 4000 Hz), normed, counterbalanced
  • May want to provide specific examples of materials or prompts, depending on your study
45
Q

Writing research methods:

Procedure

A
  • How to run the experiment
  • A description of the experimental design and how participants were assigned conditions
  • Identify IV, DV, control variables. Give variables clear, meaningful names
  • Provide the key instructions to participants
  • A step-by-step listing in chronological order of what participants did (Sometimes a picture is worth a 1000 words)
46
Q

Writing research methods:

Analyses

A
  • Any info critical for someone being able to conduct the same analysis as you did (but not the Results yet)
  • Data cleaning/pre-processing steps (If you removed certain trials or participants from analyses, justify why and report how many)
  • The type of analysis (e.g., 2x2 ANOVA with levels…), whether you will test for interactions (If you’re doing something non-standard, be sure to cite the literature to help you justify this approach)
  • Statistical software, including version number

HESP724 research design (8 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions