Week 2 Flashcards
Types of Experimental Designs
What are the four main types of experimental designs?
Between-subjects, within-subjects, mixed factorial, quasi-experimental
What makes it a Between-Subjects Experimental Design?
- Manipulate 1+ factors (IVs) to observe their effects on 1+ DVs
- Respondents are randomly assigned to one and only one group
- At least 50 participants per group
- Groups are orthogonal (equal)
2(+) factor between subject design let’s us look at ______
- main effects of each IV separately on DV
- examine the interaction between the two IV’s
- look at different combos of the IV’s on the same DV
If there is a 2 x 3 factors, how many groups are there?
6 (2*3)
a 2 x 4 has how many levels in each?
2 levels x 4 levels
True or False: The more factors you have, the better
False: the more factors you have and the more levels within each factor, the more complex analyses get
What is a 2 x 2 x 2 design?
3 factor design
Advantages Between-Subjects Designs
- simple to execute
- more conservative than within-subject designs
- respondents are less likely to suffer from survey fatigue
Disadvantages Between-Subjects Designs
- respondents give their answer in a “vacuum” with no natural anchor/comparison points
- less power than within-subjects (only 1 data point per person)
- potentially harder to get significant effects b/c differences in stimuli may be less impactful when they are presented independently
Between-subjects allows for judgments in ___-
isolation
Within-subjects allows for judgments _____
relative, contextual comparisons among multiple alternatives
What does it mean to have repeated measures?
Participants are assigned to more than 1 level of the same factor/IV (i.e. more than 1 group)
How is causality inferred in within-subject designs?
Causality is inferred by changes in the repeated measure(s)
One factor vs. two factor within-subject designs
one factor only asks 1 repeated measures, two factor has two different factors involved
True or False: within-subject designs are good for pretests?
True
Advantages of Within-Subject Designs
- gives you more power with less respondents
- better controls for error variance than between-subjects (exact same people are answering questions)
- allows for direct, relative comparisons between 2 or more stimuli (more consistent with real world)
- encourages respondents to use the provided stimuli as anchors/basis for comparisons (rather than something random we can’t control)
Disadvantages of Within-Subject Designs
Carryover and demand effects - participation in one task/stimuli may affect participation in another
- practice: respondents perform better in 2nd task because of familiarity
- fatigue: respondents perform worse in 2nd task because of survey fatigue
respondents may heel compelled to give different answers to different stimuli
less conservative than between-subjects
Two things to remember to do/is good to do in within-subject designs?
Randomize the order that the levels of the repeated measures are shown and asked about
put in a “filler” task between the repeated levels
Mixed Factorial Design
Uses both between-subject and within-subject designs in one overall design.
At least one factor is between-subject (i.e., respondents are randomly assigned to only 1 level of this factor) and at least one factor is within-subject (i.e., everyone sees all levels of this factor)
What type of design:
Respondents look at 9 bags of chips on a retail shelf (3 are pre-determined by researchers to be healthy, 3 are moderately healthy, and 3 are unhealthy). Each respondent indicates his/her PI for 1 healthy, 1 moderately healthy, and 1 unhealthy item that are prechosen by researchers.
Within-Subjects Design
What type of design:
Respondents look at 9 bags of chips on a retail shelf (3 are pre-determined by researchers to be healthy, 3 are moderately healthy, and 3 are unhealthy). Each respondent indicates his/her PI for 1 healthy, 1 moderately healthy, and 1 unhealthy item that are prechosen by researchers. We randomly assign half of the respondents to be exposed to a “organic” health claim on the front of the packages and the other half of respondents don’t see any claims at all (i.e., a control condition).
Mixed Factorial Design
Quasi Experimental Design Defining Features
Takes place in field setting where we observe a DV before and after a change in IV (which occurs either naturally or from a manipulation)
Lack of random assignment.
Example of why random assignment not always feasible/practical or ethical?
field studies; when using secondary data
studying child abuse
Where does internal validity fall in quasi-experiments?
falls somewhere b/w correlational studies and true experiments.
Is it difficult to infer causality with quasi-experiments?
Yes, results should be interpreted with caution
Advantages of Quasi-Experimental Designs
- High external validity (generalizability)
- keeps respondents from being aware of the treatments/manipulations in other groups
Disadvantages of Quasi-Experimental Designs
- lacks internal validity
- causal inferences/conclusions should be drawn since we cannot rule out alternative explanations
Non-Equivalent Groups Design
Researchers choose 2 or more existing groups in the “real world” that appear as similar as possible (ex: two similar communities, schools, stores, consumer groups).
But since people aren’t randomly assigned to the groups, the groups likely actually differ in some ways (i.e., are non-equivalent).
It’s a between-subject design w/ no random assignment
What type of design:
We want to know if grades are higher in online classes vs. in-person classes. Students sign up for 1 of 2 marketing principles sections (not knowing what the format will be). Then, 1 section is taught online and the other is taught in person. At the end of the semester, we compare the average GPA’s of each section.
Non-Equivalent Groups Design
Pretest - Posttest Design
The same DV is measured among the same group of people (one group) before (i.e., the pretest) and after (i.e., the posttest) a treatment/manipulation is administered.
Then we compare the averaged DV before and after the manipulation (i.e., looking for any change in it).
Main problem is that we don’t have another similar group (i.e., a control group) to compare this treatment group to (like we do with a non-equivalent groups design).
What type of design:
You first measure 100 donors’ attitudes toward Ole Miss. Then they all attend a huge banquet in their honor on the Ole Miss campus where they are publically recognized. After, you measure those same 100 donors’ attitudes toward Ole Miss (using the same 1 to 7 scale).
Then you compare average attitudes before and after.
Pretest-posttest design
Interrupted Time Series Design (compared to pretest, posttest)
A variant of the pretest-posttest design. Similar to pretest-posttest in that we measure a DV both before and after a naturally occurring treatment (e.g., new smoking ban on campus; new mandatory food label).
Different from pretest-posttest in that it includes multiple, equally-spaced pretest and posttest DV measurements (e.g., daily, weekly, monthly, yearly).
How is pretest-posttest similar to a within-subject design
each person first gives a response under a control condition (pretest) and then under a treatment condition (posttest).
How is pretest-posttest different to a within-subject design
the order of the conditions aren’t counterbalanced (i.e., everyone gets the control condition first, then the treatment condition).
Interrupted Time Series Design (defining measures/qualities)
compares the trend in our DV over time before treatment to the trend in that same DV over time after the treatment.
The intervention time frame needs to be clearly defined and not included in either the pre or post periods.
Arguably the best quasi-experimental design in terms of causal inference!
What type of design:
The city of Oxford increases the sales tax rate to 12% on January 1. You want to know if/how that affected sales of a retail store in Oxford. You record the overall sales in dollars of a store in Oxford for each month for the immediate 12 months leading up to the sales tax hike (so 12 measurements). Then, you record the overall sales in dollars of the store for each of the immediate 12 months following the tax hike (so 12 more measurements). Lastly, you compare the sales before and after the tax hike.
Interrupted Times Series Designs
What type of design:
You want to know if/how hiring a new manager affected sales in a retail store. You record the overall sales in dollars of a store each Sunday for two months immediately leading up to the new hire (so 8 measurements). Then, you record the overall sales each Sunday for the two months immediately after the new hire (so 8 more measurements). Lastly, you compare the sales before and after the new hire.
Basic Time Series Design
Types of Quasi-Experimental Designs
Non-Equivalent Groups Design
Pretest – Posttest Design
Basic and Interrupted Time Series Designs
When do you have a cross-sectional study?
Regardless of design, if you don’t manipulate ANY IV’s– and instead use ONLY measured IV’s – then you aren’t running an experiment. You have a cross-sectional study!
When can you NOT infer causality?
When there is not at least 1 manipulated variable (and random assignment wasn’t possible)
What type of design:
Respondents were presented with 3 different nutrition package icons (one at a time). They answered the same set of questions about each (e.g., how specific each icon is, how helpful each icon is, etc.).
One factor within-subjects design
What type of design:
You have a 2 (device type: phone vs. tablet) x 2 (shopping list: short vs. long) within-subjects design.
Each respondent is tasked with finding products on a shopping list and adding them to their basket on a certain website. Each respondent fulfills both the short and long list and does so on both a phone and a tablet.
Two factor within subjects design
What type of design:
You have a 2 (hunger level: more hungry vs. less hungry) x 3 (healthfulness level: healthy vs. moderate vs. unhealthy) within-subjects design.
Each respondent indicates his/her PI for the healthy, moderate, and unhealthy items when more hungry. Then each respondent indicates his/her PI for each item when he/she is less hungry.
Two factor within subjects design
What type of design:
Every respondent looks at a group of 9 bags of chips on a retail shelf (3 are pre-determined by researchers to be healthy, 3 are moderately healthy, and 3 are unhealthy). The only difference between the chips are the healthfulness of them.
Each respondent then indicates his/her purchase intent for 1 healthy, 1 moderately healthy, and 1 unhealthy item prechosen by researchers (on the same 1 to 7 scale).
One factor within subjects design
What type of design:
Every respondent gives their attitude toward 4 different potential packages for a new brand of cereal that differ only on color
One factor within subjects design
