Lecture 6: Experimental Research Flashcards
What is experimental research?
A research methodology where the researcher seeks to observe how changes in one or more variables impact on other variables.
It allows investigation of causal relationships.
Requires high levels of control.
What is an independent variable (IV)?
The variable that is manipulated (the assumed ‘cause’)
What is the dependent variable (DV)?
The outcome variable that is measured (the assumed ‘effect’)
What are extraneous variables?
Any other variables that impact on the relationship (e.g. ones that could weaken or distort results)
What are confounding variables (or confounds)?
A type of extraneous variable that systematically varies with the independent variable (i.e. one which might otherwise be responsible for the changes in the dependent variable)
What are controls?
When researchers aim to control or eliminate extraneous variables in the research design
What are some design issues?
Randomisation
- random assignment of participants and treatments to groups
- scatters extraneous variables between conditions
Matching
* assigning participants to conditions based on pertinent attributes
What are laboratory studies?
Experiment conducted in an artificial setting (such as a testing laboratory)
Researcher has much greater control over extraneous variables
What are field studies?
Experiment conducted in a natural setting
- difficult to control extraneous variables
- natural vs. formed conditions
What is a between-subjects design?
When comparisons are made between different groups of people
What is a within-subjects design?
When comparisons are made using the same group of people (e.g. with pre- and post- designs)
What is internal validity?
Whether variation in the DV can be confidently attributed to the variation in the IV
What is external validity?
The extent to which the results of can generalise beyond the current study.
- Is it likely that this group will also be found in non-experimental settings, with other groups, etc?
- Wider population?
What are the threats to internal validity? (7)
- Maturation effects
- Systematic changes in the participant across time points or within the course of a study
- Long-term maturation?
- Short-term maturation?
- History effects
- Where the participant’s environment has changed across studies or within the course of a study
- For example, brand crisis midway through longitudinal study
- Testing
- When an initial measure (e.g., pre-test) sensitises participants to the nature of the experiment
- Instrumentation
- Where changes in the measurement instrument change between test and re-test
- Selection
- Where participants are allocated to conditions in some systematic way
- Mortality (a.k.a. Attrition)
- Refers to participants dropping out of the study
- Problematic if there is systematic attrition
- Consider testing of social marketing programs on topics like safe sex, reduced speeding, quitting smoking
- Demand effects
- Where aspects of the experiment that provide cues to participants about researcher’s purpose, which are then acted upon by participants
- Hawthorne effect - unintended effects cause by participants behaving differently
What are maturation effects?
A threat to internal validity
- Systematic changes in the participant across time points or within the course of a study
- Long-term maturation?
- Short-term maturation?
What are history effects?
A threat to internal validity
Where the participant’s environment has changed across studies or within the course of a study
- For example, brand crisis midway through longitudinal study
What is testing (regarding threats to internal validity)?
When an initial measure (e.g., pre-test) sensitises participants to the nature of the experiment
What is instrumentation (regarding threats to internal validity)?
Where changes in the measurement instrument change between test and re-test
What is selection (regarding threats to internal validity)?
Where changes in the measurement instrument change between test and re-test
What is mortality (aka attrition)?
A threat to internal validity?
Refers to participants dropping out of the study
Problematic if there is systematic attrition
Consider testing of social marketing programs on topics like safe sex, reduced speeding, quitting smoking
What are demand effects (regarding internal validity)?
Where aspects of the experiment that provide cues to participants about reseacher’s purpose which are then acted upon by participants
How can internal validity be improved? (3)
- Control group - no treatment or filler treatment
- Blinded and double-blinded experiments
- Random assignment to conditions
What are 3 threats to external validity?
- Non-representative samples - e.g. reliance on convenience samples
- Artificial stimuli - stimuli that suit the experiment, but do not represent full range of real-life stimuli. Ecological validity?
- Artificial settings - testing in laboratory setting that does not correspond sufficiently with real-life setting
What does X mean?
Exposure to IV (or a level of the IV)
What does O mean?
Observation or measurement of DV
What does R mean?
Random assignment of units (e.g. people to conditions)
What is the one-shot case study (Quasi-experimental designs)?
A single group is exposed to some IV, and then observations are made (DV)
E.g. government announces an education policy and constituents’ voting intentions are assessed
Difficult to determine if IV has had any impact, or if observed results can attribute to the IV
What is the static group comparison (Quasi-experimental designs)?
Experimental group vs control group
Two groups are exposed to different levels of an IV (e.g. IV present, IV absent) and observations are made
Problems of exposure to pre-test is eliminated
Problems relate to lack of group equivalence, selection bias, etc
What is the one group, pre-test - post-test design (Quasi-experimental designs)?
A single group is first observed, then exposed to some IV, and then observations are made again
A pre-test baseline allows for comparison of post-test
Problems relate to maturation, history, etc
What is the time series design (Quasi-experimental designs)?
A single group is pre-tested multiple times, exposed to some IV, and is then post-tested multiple times
Reduces potential problems associated with random fluctuations in performance, and helps to demonstrate how effective the IV is in impacting the IV long-term
Problems relate to history and maturation effects, testing effects, etc
Can be costly
What is the post-test only, control group design (true experimental design)?
Randomisation
Two groups are randomly chosen, exposed to different levels of an IV (e.g. IV present, IV absent) and observations are made
Random assignment eliminates some problems associated with groups not initially being similar
What is the pre-test - post-test control group design?
Two groups are randomly chosen, observations are made for each group, they each receive exposure to different levels of the IV (e.g. IV present, IV absent) and observations are made again
Experimental group can be compared to control group, and also back to baseline
What is the Solomon four-group design?
Allows comparison of first two groups as with pre- and post-test design but addresses problems of history, maturation and testing with the provision of the two groups that do not have the pre-test
Challenges relate to having sufficient resources to execute
What are factorial designs?
The manipulation of two or more IVs
Sometimes effect differs across levels of one IV, when other IVs are taken into account
Other factors are also likely involves, that we would be interested in
The effect of one variable is known as what?
The main effect (e.g. main effect of gender or location)
An effect due to a combination of variables is known as what?
An interaction effect
- there is an interaction between gender and location
- differences in the two levels of gender are not the same across each level of location
What are test markets?
A controlled experimental procedure used by companies to trial a new product, promotion, strategy, etc. under realistic conditions with a limited audience. It’s a form of field experiment.
What are the types of test markets? (4)
- Standard test marketing
- For example, company restricts distribution to limited geographic area or stores
- High external validity
- Control method of test marketing
- For example, use of forced distribution to particular stores to ensure constantly in stock (e.g., paid shelf space)
- Online test marketing
- Panel used to test copy or respond to new product promotions
- Virtual reality test marketing
- Test market within virtual reality environment (e.g., Second Life)
What are the limitations of test markets (4)?
Takes a long time to properly evaluate performance - too short can over-estimate sales (trial only purchases)
Loss of secrecy to competitors
Competitor tactics (e.g. distortion of results, imitation)
May not yield representative results if not implemented effectively (over-zealous salespeople, regularly used test markets)
