WEEK 3 - Design and Controls Flashcards
What is internal validity?
How strongly can we assure that changes in our DV are due to the changes in our IV and not something else we haven’t controlled for (extraneous variable)
What is external validity?
How generalisable are our findings? (tied in with the representativeness of our sample)
How representative of the real world is our study (tied in with how artificial our study is)
What is the relationship between internal and external validity
The more stringently we try to control internal validity the potentially more artificial our study becomes and hence less representative of reality and therefore less generalisable and therefore less externally valid
What are the four steps to internal validity?
- Sound operationalisation of our DV (measure should be reliable and valid)
- Strong design logic
- Sound operationalisation of our IV
- Consideration and use of appropriate remedies to control for extraneous variables
What are the three types of research designs?
- Experimental - cause and effect, manipulation, control
- Quasi-experimental - Similar to experimental designs, however less randomisation of key independent variables.
- Non-experimental - Relationships but NOT Cause and Effect (quantitive or qualitative)
What is an experiment
An experiment is strictly controlled study in which the ultimate aim is to infer causality on the part of the IV and DV
In other words, in order to say that changes in our IV CAUSED changes in our DV we need to make sure that any and (hopefully) all (if not most) alternative explanations have been accounted for
Experimental Vs Quasi-Experimental
Experimental design involves experimental manipulation directly determined by researcher in controlled environment
Quasi-Experimental: where manipulation not controlled by researcher e.g., where levels in IV determined by participant characteristic i.e. individual difference manipulation e.g. demographics, self report measures.
What are the two ways an IV can be manipulated?
Experimental manipulation
Individual difference manipulation
What is experimental manipulation?
Experimenter determines which level of the IV a participant is tested at:
- event manipulation
- instructer manipulation
What is individual difference manipulation?
A characteristic of the participant determines the level of the IV of which they are tested (not strictly an experiment - quasi-experiment)
- demographics
- self-reported measures
What are the types of experimental research designs?
Repeated measures (within groups, dependent group) - each participant tested at each level of the IV
Between groups (intended groups) - each participant only tested at one level of the IV
Mixed: more than one IV with at least one IV manipulated between groups and at least one within groups.
What are repeated measure design’s
- each participant tested at each level of the IV
- Need less subjects
- More sensitive design (easier to detect the effect of interest, as individual differences controlled for)
- Cant always use this design
What are between groups designs
- Each participant tested only at one level of the IV
- less sensitive design
- Often forced to use this design if:
–> If the IV is an individual difference variable (eg. gender)
–> if participating in one condition precludes participating in another
What are mixed designs?
- more than one IV with at least one IV manipulated between groups and at least one within groups
When can you not use repeated measure designs?
- If participants at one level of the IV precludes later participating at another level, for example by causing permanent change in the participant e.g. exposure to one therapy may cause permanent improvement
- It is not physically possible for a participant to participate in all levels e.g. cant be both short and tall
When can you not use a between group design?
Cant use a between-group design if you wish to detect changes in individuals across time example- learning studies or developmental studies
What is a factorial design?
- A design with more than one IV
- May have all repeated measures IV’s or all between group IV’s
- Mixed: more than one IV with at least one IV manipulated between groups and at least one within groups.
- Allows examination of interplay between two or more IVs and the splitting up of these effects into interactions and the main effects
What are strengths of a factorial design?
- more than one IV allows for a more precise hypothesis
- Control for extraneous variables by including it as an independent variable
- ability to determine the interactive effect of two or more variables
What are factorial designs main effects?
- The influence of one independent variable on the main variable
- One main effect for each IV in a study
example: can look at the main effect of age and the main effect of alcohol
What are factorial designs interactive effects?
- looks at whether the effect of one IV is different at different levels of another IV
What is the factorial design notation?
number of numerals = number of IV’s
each number indicates the number of levels for each IV
for example:
2x2 design
IV1 = 2 levels
IV2 = 2 levels
2x 3 design
IV1= 2 levels
IV2 = 3 levels
What are weaknesses of factorial designs?
- using more than two independent variables may be logistically cumbersome
- examples
- 2 x 2 design = 4 cells, 2 main effects, and 1 interaction
- 2 x 3 design = 6 cells, 2 main effects, and 1 interaction
- 2 x 2 x 3 design = 12 cells, 3 main effects, and 4 interactions
- higher-order interactions are difficult to interpret
In order to maximise our chances of getting a true picture of how our independent variable affects our dependent variable we want to…
(separate and compress)
- Maximise the impact on our dependent variable that is related to the independent variable
–> increase between group/condition/level variation
-Minimise variation in our dependent variable that is not related to our independent variable
–> compress within group/condition variation
What is seperate?
maximise the variation between groups/levels of the IV
What is compress?
minimise the variation within groups/levels of the IV
How do you determine levels of the IV?
When considering the operationalisation of an IV you need to consider how to ensure you include as extreme or distinctly separate levels of your IV as possible
If you are looking at the impact of alcohol on performance you would make sure your conditions/levels differ by enough micrograms of alcohol to have a discernible impact
You might choose to include a number of intermediate levels of an increasing intensity IV to help pick up complex patters like linear or curvilinear impacts if you believe they may exist
How is compression achieved?
by reducing error variance
What are the three important sources when aiming to compress a study
- Measurement error
- Individual differences (people differ more within the groups rather than between)
- Other factors that influence peoples scores
What are the two forms of extraneous variables?
Noise creating (nuisance)
confounding
What are nuisance variables?
randomly impact the DV, not related to the IV, but potentially create extra variation in the DV not due to the IV, reducing power
What are confounding?
- systematically impact the DV, related to the IV, potentially
explaining changes in the DV that you would be expecting the IV to make, - want to control for this by eliminating, keeping constant or building into study so can measure impact.
- Confounds reduce internal validity.
- Need to minimize in all forms of research (especially experimental).
What are the troubles with between group desings?
The trouble with Between Groups designs is precisely the fact that they are between groups. Two separate groups of people could differ on a whole range of things. Both relevant and irrelevant to the study at hand
What is selection?
The process of assigning people to experimental conditions.
Types:
self assignment
experimenter assignment
Arbitrary assignment
Random assignment
What is self assignment (in selection)
Subject selects which treatment group
What is experimenter assignment (in selection)
Experimenter selects which treatment group
What is arbitrary assignment (in selection)
- Selection based on seemingly non-relevant criteria
- All above have potential for bias to confound results
