Experimental Design Flashcards
What a should a sample be?
A representative sub-set of the concrete (study) population.
What is the problem with using WEIRD participants in studies?
Some basic and complex processes differ in people from different backgrounds, so inference from a study is limited.
Define sampling bias.
When a sample is not representative due to a systematic aspect of the sampling method.
Define sample aliasing.
When a sample is not representative due to the interval/point in the data that was selected.
Give 2 reasons self-selected/volunteer sampling could create a bias.
People may go for the payment and people may go out of interest (WEIRD).
Give a reason snowball sampling could create a bias.
It entirely depends on the first contact and who they represent.
Give 2 strengths of snowball sampling.
Good for initial ideas and effective for sensitive information/populations.
Give a reason for and against cluster sampling over simple random sampling.
For: less expensive.
Against: more error.
Describe systematic stratified sampling.
Lists set up using stratification, order in each list randomised, systematically choose entries in each list.
In what type of studies is non-random sampling most often used?
Experimental.
Define internal validity.
Can the statistical and/or causal conclusions be believed?
Define external validity.
Can the conclusions be generalised?
Why are internal and external validity sometimes in opposition?
Because internal validity required tighter control but this can decrease external validity.
What are secondary dependent variables?
Simple secondary measures that may be affected by/as well as the initial DV.
Give 2 ways you can maximise the chance of detecting effects of IV.
Setting high/low values of IV (but not too extreme) and using the minimum number of values for each IV.
Describe error about a participants true value.
The trial or mean of trials may not he representative of the participant.
Vaguely, what are the 4 things we observe on each trial of an experiment?
Population mean, participant noise, trial noise and other factors.
How do we reduce participant noise?
By increasing n participants.
How do we reduce trial noise?
By increasing n trials each participant does.
How can we calculate how much data we need and why should we do this?
We can use “power analysis” because we should only get enough data to answer our questions and no more and because collecting data costs time, effort and money.
When should we account for blocked control variables?
Only when we are sure that they matter (to avoid reducing chance of detecting effects needlessly).
Give the term for changes in performance because someone is aware they are participating in a study.
Reactivity.
Briefly describe the Neyman-Pearson lemma.
One of the null and alternate hypotheses must be true.
