Group Design Flashcards
6 steps in conducting an experiment
- formulate hypothesis
- select variables
- limit alternative explanations
manipulate IV and measure DV - analyze variation in DV
- draw inferences about relationship between IV and DV
select setting
field or lab
select type of approach
longitudinal or cross sectional
equivalence in groups
making sure all confounding are considered and groups are equivalent
how do we ensure group equivalence
- assign participants independently and randomly to conditions (levels of IV)
dependent assignment
If participant 1 went into group A, participant 2 went into group B (might make sense if you want to avoid the influence of the preexisting relationships between participants)
who is good for studying nature vs. nurture
Twin studies are good for researching nature vs. nurture, don’t assign participants randomly or twins may end up in same group
independent design
Assign participants independently and randomly to conditions (levels) of the IV
completely randomized group design with 2 levels
- The simplest type of the independent random design is completely randomized design: participants are randomly assigned to 2 different levels of one independent variable.
- 2 variable so can use t-test
if there is more than 2 conditions being testes the correct statistical test is
ANOVA ANalysis of VAriance
2x2 design has _____ conditions
4
if we do not see main effect ad only see effect in particular condition(s)
we cannot interpret main effects without stating the interaction differences
2 major types of dependent design
matched group design
within participant / repeated measure design
matchced group design
- We assign participants non-randomly into groups are matched (equalized) at least on one matching variable
- You select two participants with a similar performance /ability /characteristic and assign one participant to condition 1 and another to condition 2. The variable that the participants perform similarly on is called the matched variable.
- powerful technique since matching reduces variability between participants, and yet you don’t have to worry about carryover effects or differential order effects since one participant does not do all the conditions but only one.
- is crucial that the matched variable is related to (correlated with) the dependent variable.
within participant or repeated measure design
- The same participants are used in all treatment conditions. Participants are equivalent to themselves
- Design is economical
dependent design
- Dependent designs can have one or more IVs, just like independent designs
- Often time serves as one of the IVs, e.g., if the same group of participants is measured at different occasions
factorial dependent experiment
all combinations of all levels of all IVs are present in such an experiment, we call it
carry over effects
Since a participant sees all experimental items (stimuli), we should be concerned with the effects of items they see early on in their performance later in the experiment: carryover, order, transfer effects. Fatigue and practice are common examples of carryover effects
complete counterbalancing
each condition occurs equally often and precedes and follows all other conditions the same number of times. It works beautifully but requires a large number of participants
differential order effects
- example of confounding variable
- if some conditions change the behaviour of participants in an irreversible way, counterbalancing won’t help with this, as this is not about practice or fatigue.
mixed designs
some variables are assessed within participants and some between participants. If carryover is expected for a variable, it can be treated as a between-participants, others - where no such problem is expected - as within-participants
ANOVA DV and IV
- The dependent (Y) variable is the interval / ratio numeric variable
- But the independent (X) variable is is categorical (nominal / ordinal)
how does ANOVA work
analyzed variance between and within each sample
total sum of squares
SSt = SSw + SSb
if samples have difference means the variance between them will be _____
larger than the variance within samples
calculating the ratio of variance we use a
F-test
The ratio of variances follows the F-distribution with two values of degrees of freedom
total number of data points across samples, minus one; and number of samples minus 1
ANOVA tells us
us is that there is a significance difference between groups. It does not tell us where it is
t-test i excel
= Z-TEST (range)
how is correlation computed
graphically represented by the regression line
regression line
the line that maximizes the sum of residuals
residuals
distances from the observed values values to the line
if r (correlation coefficient) is closer to 1
stronger correlation
small r
0.1
medium r
0.3
large r
0,5
correlation only measure ______) relationships
linear
can you meaningfully use z-scores in non- normal distributions
no
spearman’s p
ranks one variable then the other, for each pair compute the difference of ranks d and square it
spearman’s p is maximal when
- X consistently increases with the increase/decrease in Y.
- Unlike Pearson’s r, it does not matter how strong the increase in X is, only how systematic it is
survey
looks into attitudes and opinions
face-to-face interviews
• Consider the location you are at: at the airport you generally do not see impoverished people, and you do see a larger percent of people who are on the road due to their occupation (e.g., businesspeople).
disadvantages of face-to-face interviews
- sampling procedure
- interviewer’ bias
- social desirability and other types of reactivity (imagine asking high-school students whether they smoked marijuana).
- Costs
phone interview
- Usually structured interviews: a prepared list of questions, or a script.
- Must be shorter than face-to-face, no more than 10 minutes.
- Advantages: relative anonymity, better sampling.
disadvantage to phone interview
selection bias
self administered questionnaire
- A responder reads and responds to a SAQ, no contact with the researcher.
- Cheap and fast: often a tool of preference.
disadvantage to self advminstered questionnaire
responders must follow the instructions and be literate; researchers must do a very good job writing up the questionnaire
group administered questionnaires
- The questionnaire is distributed among people who naturally form a group: a class in school, a society, a community, etc..
- Better targeting of populations you are interested in
internet questionnaires
- The cheapest and fastest way of collecting information.
- Good targeting of populations of interest: for data on opinions about animal rights you can post your survey on a website defending these rights.
disadvantages to internet questionnaires
Problems: multiple submissions; self-selection
program evaluation
- a common task that researchers face is an evaluation of the efficiency of large-scale social programs
- Researchers assess whether the program
Has met its goals
Is really needed
Is cost efficient
Is well run
Can be improved and how
how is program evaluation done
- (Re-)defining goals in measurable terms. Selecting outcome variables: the more objective the measures, the better. Important to reduce bias.
- Checking the deliverables of a program against its advertise goals.
- User satisfaction is often measured as well as indices of improvement in the life of targeted individuals
archival research
- Time-series data rely on information documented before an event and after the event. Researchers retreat to existing and available collections of older data.
- For linguists, corpora are an example of archival data.
advantages of archival research
- Huge information base.
- No problem with participants’ reactivity.
- Experimenter’s bias is important though, and needs to be reduced. Might reflect in what data is chosen and how.
meta-analysis
- Review all published studies on a topic.
- Compare effects and effect sizes over multiple studies.
- The idea is to compute the effect size combined over experimental and correlational studies of the same underlying variable
PIAAC
- Program for the International Assessment of Adult Competencies
- ages 16-65
- multi-stage sampling
- combined with quota
- can handle 2 IV
does meta-analysis have high or low statistical power
high
benefit of meta-analysis
useful for summarizing the results of existing studies and for working out recommendations for further research
