W8 Flashcards
Advantages of experiments
- allow to determine which variables have an authentically causal design
- identify variables that have significant causal relationships
- identify variables that have no significant effect on other variable
Disadvantages of experiments
- artificiality of experimental conditions (lab setting - not real life scenario) –> lack of ecological isomorphism
- more sophisticated designs may require large number of people who are willing to become experimental participants, maybe for extended periods of time
Lack of ecological isomorphism
conditions in lab as not the same as the ones in the outside world, meaning the study is not as replicable
Ex-post facto designs
no manipulation of the IV, but still the relationship between variables and potential causes are studied
research analyzes the effects of an IV that has occurred naturally (e.g. gender, level of education) on a DV (e.g. academic performance, communication style)
Field experiments
expose participants to different conditions to compare the effects of the manipulation on the IV (diff conditions)
e.g.
condition 1: studying in a group
condition 2: studying individually
observation: test performance
Ex-post facto vs field experiments
ex-post facto designs and field experiments may not be as reliable as the researcher has no idea how individuals in the two groups differ outside of the conditions they were assigned to
it may be that some other circumstance other than their condition explains the difference in results.
Basic experimental design
x= manipulation of variable (conditions)
r = random assignment of individuals to groups
O1 O2 etc = observation 1, observation 2, etc.
One group pretest-posttest design
O1 X O2
to be certain that a causal relationship is found rule out two possibilities:
1) O2 may have occurred anyway for some reason (snot necessarily due to O1)
2) some influence other than the study conditions caused the change
rule out all other possible explanations before deciding that only the study conditions are explaining the difference in the observations
Designing for control
CV - not exposed to any change
control - remove all possible variables from experimental design to ensure only the IV is causing the changes
Two group pretest-posttest design
O1 X O2
O1. O2
both groups are measured before and after one group experiences an experimental condition, the other group is the control group
Designing for random assignment
allows to assume the probability of something occurring in one group is no greater or less than the probability of it occurring in another group
Random assignment for 2 group pretest-posttest design
R O1 X O2
R O1. O2
Solomon four-group design
1) compare pretest with posttest
2) compare control groups with experimental groups
3) also takes a look at a group to which nothing has happened except for a final test
Factorial designs
analysis that examine the relationship among 3 or more variables - multivariate analysis
experimental designs that manipulate two or more variables - factorial designs
Between subjects design
diff groups of participants are exposed to different conditions or levels of the IV, each participant experiences only one condition
Within subjects design
same participants are exposed to all conditions of the IV, every participant experiences every level or treatment
Time series analysis
when we cannot know if the results obtained at the end of the experiment will still be true at some point in the future, problem
Internal validity - spurious relationship
not a genuine relationship between two variables but one that only seems to exist
Threats to internal validity - selection bias
occurs when the sample used in the study is not representative of the population being studied
Threats to internal validity - attrition
participants get overstimulated from the study and decide to drop out - boredom
Threats to internal validity - repeated testing
participants become due to repetitions, more and more familiar with the study and might develop certain expectations on what is it about or the aim of the study
Threats to internal validity - maturation
people change over time and if there is a long period of time between experiments the results may become susceptible to maturation.
Threats to internal validity - diffusion
when the condition slightly spreads into the control group making the whole experiment less reliable
Threats to external validity - ecological isomorphism
conditions in the lab are not the same as conditions in the real world
Threats to external validity - hawthorne effect
when participants change their behavior because they know they are being observed
Internal validity
is the effect you find actually caused by your independent variable or by something else?
the certainty with which you can find causal relationships in your research
e.g. adolescents feel lonely the more often they use instagram (make sure loneliness is only being caused by instagram)
Causality principals (internal validity)
1) co variance - y must change as x changes
2) chronology - x must occur before y
3) non spuriousness - nothing except x must be influencing y
if you can control the circumstances in which the participants are being researched, you can rule out alternative explanations - make sure no other factors are affecting your results = high internal validity
External validity
Extent to which results can be generalized to a context outside than the study itself
To what extent do the results of your research say something about the population you are interested in?
- we need a sample and circumstances for our research to allow us to make general statements
Population validity (external)
results generalized to population to which you want to make statements
for high population validity, do not exclude any groups from your population
e.g. freq of exposure to election campaign - intention to vote
sample: high educated students
validity: low (bc sample does not represent the entire population)
Ecological validity (external )
results generalized to real conditions / different circumstances
research and results should represent “real life” situation, be as real as possible
True experiments
experimental designs - maximize internal validity (aka randomized control trials / true experiments)
manipulation: cause -> effect, cause is manipulated by researcher to have an effect
comparison: cause -> effect, ensures effect does not occur naturally
random assignment: random assignment of population into different groups, equal distribution, no systematic difference between the groups, if it fails replicate the study to ensure results are consistent
Manipulation
control over the IV
the level indicated by independent variable is determined by researcher
control over external variables - rule out alternative explanations
experimental variable: IV fully controlled by researcher
individual difference variables: variables that cant be controlled by researcher (age, gender,etc. )
seemingly non manipulative variables that can be manipulated: e.g. self esteem
manipulation check: important to perform in order to. make sure you are not giving away (spoiling) the purpose of the experiment
control of variables of desinterest: ideally the only difference in the experiment is the IV (Ceteris paribus principle = ALL OTHER THINGS ARE EQUAL)
randomization
eliminates all systematic differences between participants in different conditions
Advantages of surveys
- respondents can answer large numbers of questions rapidly
- efficient as large numbers of people can answer surveys rapidly
- you can make generalizations to large populations with confidence
Disadvantages of surveys
- question formats, limited response options such as yes or no
- do not allow to asses causal relationships
having to decide whether or not the responses you received are valid
Cross sectional surveys
collects data from a population at a single point in time, it does not gather info about changes over time
longitudinal studies
follow a subject/changes over time
trend studies
measure the same variable over time using different samples from the population at each survey
panel studies
group of individuals is recruited and surveyed over time
cohort studies
track a specific group of people who share a common event
cross lagged surveys
measure both dependent and independent variables at two different time, allowing researchers to infer causality
probability sampling
simple random sample - every single person has the same chance of being selected from a population
you can find possible biases within the simple random sample
under coverage bias
not everyone in the population is included in the sample frame
sampling bias
not every person in the sampling frame is equally likely to be included in the sample, if you fail to make a random sample
non response bias
some respondents may not agree to participate or answer all questions
response bias
maybe some respondents feel that some answers are socially unacceptable, hence, are not true to their responses
