Unit 6 Flashcards
chapter 8, 9, 13
why are experiments better than non experiments?
from experiments we can learn that altering one variable causes another variable to change (manipulate independent variable)
what is a straightforward manipulation?
operationalize independent variables using instructions and stimulus presentations (verbally, written form or video)
ex. in a study they worded job interviews feminine or masculine, it was discovered that women liked feminine and men liked masculine
ex. looking at pictures
what is high mundane realism?
whenever the tasks involved in a study mimic experiences and conditions present in everyday life
ex. looking at job interviews is more realistic to everyday life
what are staged manipulations?
series of events that occur during the experiment to manipulate the independent variable (can involve actors)
deception is often involved in staged manipulations
what are the two main reasons why staged manipulations are employed?
- researcher may be trying to create a certain psychological state in participants such as frustration, anger or temporary lowering of self esteem
- used to simulate situations that occur in the real world
what is a confederate?
introduced as a participant in the experiment but may be used to create a particular social situation or administer the independent variable
(an example of staging)
what is the difference between experimental realism and mundane realism?
the tasks in a study might not resemble real world experiences (low mundane realism) but can still engage participants in a meaningful way producing psychological experiences that are impactful (high experimental realism)
what are 3 considerations when manipulating the independent variable?
strength of the manipulation
cost of the manipulation
manipulation checks
it is a good idea to try to make the manipulation as strong as possible, why?
in order to have the levels of the independent variable maximally different while keeping everything else between the two groups the same (this will increase the chances that your results will reveal an effect of the independent variable on the dependent variable)
what is a manipulation check?
directly measures whether the manipulation of the independent variable was successful inducing the participants
provide evidence for the validity of your manipulation
can include self reports, behavioral measures or a physiological measure
examples of self report techniques or tools?
paper and pencil questionnaire
face to face interview
online questionnaire
when would a self report measure be used?
attitudes about something
intentions to do something
a persons values, self esteem, mood, anxiety, relationship satisfaction, personality traits
what are examples of behavioural techniques or tools?
audio or video recorder
eye tracker
electronic activated recorder
weight scale
still camera
when would behavioural measures be used?
self control (amount of ice cream eaten, length of persistence on boring task)
creativity (number of ideas generated per minute)
reaction time (speed of detecting a flashing light)
facial expression of emotion (coded photographs)
attention (eye tracker, number of hazards avoided in driving simulator)
liking (distance seated apart from someone)
efficacy of a bulimia intervention (weight gained or lost)
memory (number of items recalled)
generosity (amount of money donated)
what are examples of physiological techniques or tools?
GSR
EMG
ECG
EEG
blood analysis
saliva analysis
heart rate
breathing rate
blood pressure
MRI
fMRI
when would physiological measures be used?
stress (sweating from GSR, cortisol in saliva)
genetic marker for mental illness (blood analysis)
physical fitness (heart rate change during exercise)
size of amygdala or damage to hippocampus (MRI)
brain activation when looking at image of romantic partner (fMRI)
what are the two advantages to manipulation checks?
- used in an early pilot study and reveals your manipulation is not effective, you can change the procedures before running the actual experiment
- manipulation check is advantageous if the results show no effect of the independent variable on the dependent variable
what are considerations when measuring variables?
sensitivity
multiple measures
cost of measures
what is sensitivity?
ex. “do you like someone, yes or no” is less sensitive than asking “how much do you like someone on a scale of 1-7”
important for behavioral measures of performance
what is the ceiling effect?
when everyone does well, there wont be much variability in the scores and so the measure lacks sensitivity to detect differences
what is the floor effect?
when all the scores are low
what are multiple measures?
when researchers use multiple measures of the same variable (to avoid carry over effects present the important measure first and the less important ones later or counterbalance the order presenting the measures or rely on complete randomization of order)
what does it mean to set the stage?
prepare consent forms
explain to participants why the experiment is being conducted (might bend the truth a bit and tell them more generally your interest)
what does a good research design mean?
eliminating as many alternative explanations for the results as possible
ex. avoiding confounding variables
what are considerations for ensuring control?
controlling for participant expectations
controlling for experimenter expectations
what does controlling for participant expectations consist of?
demand characteristics
placebo effects
what is a demand characteristic?
any feature of a study that might inform participants of the study’s purpose and consequently affect their behaviour (participants who know the hypothesis tend to act in ways that confirmed it)
what is a way to control for demand characteristics?
use deception to mislead participants of the study
what are filler items?
a set of unrelated items that disguise the dependent measure by using an unobstructive measure
when are demand characteristics eliminated?
when people are not aware that an experiment is taking place or that their behaviour is being observed
in a drug study if the placebo group improves as much as the experimental group, what does that mean?
the improvement observed due to the experimental drug is likely just a placebo effect
what is an example of a balanced placebo design?
theres 4 groups to study nicotine
1. given nicotine and told nicotine
2. given no nicotine and told no nicotine
3. given nicotine and told no nicotine
4. given no nicotine and told nicotine
what is experimenter bias/ experimenter expectancy effect?
when experimenters are aware of the purpose of the study and likely have expectations about how participants should respond, these expectations can bias the results
what are the two potential sources of experimenter bias?
- experimenter might unintentionally treat participants differently depending on what condition they are in
- when experimenters record participants behaviours with subtle differences emerging in how the experimenter interprets and records behaviours for people in different conditions
what are examples of research on expectancy effects?
ex. clever hans, a horse that could do math (really the experimenter was giving the horse subtle cues)
ex. participants were told that the study would cause them to walk fast or slow, this unintentionally influenced participants responses
what are solutions to the expectancy problem?
- run everyone in all conditions simultaneously so that the experimenters behaviour is exactly the same for all participants
- use experimenters who are unaware of the hypothesis (double blind) (single blind is only when participants dont know)
what are two decisions that must be made before applying for ethics approval?
participant selection process
debriefing procedures (explain the ethical and educational implications of the study verbally or written)
what is funnelled debriefing?
questions asked to the participants that begin broadly but then narrow in on the key deception
what are two issues to keep in mind when collecting data?
whether to complete a pilot study
the commitments that researchers have to the participants (contracts with participants)
when is a pilot study conducted?
when procedures are costly or when there will be only a single opportunity to collect data researchers might run a pilot study
what is a pilot study?
trial run with a small number of participants to test out the procedures (must be included in ethics application)
what occurs after data have been collected?
analyze data then write a report that details why you conducted the research (professional conferences, journal articles)i
what are the 8 steps in the experimental design?
- develop a theory
- review the literature
- generate a testable hypothesis
- define your variables
- identify and sample your population
- assign the sample to experimental and control groups
- run the study
- analyze your findings and then report them
what is a directional hypotheses?
specify precisely the outcome of any research project
the results from the population that is exposed to the independent variable (μ1) will be smaller or larger (directional) than the results from the population that is not exposed to the independent variable (μ2)
ex. group Z will score significantly higher than group T
ex. social media will increase grades
if the results from the population that is exposed to the independent variable (μ1) is smaller (directional) than the results from the population that is not exposed to the independent variable (μ2), how is this represented?
Ha: μ1 < μ2
or
H1: μ1>μ2
if the results from the population that is exposed to the independent variable (μ1) is larger (directional) than the results from the population that is not exposed to the independent variable (μ2), how is this represented?
Ha: μ1>μ2
or
H1: μ1>μ2
what is a nondirectional hypothesis?
do not precisely specify the outcome of a research project
ex. group Z scores will differ significantly from the scores of group T
ex. social media will affect grades
if you plan to perform statistical procedures with the data you gather, you must develop statistical hypothesis, what is that?
created by simply restating the research hypotheses into two distinct statistical hypothesis (H1 and H0)
what does the null hypothesis state?
independent variable will have no effect on the dependent variable (μ1=μ2) or no effect in the population (r=0.00)
ex. estrogen will not reduce bone loss
what is the alternative research hypothesis state?
states that independent variable does have some effect on the dependent variable (μ1 ≠ μ2)
ex. estrogen will reduce bone loss
what is the independent variable?
variable that the experimenter has control over (required in order to make a causal inference or cause and effect statement)
what does it mean that as we change the independent variable the dependent variable must change in some predictable fashion as well? example
ex. independent variable (pain medication) on the dependent variable (joint pain)
when we manipulate te independent variable (give more or less meds) pain perception increases or decreases)
what is a population? what is population mean denoted as?
complete group of people or events that are of interest to the researcher
pop mean: μ
what is a sample?
group that is selected to represent the population
what does the simplest possible experimental design have?
two variables: independent (experimental and control group) and a dependent variable
what is an experimental group?
the group of participants that will be exposed to the manipulation of the independent variable
what is the control group?
group that undergoes the exact same experimental procedure as the experimental group except the independent variable is not manipulated
also called placebo or sugar pill group
why is the control group important?
because it helps to ensure that any causal inferences we make regarding our findings are accurate since the control group helps to eliminate all other possible explanations for the observed relationship
what is a confounding variable?
an uncontrolled variable that is unintentionally allowed to disrupt the results of a study
(variable not of interest to the researcher that changes along with the independent variable and could provide an explanation for the results observed)
what are the two frequent forms of bias?
demand characteristics
experimenter bias
what are demand characteristics?
cues or features within an experiment that may lead participants to respond in a particular fashion
ex. if a participant knew or thought they were getting alcohol they may act drunk
what is the good participant effect?
when participants behave in a manner that they perceive the experimenter wants them to behave in
what is experimenter bias/ rosenthal effect?
occurs when an experimenter knows how participants are supposed to be responding and may unwittingly coach them to respond in an appropriate fashion or may interpret a participants behaviour to favour the preferred response
what is the best method to avoid bias?
run the study blind
what is single blind?
the participants (or experimenter) do not know if they are in the control or experimental group
what is double blind?
both the participants and the experimenter are unaware of which is the control and experimental group
what are the steps needed to follow to discover whether the data you collected supports the alternative or null hypothesis?
- check that the data makes sense (look at outliers)
- summarize the data you collected on the dependent variable using descriptive statistics (mean, SD)
- compare means and standard deviations
- maybe state that the independent variable had a significant effect on the dependent variable
how large should the difference in means, between the experimental and control groups be?
means have to be sufficiently different so the difference could not readily be obtained by chance
what is inferential statistics
a way to help us infer whether a specific result observed in a sample reflects what we would observe in the population
what is the most common form of inferential statistics?
null hypothesis testing (p- value)
if inferential statistics conclude that the difference reflects a real difference in the population, what can be said?
statistically significant
what are the goals of any statistical test?
- inform a judgment about whether an effect observed in a sample is good evidence of a real effect in the population
- smaller than alpha is deemed statistically significant p<0.05
- obtain statistically significant results when you have a large sample size
- statistically significant results when effect size is large
if we can determine that the alternative hypothesis is likely to be true in the population, what do we say?
we reject the null hypothesis and conclude that the results are consistent with the research hypothesis
what is probability?
the likelihood or chance that some event or outcome occurs (we want to specify the probability that an event will occur if there is no difference in the population)
what is a probability distribution?
the outcome of a mathematical function that provides the probabilities of different possible outcomes
what is a binomial distribution?
when each trial deals with two possible outcomes
ex. heads or tails
what is a sampling distribution?
a probability distribution of a statistic that is obtained through repeated sampling of a specific population
ex. if you flip a coin over and over again an infinite number of times you will most frequently get 50% correct
what are the basic features of a null hypothesis sampling distribution?
- assumes null hypothesis is actually true in the population
- is a frequency distribution of all possible results that would occur if a study were repeated an infinite number of times, using the same sample size drawn from the same population each time
- is a distribution (represents a set of possible outcomes)
- no effect is the most frequently occurring result because this sampling distribution assumes that the null hypothesis is true
what is statistical significance?
a statistically significant result is one that has a low probability of occurring if there is no effect in the population (unlikely that the difference between the sample means was due to random error)
what is level of significance (alpha level/ α)?
the probability of rejecting the null hypothesis when it is true. (pass or fail)
ex. a significance level of 0.05 indicates a 5% risk of concluding that a difference exists when there is no actual difference.
what is the importance of having an alpha level of 0.01 over 0.05?
it reduces the chance of a false positive, thus with 0.01 there is more of a chance of being wrong
what statistical tests are used for between subjects experimental designs?
t- test
F test
(can be adapted to use for within subjects design)
what is a t- test?
statistical test most commonly used to examine whether the difference in mean scores between two groups is statistically significant
compares two means
what is the equation to find the t value ratio?
t= (group difference/ within group variability)
group difference: difference between your obtained means
within group variability: indicator of the amount of error in your dependent variable (captures the amount of variability of scores around each mean (SD and variance))
if the t increases what happens to the difference between your observed sample mean?
increases
to use the t test sampling distribution to evaluate data there are 4 steps, what are they?
- calculate the value of t from the data
- identify the critical level of t that reflects the chosen alpha level (ex. 0.05)
- compare the obtained t to the critical value of t
- if t exceeds critical value we reject the null
what would this mean, t(36)= 4.37, p< 0.05
t test was used with 36 degrees of freedom resulting in an obtained value of 4.37 which corresponds to a p- value of less than 0.05 if the null hypothesis is true
what is degrees of freedom?
adjustments used in statistical analyses to account for the fact that we are estimating population values using data from small samples (the smaller the sample, the larger the impact degrees of freedom has on the critical value)
how do you find the degrees of freedom?
add all the sample sizes together and subtract 2 from that total
what is a one- tailed versus two tailed t- test?
one-tailed tests will only have one critical region whereas two-tailed tests will have two critical regions
what is an F test?
a general statistical test that can be used to examine differences among three or more groups, useful for factorial designs
compares three or more group means
what is ANOVA (analysis of variance)?
ratio of two types of variance that parallels the t ratio
its the same thing as F test
what is systematic variance/ between group variance?
the numerator
the deviation of the group means from the grand mean (which is the mean score of all participants across all conditions in the study)
small when the difference between group means is small and increases as the differences in group means increases
what is error variance/ within group variance?
the denominator
captures how much individual scores in each group deviate from their respective group means
the larger the F ratio, the greater the group differences are relative to the amount of error and the more likely it is that the results are statistically significant
what are the two correct decisions of the null hypothesis?
reject the null hypothesis
fail to reject the null hypothesis (retain the null hypothesis) (no relationship between the variables)
when is a type I error made?
made when we reject the null hypothesis but the null hypothesis is actually true
(false positive)
ex. finding the defendant guilty when the person is innocent
publication bias inflates overall type I error rates, what is publication bias?
the bias that emerges due to the fact that statistically significant results are far more likely to be published than statistically non- significant results
when is a type II error made?
when the null hypothesis is retained based on sample data but in the population the null hypothesis is actually false (population correlation is not zero)
(false negative)
probability of a type II error is relatively low
ex. finding the defendant innocent when the person is guilty
traditionally, which is worse, type I or II?
type I
what does it mean that unreported type II errors reduce long- term accuracy?
because they go away in file drawers
what can help reduce type I errors?
replicating findings
increasing sample sizes when possible
fully disclosing measures and analyses so others can evaluate and verify them
what is the power of a statistical test?
the probability of correctly rejecting the null hypothesis, presuming it is actually false. a power of 0.80 means that you will find an effect 80% of the time
power= 1 - p(type II error)
p(type II error): probability of making a type II error
when is the chi square statistical test used?
when both variables are on a nominal scale (experimental and control conditions)
the null hypothesis test is a pass or fail form of test (p- value), what can we use to see if the estimate is tiny or huge and about the uncertainty around the estimate?
effect size
confidence intervals
what is a p- value?
the probability that data could arise, if the null hypothesis is true
tells us the probability of observing data similar to what we have observed and does not tell us anything about the probability of the research hypothesis or null hypothesis
what is effect size?
tells you how meaningful the relationship between variables or the difference between groups is. A large effect size means that a research finding has practical significance
smaller effect size requires larger samples
what are different types of effect sizes?
pearson correlation (r)
variance in a variable (r^2)
how it is possible for effects of any size to be statistically significant when sample sizes are large enough?
because of how the p- value is calculated, any effect that is not exactly zero can become statistically significant with a large enough sample
what are confidence intervals?
provide information about the uncertainty around the estimate. as sample size increases, the confidence interval narrows
what is conclusion validity?
the extent to which the conclusions about the relationships among variables reached on the basis of the data are correct or reasonable
if the independent variable is nominal and dependent is interval or ratio (ex. academic average), what statistical test would you use?
t test or one way analysis of variance
if the independent variable is interval or ratio and dependent is interval or ratio, what statistical test would you use?
pearson correlation
the probability of making a type II error is called beta (β) is related to 3 elements, what are they?
- alpha level (if we set a very low alpha level to decrease the chances of a type I error we increase the chances of a type II error)
- sample size (true differences that exist in the population are more likely to be detected as the sample size increase)
- effect size (as the population effect- size increases the likelihood of making a type II error decreases)
what are the two broad classes of experiment?
between- subjects design
within subjects design
what is independent groups design/ between- subjects design?
different people experience the levels of the independent variable
what are the three steps to between- subjects experiment/ independent groups design?
- obtaining two approximately equivalent groups of participants
- introducing different levels of the independent variable to the participants (operationally define the independent variable with at least 2 levels)
- measuring the dependent variable
(within- subjects also has step 2 and 3)
in a between- subjects experiment how do you decide how to assign participants into equivalent groups independent variable?
create equivalent groups
eliminate any potential selection differences
what are selection differences?
the people selected to be in the experimental condition should not differ in any systematic way from those selected for the control condition
ex. mostly high income participants are control and mostly low income participants are the experimenta; then this would result in a selection difference
for between- subjects experiment how do researchers ensure that the participants in each condition are approximately equivalent?
random assignment (groups determined by chance, most common method)
matched- pairs design/ yoked design
what is the matched pairs design?
groups are made equivalent by first selecting pairs of participants who score the same (matched) on some variable of interest, then use random assignment to determine which person in each pair will experience which condition
when is a matched pairs design used?
when it is not possible to collect a large sample
with a large number of participants, random assignment increase the likelihood that nuisance variables related to participant characteristics will be approximately equally distributed across conditions, what are nuisance variables?
variables not of interest
(participant characteristics cannot be an alternative explanation for the experimenters results)
what is within subjects design/ repeated measures design in respect to being equivalent?
the groups of participants for each level of the independent variable are already equivalent by virtue of the design
(removes selection bias or differences in participant characteristics)
what does it mean to operationalize for each variable?
turn a conceptual variable into a set of operations (specific instructions, events, and stimuli to be presented to participants)
why is it important to operationalize the dependent variable? step 3
allows us to measure the effect of the independent variable on the dependent variable (same measurement procedure is used for both conditions in order to be compared)
when the groups are equivalent and there are no confounding variables, or other threats to internal validity, what can be concluded?
difference between conditions for scores on the dependent variable are caused by the independent variable
what are different types of independent groups design using random assignment?
pretest/ pretest- posttest design
prosttest only design
what is a pretest used for?
to measure levels of the key dependent variable
to ensure that participants in both conditions had approximately equal levels of the construct presumed to be affected by the independent variable
to more accurately measure change in this construct
why is a pretest used?
to measure that variable before any experimental manipulation, then the scores for the two groups are compared to ensure that the two groups were approximately equivalent on the critical variable (before manipulation)
why is the pretest- posttest design so important?
makes it possible for researchers to be absolutely sure that the groups were equivalent at the beginning of the experiment for a crucial variable
is posttest the independent or dependent variable?
dependent variable, measured after the experimental manipulation of the independent variable
what are the 3 main reasons why a researcher may add a pretest?
- to counter problems associated with a small sample size
- to select appropriate participants
- when participants might drop out of the study
why is it important to counter problems associated with a small sample size?
because as sample size decreases it becomes less likely that the groups will be approximately equal
why might researchers need to use a pretest to select appropriate participants?
in order to find the lowest or highest scorers on a measure of smoking, math anxiety or prejudice for example (once participants are identified, they would be randomly assigned to the experimental or control group)
participants choosing to leave a study can produce a problem known as selective attrition, what is it?
the tendency of some people to be more likely to drop out of a study than others
if people drop out for some reason related to the manipulation, this selective attrition may cause a difference between groups
when is a posttest given? example
ex. study of a treatment to reduce smoking. one possibility is that the heaviest smokers (most addicted) in the experimental condition might be more likely to leave than those in the control condition (no treatment), thus the posttest is given only the light smokers would remain in the experimental condition
what is a disadvantage of a pretest?
it can sensitize participants to what you are studying, enabling them to figure out your hypothesis, in turn, influencing the way participants react to a manipulation
how can a pretest be disguised through deception?
embed the pretest in a set of irrelevant measures so that it is not obvious that the researcher is interested in a particular topic
what do independent group/ between subject designs have in common with many other experiment types?
having separate groups of randomly assigned participants for both the experimental and control conditions
what is a within subjects design/ repeated measures design?
each participant experiences each condition of the experiment (each participant is exposed to the independent variable, while the experimental condition and control condition are not exposed to the independent variable)
example of within subjects design/ repeated measures design?
you want to know the effect of a new drug on memory, you have 20 participants. day 1 participants are given a memory task, day 2 participants are given the drug and memory task (or can randomly select half the participants and given them the drug on day 1, then on day 2 participants who did not get the drug on day 1 will get the drug and those who already got the drug wont get it)
what is the advantage of a repeated measures design?
the important source of unwanted variation or error and individuals differences is totally eliminated since each participant serves as their own control
fewer research participants are needed because everyone participates in all conditions (not as costly)
extremely sensitive (able to detect small differences between conditions)
what is a more powerful design, repeated measure design or between subjects design?
repeated measure design
what does it mean when a design is more powerful?
there is a greater chance that you will find a real difference between the experimental and control condition
what is a disadvantage of repeated measures designs compared to between- subjects design?
time consuming
demanding on participants
carry over effect
conditions must be presented in a particular order
what is the order effect?
occurs when participants’ responses in the various conditions are affected by the order of conditions to which they were exposed
what is a time related order effect?
occurs when there is a sequence of tasks to perform
includes practice effects, fatigue effects, contrast effect
when does a practice effect occur?
when performance improves because of repeated practice with a task
when does a fatigue effect occur?
when performance worsens as participants become tired, bored or distracted
when does a contrast effect occur?
when the response to the second condition in the experiment is altered because experiences of the first highlight how they are different
what are two ways to deal with order effects in within- subject designs?
using counterbalancing techniques
ensuring time between conditions is long enough to minimize the influence of the first condition on the second
what is complete counterbalancing? example
any effect of order is equally distributed between the conditions (does not remove potential influence of order effects)
ex. for a study studying if kids have better comprehension listening to books or reading them, you will have one group that listens then reads, and the other group reading then listening
one technique to control for most order effects without having to run all possible orders is to employ a latin square design, what is it?
partial counterbalancing method that uses a limited set of all possible orders carefully constructed to ensure that each condition appears first, second, third… and also ensures that each condition appears directly after each other condition exactly once
what does it mean to ensure time between conditions is long enough to minimize the influence of the first condition on the second
for an experiment on sleep, it takes time to recover from the effects of sleep loss so the researchers might give participants a week to recover
what is the downfall to ensuring time between conditions is long enough to minimize the influence of the first condition on the second?
more difficult to recruit volunteers
selective attrition may become a problem (some people leave the experiment)
when does an experiment have high internal validity?
when the results of an experiment can confidently be attributed to the independent variable (researcher must design and conduct the experiment so that only the independent variable can be the cause of the results)
