Final Flashcards
Bivariate correlation
an association that involves exactly two variables
Mean
the arithmetic average (use for categorical data, bar graph?)
T-Test
tests whether the difference between means (group averages) is statistically significant
Construct validity for association claims
How well was each variable measured?
Statistical validity for association claims
How well do the data support the conclusion?
Internal validity for association claims
Can we make a causal inference from association?
Correlation is NOT causation
External validity for association claims
To whom can the association be generalized?
Effect size
describes the strength of a relationship between two or more variables
- Larger effect sizes allow for more accurate predictions.
- Larger effect sizes are usually more important.
Statistical significance
refers to the conclusion a researcher reaches regarding the likelihood of getting a correlation of that size by chance.
Outlier
an extreme score; a single case or a few cases that stand out from the pack.
-Outliers matter the most when a sample is small.
Restriction of range
when the full range of scores for one of the variables in a correlational study is not provided.
-This can make the correlation appear smaller than it actually is.
Curvilinear association
when the relationship between two variables is not a straight line. It might be positive up to a point, and then become negative.
Three Causal Criteria
- Covariance of cause and effect
- Temporal precedence
- Internal validity
Covariance
results must show a correlation between the cause variable and the effect variable
Temporal precedence
the cause variable must precede the effect variable; it must come first in time
Internal validity
there must be no plausible alternative explanations for the relationship between the two variables
Directionality problem
we don’t know which variable came first (temporal precedence criterion)
Third-Variable Problem
When we come up with an alternative explanation for the association between two variables, that the alternative is some lurking third variable
(internal validity criterion)
Moderator
When the relationship between two variables changes depending on the level of another variable, that other variable is called a moderator.
Multivariate Designs
involve more than two measured variables
Longitudinal Design
can provide evidence for temporal precedence by measuring the same variables in the same people at several points in time
Cross-Sectional Correlations
Test to see whether two variables, measured at the same point in time, are correlated.
Autocorrelations
determine the correlation of one variable with itself, measured on two different occasions
Cross-lag correlations
show whether the earlier measure of one variable is associated with the later measure of the other variable. Three possible outcomes.
–Help to establish temporal precedence.
Multiple Regression
Using this technique, researchers can evaluate whether a relationship between two key variables still holds when they control for another variable. (can help rule out third variables)
Criterion variables
When researchers use multiple regression, the first step is to choose the variable they are most interested in understanding/ predicting
-Dependent variable
Predictor variables
The rest of the variables in a multiple regression analysis
-Independent variable
Beta Values
tell you the strength and direction of the relationship for multiple-regression analysis
Adding predictors to a regression
helps us:
- Control for several third variables at once.
- Get a sense for which predictors most strongly impact our criterion variable.
Does regression establish causation? Why or why not?
NO
- Doesn’t always establish temporal precedence.
- Can’t control for variables you don’t measure.
Parsimony
the degree to which a scientific theory provides the simplest explanation of some phenomenon. The simplest explanation of a pattern of data.
Mediator
mediating variable.
Can get at the why behind a relationship between variables
Experiment
when a researcher manipulates at least one variable and measured another
Manipulated Variable
a variable that is controlled, such as when researchers assign participants to a particular level of the variable
Measured Variable
take the form of records of behavior or attitudes such as self-reports, behavioral observations, or physiological measures
Independent Variable
the manipulated (causal) variable
Conditions
the IV’s levels
Dependent Variable
the measure or outcome variable
Control variable
any variable the experimenter holds constant on purpose. Trying to control for potential third variables.
Comparison group
Usually used when there is no control group. When the levels of the independent variable differ in some intended and meaningful way.
Control group
a level of an independent variable that is intended to represent “no treatment” or a neutral condition
Treatment group
the other levels of the independent variable that are not neutral make up the treatment group
Placebo group
when the control group is exposed to an inert treatment
Confounds
possible alternative explanations; potential threats to internal validity
Design Confound
an experimenter’s mistake in designing the independent variable. It is a second variable that happens to vary systematically along with the intended independent variable and therefore is an alternative explanation for the results.
Systematic variability
variability between groups that causes a confound
Unsystematic variability
random or haphazard variability - across both groups would NOT be a confound.
Selection effects
when the kinds of participants in one level of the independent variable are systematically different from those in the other. Can happen when experimenters let participants choose what group they want to be in.
Random assignment
helps us avoid selection effects
Matched groups
First measure the participants on a particular variable that might matter to the dependent variable. Next, they would match participants in pairs, and then randomly assign one of the two of them to each of the conditions.
Independent-groups design
different groups of participants are placed into different levels of the independent variable.
Within-groups design
there is only one group of participants and each person is presented with all levels of the independent variable.
Posttest only design
Participants are randomly assigned to independent variable groups and are tested on the dependent variable once.
-Satisfy all three criteria for a causal claim.
Pretest/posttest design
Participants are randomly assigned to at least two different groups and are tested on the key dependent variable twice - once before and once after exposure to the independent variable.
Within-groups design
there is only one group of participants and each person is presented with all levels of the independent variable.
Repeated-Measures Design
A type of within-groups design in which participants are measured on a dependent variable more than once, after exposure to each level of the independent variable.
Concurrent-Measures Design
A type of within-groups design where participants are exposed to all the levels of an independent variable at roughly the same time, and a single attitudinal or behavioral preference is the dependent variable.
Power
the probability that a study will show a statistically significant result when an independent variable truly has an effect in the population
Threats to internal validity in within groups design
- order effects
- practice effects
- carryover effects
Order effects
when being exposed to one condition changes how participants react to another. This is a confound.
Practice effects
(aka fatigue effects) where a long sequence might lead participants to get better at the task, or get tired or bored towards the end
Carryover effects
when some form of contamination carries over from one condition to the next
Counterbalancing
when researchers present the levels of the independent variable to participants in different sequences
Full counterbalancing
all possible condition orders are represented (when a within-groups experiment has only two or three levels of an IV)
Partial counterbalancing
some of the possible condition orders are represented
- randomized order for every subject
- Latin square
Demand characteristic
A cue that can lead participants to guess what an experimenter’s hypothesis is. Experiencing all levels of the independent variable (IV) changes the way participants act.
Manipulation check
an extra dependent variable that researchers can insert into an experiment to convince them that their experimental manipulation worked.
Pilot study
a simple study using a separate group of participants that is completed before (or sometimes after) conducting the primary study of interest.
Construct validity of causal claims
How well were the variables measured and manipulated?
External validity of causal claims
To whom or what can the causal claim generalize?
Statistical validity of causal claims
How well do the data support the causal claim?
Internal validity of causal claims
Are there alternative explanations for the outcome?
One-group, pretest/posttest design
a researcher recruits one group of participants, measures them on a pretest, exposes them to a treatment, intervention, or change, and then measures them on a posttest
BAD experiment
Maturation threat
A change in behavior that emerges more or less spontaneously over time. People adapt to changing environments.
Prevention: Use a true treatment (pre/post design which has two groups), Having a comparison group in a true experiment
History threat
An experimental group that changes over time because of an external factor that affects all or most members of the group.
Must affect most people in the group in the same direction (systematically), not just a few people (unsystematically)
Prevention: Have a comparison group with equal exposure to all factors
Regression to the mean
When group average is extreme at Time 1 (pretest) it is more likely to be less extreme at Time 2 (posttest)
Prevention: with a comparison group and an analysis of the pattern of results
(If both the comparison and experimental groups are equally as extreme at pretest, then researchers can account for any regression effects and their results)
Attrition threat
When only a certain kind of participant drops out (systematic) before the end of study
Prevention:
- researchers will remove the scores of the participants that dropped out
- check the pretest scores of the dropouts because having more extreme scores is a greater threat to internal validity than those scores close to the group average
Testing threat
a specific kind of order effect, refers to a change in the participants as a result of taking a test (dependent measure) more than once.
Prevention: researchers may eliminate a pretest all together and use a posttest only design
- If they do use a pretest, they might use alternative forms of the test for the two measurements
- Comparison group
Instrumentation threat
occurs when a measuring instrument changes over time
Prevention:
- switch to posttest only design
- make sure that the pre-test and post-test measures are equivalent
- use clear coding manuals.
- Counterbalance the versions of the test
Selection-history threat
an outside event or factor affects only those at one level of the IV
Prevention:
Selection-attrition threat
only one of the experimental groups experiences attrition
Prevention:
Observer bias
occurs when researchers’ expectations influence their interpretation of the results
Prevention: double-blind study, masked design
Demand characteristics
a problem when participants guess what the study is supposed to be about and change their behavior in the expected direction
Prevention: double-blind study, masked design
Double-blind study
neither the participants nor the researchers who evaluate them know who is in the treatment group and who is in the comparison group
Masked design
blind design) participants know which group they are in, but the observers do not
Placebo effect
occurs when people receive a treatment and really improve- but only because the recipients believe they are receiving a valid treatment
Prevention: use a double-blind placebo control study
Null effect
when there is no significant covariance between the IV and the DV.
2 reasons:
-Actually no effect of IV on DV
-The study was not designed or conducted carefully enough
Why might there not be enough between groups difference?
Weak manipulations
insensitive measures
ceiling and floor effects
Ceiling effect
all scores are squeezed together at the high end.
Can be the result of IV or DV
Floor effect
all scores are clustered together at the low end
Noise
too much unsystematic variability within each group
Reasons for high within-group variability
Measurement Error
Individual Differences
Situation Noise
Measurement Error
a human or instrument factor that can inflate or deflate a person’s true score on the DV.
Situation Noise
External distractions
Solutions for Noise
Solutions for Measurement Error: 1. Use reliable, precise tools. 2. Measure more instances. Solutions for Individual Differences: 1. Change the design. 2. Add more participants.
Interaction effect
when the effect of one IV depends on the level of another IV
Crossover interaction
The “it depends” effect OR “a difference in differences”
Spreading interaction
“only when” effect
Factorial design
when there are two or more independent variables (also referred to as factors).
2x2 = 4 conditions
Participant variable
a variable whose levels are selected (measures), not manipulated
Ex: age, gender, ethnicity
Why are factorial designs useful?
- Can test limits
- Serve as a way to check external validity.
- Interactions show moderators.
- Help us test theories.
Main effect
The overall effect of one IV on the DV
- In a factorial design with two IVs there are two main effects.
- Main effects may or may not be significant.
Marginal means
the arithmetic means for each level of IV
Independent-groups factorial design
(between-subjects factorial) both IVs are studied as independent groups
-If a design is a 2x2, there are 4 different groups of participants in the experiment
Within-groups factorial design
(repeated-measures factorial) both IVs are manipulated as within-groups
- If the design is 2x2, there is only one group of participants, but they participate in all four combinations, or cells, of the design
- Requires fewer participants
Mixed factorial design
one IV is manipulated as independent-groups and the other is manipulated as within-groups
Three-way design
- (2x2x2) there are 2 levels of the first IV, 2 levels of the second, and 2 levels of the third
- Creates 8 cells (conditions)
- Construct 2x2 table twice, one for each level of the third IV
- To graph, create two side-by-side line graphs
Main effects in three-way designs
- represents a simple overall difference, the effect of one IV, averaged across the other two IVs
- 3 Main Effects (2x2x2) because of 3 IVs
- When describing each main effect, you don’t mention the other two IVs because you averaged across them
Interactions in three-way designs
3 separate two-way interactions
Three-way interaction
results from a three-way design; if significant, it means that the two-way interaction between two of the IVs depends on the level of the third IV
Quasi-experiment
A quasi-experiment differs from a true experiment in that the researchers do not have full experimental control. Participants may not be randomly assigned but are instead assigned
Nonequivalent control group design
A type of quasi-experimental design that has at least one treatment group and one comparison group, but unlike a true experiment, participants have not been randomly assigned to the two groups.
Nonequivalent control group pretest/posttest design
A type of quasi-experimental design where the participants were not randomly assigned to groups and were tested both before and after some intervention.
Interrupted time-series design
A quasi-experimental study that measures participants repeatedly on a dependent variable before, during, and after the “interruption” caused by some event.
Nonequivalent control group interrupted time-series design
It combines Nonequivalent control group pretest/posttest and Interrupted time-series designs.
The independent variable is study as both a repeated-measures variables (interrupted time-series) and an independent-groups variable ( nonequivalent control group).
Wait-list design
(to control for selection effects) all the participants plan to receive treatment but are randomly assigned to do so at different times
-is a true experiment because it ensures that the same kinds of people are in each group
Selection effects
relevant only for independent-groups designs, not for repeated-measures designs. Applies when the kinds of participants at one level of the IV are systematically different from those at the other level
Design confounds
Some outside variable accidentally and systematically varies with the levels of the targeted independent variable
Maturation threat (in quasi)
a change in behavior that emerges more or less spontaneously over time
Prevention: ave a comparison group so that you can tell whether the variable actually has an effect or if it is just maturation.
History threat (in quasi)
Some kind of event that occurred during the study period and it is reactions to these events that caused the outcomes we observe.
Prevention: Random assignment and a control group.
Regression to the mean (in quasi)
Occurs when there is combination of random factors in Test 1, and it is unlikely you will get the same combination of factors during Test 2
-Can only threaten internal validity for pretest/posttest designs and when a group is selected for extremely high/low scores
Attrition threat (in quasi)
occur in designs with pretests and posttests when people drop out of a study over time.
Prevention:drop the scores of the participant that dropped out of the study if they are systematic
Testing threat (in quasi)
- Does the pre-test affect the post test?
- Experiments that pretest the subjects may influence the performance of subjects on following tests simply due to the fact that participants have already seen or completed the test before.
- People tend to perform better at any activity the more they are exposed to it.
Instrumentation threat (in quasi)
- Did the measurement method change during the research?
- Changes in testing instrumentation during a study may affect what is being measured and how it is measured.
- Similarly, if human observations are involved, the observations or perceptions of the of the observers may change over time, rather than the actual performance of the test subjects.
Observer bias
When the experimenters’ expectations influence their interpretation of the results
Suggestion: simply ask who measured the behaviors
Was the design blind or double-blind?
Demand characteristics
When participants figure out what the study is about and change behavior to fit the desired outcome
Suggestion: investigate whether participants were able to guess the purpose of the study and respond accordingly
Placebo effects
When participants improve but only because they believe they are receiving an effective treatment
Suggestion: ask whether the design of the study included a comparison group that received an placebo treatment
Small-N Design
when researchers obtain a lot of information from just a few cases
Single-N Design
when researchers restrict their study to a single animal or one person
Stable-baseline design
A study in which a practitioner or researcher observes behavior for an extended baseline period before beginning a treatment or other intervention.
-If behavior during the baseline is stable, the researcher is more certain of the treatment effectiveness.
Multiple-baseline design
Researchers stagger their introduction of an intervention across a variety of individuals, times, or situations to rule out alternative explanations.
Reversal design
A researcher observes a problem behavior both with and without treatment, but takes the treatment away for a while (the reversal period) to see whether the problem behavior returns (reverses). They subsequently reintroduce the treatment to see if the behavior improves again.
Replicable
the same results have actually been reproduced
Direct Replication
When researchers repeat an original study as closely as they can to see whether the effect is the same in the newly collected data
Conceptual Replication
When researchers explore the same research question but use different procedures. The conceptual variables in the study are the same, but the procedures for operationalizing them are different
Replication-plus-extension
When researchers replicate their original experiment and add variables to test additional questions.
- Add another level to one of the independent variables.
- Introduce a new independent variable.
Open science
the practice of sharing one’s data and materials freely so others can collaborate, use, and verify the results
Preregistration
scientists can pre register their study’s method, hypotheses, or statistical analyses online in advance of data collection
Scientific literature
consists of a series of related studies conducted by various researchers that have tested similar variables
Meta-analysis
a way of mathematically averaging results of all the studies that have tested the same variables to see what conclusion that whole body of evidence supports
File drawer effect
meta-analysis might be overestimating the true effect size of an effect because null effects, or even opposite effects, have not been included in the collection process
Ecological validity
a study’s similarity to real-world context. The extent to which a study’s tasks and manipulations are similar to the kinds of situations participants might encounter in their everyday lives
Theory-testing mode
when researchers design correlational or experimental research to investigate support for a theory.
-internal validity matters more than external validity.
Generalization-mode
when researchers want to generalize the findings from the sample in a previous study to a larger population
- frequency claims
Cultural psychology
a sub-discipline of psychology focusing on how cultural contexts shape the way a person thinks, feels, and behaves.
-Challenge researchers who constantly work in theory-testing mode
WEIRD
Western, Educated, Industrialized, Rich, and Democratic
-Most research in psychological science has been conducted on North American college students.
Field setting
another way to refer to a real-world setting in research
Experimental realism
lab experiments that create situations where people experience authentic emotions, motivations, and behaviors
