Exam 1 Flashcards
Empiricism
The practice of basing ideas and theories on testing and experience
Characteristics of scientists
Empiricists, they test theories, they tackle applied and basic problems, they make their research public through the publication process, they talk to the world in popular media
Theories
A systematic body of ideas about a particular topic/phenomenon
describes a relationship among variables
organizes/summarizes knowledge or findings
describes, explains, or predicts behavior
supported by data
FALSIFIABLE
Confounds
Plausible alternatives for the fidning – something that varies along with our IV
What is the problem with using experience to come to conclusions
No comparison group, has tons of confounds, not probabilistic
What is the problem with using intuitions as science
Intuitions are inconsistent, describe the past, and may lead us astray
Availability Bias
Things that come to mind easily are more available to memory and can guide
and/or bias our thinking. Especially true of memories that are recent or vivid
Present/Present Bias
-Examples that are easier to call to mind are
more “available” and can guide/bias our
thinking
- Very similar to availability, but more specifically
deals with the fact that we often fail to look at
absences. Failure to consider appropriate
comparison groups!
Confirmation Bias
The tendency of people to favor information that confirms or strengthens their beliefs or values and is difficult to dislodge once affirmed
What is the order of a Scientific Paper
Introduction, method, results, discussion
Steps of Reading a paper
- Skim
- Re-Read
- Interpret
- Summarize
Measured Variables
Observed and recorded
Manipulated Variables
Controlled for
Conceptual Variable (construct)
abstract, general, theoretical
Operational Definitions
concrete, a specific way to measure something
What are some physiological measures in psychology
fMRI BOLD Signal
What is better physiological or observational measures?
It depends!
Nominal Variables
Categorical
names or categories
Ordinal
Rankings
Quantitative
Interval
Equally spaced numbers
quantitative
scale
Ratio
Meaningful Zero
Quantitative
Scale
Frequency Claims
One variable, measured
Association Claims
2 Variables are linked, both are measured
Causal Claims
1 variable causes change in the other
One must be manipulated
Covariance
as A changes, B also changes
(same direction or
different direction)
In order to make a causal claim, you must have
covariance, temporal precedence, and internal validity
Temporal Precedence
Experimental manipulation -> change in outcome
behavior occurs before effect
Internal Validity
The study’s method ensures that there are no plausible
alternative explanations for the change in B; A is the only thing that is changed
What are the 4 types of validity
Construct, Statistical, Internal, and External
Construct Validity
- Quality of the measures and manipulations
- Did you measure what you said you were going
to measure? - How good is the operationalization
- How reliable are the measures (more later)
External Validity
- How might we want to generalize?
- To other participants
- To other settings (lab/field, cultures/countries, work/home/school)
- To other operationalizations of the same construct
Statistical Validity
- Appropriate and reasonable statistical
conclusions - How well do the numbers support the claim?
- Do you believe the numbers or think the stats
are lying to you? - Do they make the right decision based on the p-value?
-What is the effect size? - Is it well-powered?
Internal Validity
- Was the study free of confounds/alternative explanations?
- Experiment with random assignment to condition?
- Strong control over variables?
- No differences between conditions other than the IV?
Which Validity is typically prioritized
Internal
A Valid claim is ___, ___, and ____
A valid claim is reasonable, accurate, and justified
- Reliability is necessary but not sufficient for validity
Ceiling Effect
all observed scores are on
the high end
Floor Effect
all observed scores are on
the low end
Confound
A design confound is when a second variable varies systematically along
with the IV and provides an alternative explanation for the results; experimenter’s mistake
Internal validity is only threatened if there is ___ variability with the IV
systematic
Systematic variability
trends together
Unsystematic variability
random or haphazard, affects both groups; not a confound!
Selection Effects
when the kinds of participants in 1 group are systematically different than another group
You can avoid selection effects with
Random assignment
Posttest design
Just test after one trial
Pretest/posttest design
Test before and after
Within Groups designs
All participants are exposed to all levels of the independent variables
repeated measures designs
concurrent measures designs (not popular)
Repeated Measures
Participants respond to a dependent variable twice (at
least), after exposure to each independent variable
Pros of Within Groups design
- All levels of the IV revealed to participants
- Participants serve as their own comparison
- Fewer participants needed for a study
- No concern of selection effects
- More statistical power (eliminating 1 source
of noise/unsystematic variance)
Threats to internal validity with repeated measures
order effects, carryover effects, practice effects, fatigue effects
Order Effects
occurs when participants’ responses in the various conditions are affected by the order of conditions to which they were exposed
Carryover effects:
contamination carrying over from one condition to the
next. You drink caffeine and then take a test. Then you drink decaf coffee and take a test. But caffeine is still in your system by the second test
Practice Effects
participants get better at a task over time
Fatigue Effects
participants get worse at a task over time
Full counterbalancing
all possible condition orders
partial counterbalancing
present only some condition orders
The Really Bad Experiment
classic pretest/posttest design but only on a single group
no comparison group
Ambiguous temporal precedence
when it is unclear if IV causses the DV or the other
History Effects
History refers to any event that
occurs between the beginning of
treatment and the measurement of
outcome that might have produced
the observed effect
Maturation Effects
Maturation is a change in behavior
that emerges spontaneously over
time. Changes in the organism that
occur regardless of treatment might
masquerade as a treatment effect
Attrition Effects
Attrition/Mortality refers to who is
dropping out of your study (or
dying)
Regression Effects
Regression to the mean is when
extreme scores become less
extreme over time
Testing Effects
Testing effects refer to a change in
participants as a result of
experiencing the DV more than
once
Instrumentation Effects
Instrumentation effects refer to a
change because the measurement
changes over time, perhaps
becoming more/less reliable
Goodhart’s law
When a measure becomes a target it ceases to be a good measure
Demand Characterisitics:
participants guess what it’s supposed to be about and then change their behavior on expected direction
Why might manipulations be weak
operationalization is super hard
Measurement Error
use reliable, precise measurements
establish construct validiity of measures
use established measures
measure more instances
Statistical Power
the ability to detect an effect if one is there
finding a statistically significant result if the IV really has an effect
studies that are well-powered are able to detect true differences
What is the IQ example for confirmation bias
people who scored low on IQ tests spent longer reading articles that criticized IQ test and people who scored high spent longer reading articles that supported the tests
How many levels do variables need to have
two or more
What is a constant
something that could potentially vary but that has only one level in the study in question
Claim Definiton
argument someone is trying to make
What are the three types of claims
Frequency, Association, Causal
Processing Fluency Account
when information is repeated, ti is processed more fluently and is consequently perceived to be more truthful
What amount of repetition shows the largest increase in truth perception
1 –> 2
Concurrent-measures
Participants are exposed to all the levels of an independent variable at roughly the same time, and a single attitudinal or behavioral preference is the dependent variable.
What are the disadvantages of within group design
Potential order effects (typically resolved with counterbalancing)
May not be possible or practical
When ppl see all levels, they change the way they would normally act
Effect Size
D of .2 is small
D of .5 is moderate
D of .8 is large
When a study has a relatively small sample and more variability in the data, the CI will be relatively ____
wide (less precise)
Two ways to avoid confounds
Matching Groups
Inclusion/exclusion criteria
pros of post test
No practice effect
Less time and money
“Blind”
Less attrition/mortality
Temporal precedence
Cons of post test
Cant get at the change
You need a much larger sample size
Pros of pretest-posttest
Baseline
Each person is their own control
Can have smaller sample size
cons of pretest post test
Time
Practice effects
Hard to recruit and attrition
Factorial Design
when there are two or more independent variables
Participant variable
a variable whose levels are selected/measured not manipulated like age, gender, and ethnicity
Interaction Effect
whether the effect of the original independent variable (cell phone use) depends on the level of another independent variable (driver age)
testing for moderators
The process of using a factorial design to test limits
Main Effect
the overall effect of one independent variable on the dependent variable, averaging over the levels of the other independent variable
In a factorial design with two independent variables, how many main effects are there
two main effects
Marginal Means
the arithmetic means for each level of an independent variable, averaging over levels of the other independent variable.
How do you compute an interaction
you see if there is a difference between differences, and if you are looking on a graph you can tell an interaction if the lines are not parallel
What is more important: Interactions or Main Effects
Interactions
When a factorial design has three independent variables, how many main effects and interactions are there
three main effects (one for each independent variable), plus three separate two-way interactions and a three-way interaction
Three way interaction
if it is significant, means that the two-way interaction between two of the independent variables depends on the level of the third independent variable
When would you find a three-way interaction?
whenever there is a two-way interaction for one level of a third independent variable but not for the other (or different two way interactions)
meta-analysis
Meta-analysis is the statistical combination of the results of multiple studies addressing a similar research question. An important part of this method involves computing an effect size across all of the studies, this involves extracting effect sizes and variance measures from various studies.
Confederate
A person playing a specific role for the sake of the study
Obscuring factors that can be detected with a manipulation check
floor effects
ceiling effects
weak manipulations
Interactions are
symmetric
Interactions test
whether the effect of one IV depends on the level of the other IV
reliable, precise scales can help with
measurement error
experimental control can help with
situation noise
within-groups design can help with
individual differences
What are obscuring factors that can be detected with a manipulation check?
ceiling effects, floor effects, weak manipulations
can measurement error cause null effects and why
yes because it leads to a lot of within group variability
power increases with a ___ sample
larger
What type of validity can a factorial design increase
external validity
What does a moderator do
Changes the relationship between an independent and dependent variable
how many interactions could there be in a 4 x 3 design
one
