final exam Flashcards
Why can’t we say that we have “proven” anything?
because they use that term to refer to the result of a logical deduction. In this rigorous sense, scientific theories can never be proven; they can only be confirmed. (weight of evidence)
can research explain all cases?
no, because research is probabilistic, but can explain a portion of the cases
research is done on a sample, so… there is always some error
sampling error
small chance that we made error, so we set probability to p
statistics
something that varies/changes
variable
factor in an experiment that researchers manipulate so that they can determine its effect, factor in a controlled experiment that is deliberately changed; also called manipulated variable
independent variable
the reaction to the independent variable changing
dependent variable
does not change (only has one level) or is kept the same
constant
carefully define concept at theoretical level (conceptual definition)
concept variable = construct
a variable of interest, stated at an abstract level, usually defined as part of a formal statement of a psychological theory
construct
to turn a conceptual definition of a variable into a specific measured variable or manipulated variable in order to conduct a research study
operationalize
reasonable, accurate, justifiable
validity
how consistent your results are
reliability
you are ONLY able to make casual claims with a…
true experiment
what 3 criteria must you adhere to have a true experiment?
random sample, random assignment, and an IV with 2 levels at least
why do researchers use random assignment to treatment groups?
increases internal validity
used only in experimental designs to assign participants to groups at random (increases internal validity)
random assignment
everybody has equal chance of being chosen (increases external validity)
random selection
A “false positive” result from a statistical inference process, in which researchers conclude that there is an effect in a population when there really is none
Type I error
a “miss” in the statistical inference process, in which researchers conclude that there is no effect in a population when there really is one
Type II error
a variable of interest, stated at an abstract, or conversational, level
conceptual variable
what are three common measures?
- self-report
- observational measures
- physiological measures
concepts can be operationalized in lots of different ways so…
it is a good idea to use more than one concept to see if they correlate
why are scatterplots used?
compile data after test (ex. IQ)
how can we use correlation coefficient “r” to quantify reliability?
slope direction, strength of relationship
how do we interpret “r”?
r = +- 1.0
how do validity and reliability relate to one another?
something can be reliable and not valid - however, you can not have something valid that is unreliable too
what are the different types of validity?
face, content, criterion- known groups evidence; convergent and discriminant
how can question order impact someone’s answers? what can be done about it?
questions asked earlier in the survey may influence the way a person answers a question later in the survey
how can you get people to respond honestly and/or accurately?
conduct pilot tests or focus groups early in the survey
a shortcut respondents may use to answer items in a long survey, rather than responding to the content of each item
response set
examples of response sets
non differentiation, acquiescence or yea sayers, nay sayers, fence sitting
observer bias- how can it be addressed?
systematic errors in observation that occur because of an observer’s expectations
what are observer effects? (also known as expectancy effects)?
bright vs. dull maze rats, clever Hans-researcher subtly communicated to participants “how they should behave”
How do we prevent observer bias/expectancy?
use masked or blind designs, video or audio record, use more than one observer to assess inter-rater reliability, make sure coding/observing system is well thought out, observers well trained
What can be done about reactivity?
unobstrusive observations and deception
extent to which research results apply to a range of individuals not included in the study
generalizability
extent to which we can generalize findings to real-world settings
external validity
why are generalizability and external validity important for frequency claims?
frequency claims need random sample; when a random sample can’t be used, assess whether potential bias will have impact on results; casual and association claims make external validity a lower priority
population vs. sample
whole set vs part of population
simple random sample, cluster samples, multistage samples, stratified random samples, oversampling, systematic sampling, weighting - combo of sampling techniques
probability sampling
convenience sampling, purposive sampling, snowball, and quota
non-probability sampling
what are the three criteria for a casual claim?
- covariance
- temporal precedence
- internal validity
casual claims can only be made or based on what?
true experiments
bivariate correlation
calculates correlation
argues that one level of a variable is likely to be associated with a particular level of another variable
association claim
random assignment to treatment, random selection of a sample, and an IV with at least 2 levels
criteria to be considered an experiment
why can’t we talk about cause and effect with casual claims?
can only be made or based on experiments
how do experiments differ from non-experimental designs we have talked about?
experiments are more powerful than non-experimental designs because they have two group designs (one receives IV and other group doesn’t or IV has two levels - level 1 = IV level and level 2 = another level of IV)
experiment with one independent variable with two levels
simple experiment
know how to pick out IV and DV
example experiment
what is a control variable?
independent variable (you control it)
why are comparison groups so important?
don’t know if your experiment is working or not
level of IV that is intended to represent no treatment or a neutral condition
control groups
group that receives the IV
treatment groups
when control group is exposed to an inert treatment such as a sugar pill
placebo groups
experimenters control…
temporal precedence
threats to internal validity but don’t know what is causing the change
confounds
experimenter’s mistake in designing IV; second variable happens to vary systematically along with intended IV and is an alternative explanation for results
design confound
systematic variability impact on an experiment?
levels coincide in a predictable way with experimental group membership that creates a potential confound; seriously jeopardize internal validity
unsystematic variability impact on an experiment?
levels of a variable fluctuate independently of experimental group membership that contribute to variability within groups (no confound)
selection effects - how can you control for it?
happens when participants in one level of IV are systematically different from those in the other or when experimenters let participants choose their groups; use random assignment (desystemize types of participants who end up in each group) or matched groups
match participants on a variable (other than the IV) that might otherwise impact how they behave in your experiment (your DV)
matched groups
independent groups design = between subjects design = random assignment design
synonyms; randomly assign participants to groups (or treatment levels); independent because there are no connections/ties between subjects; two basic forms: posttest only design and pretest/posttest design
assigned to IV groups are tested on DV once; testing after experiment
posttest only design
DV is measured more than one time (before and after treatment on DV), but there is only one administration of the IV whereas a within subjects design, the participants get exposed to multiple levels of an IV
pretest posttest only design
within groups design = repeated measures = within subjects or correlated groups design
synonymous; test the same participants in both treatment conditions (IV and DV); very powerful
strengths/weaknesses of within groups design
weaknesses: order effects, might not be possible or practical, people see all levels of IV and then change the way they would normally act.
Strengths: ensures participants in the two groups will be equivalent
what is meant by power?
probability that a study will show a statistically significant result when an IV truly has an effect in the population
with within groups design there are order effects. what are examples of order effects?
practice or testing effects and carryover effects
long sequence might lead participants to get better at the task
practice or testing effects
some form of contamination carries over from one condition to the next
carryover effect
how can order effects be avoided?
counterbalancing
used to deal with order effects; sample is divided in half, with one half completing the two conditions in one order and the other half completing the conditions in the reverse order
counterbalancing
all possible condition orders are represented
full counterbalancing
only some of the possible condition order are represented
partial counterbalancing
technique for partial counterbalancing; a formal system to ensure that every condition appears in each position at least once
Latin square
create an alternative explanation for a study’s results
demand characteristics
how do we use statistics?
to interpret data (graphs)
what does it mean if something is statistically significant?
unlikely to have been obtained by chance from a population in which nothing is happening
what are threats to internal validity?
maturation history regression attrition testing instrumentation
why is a one group pretest posttest only design a bad research design?
not enough between-groups difference
any study can suffer from…
observer bias; demand characteristics; good participant effect - prevent with masked, single or double blind
other problems that can make or break an experiment
not enough difference between groups - weak manipulations, insensitive measures, ceiling and floor effects
does too much within group variability make it harder to detect group differences?
yes, the less within-group variability, the less likely it is to obscure a true group difference
reason for high within-group variability; human or instrument factor that can inflate or deflate a person’s true score on DV
measurement error
how does adding more participants help?
reduces the influence of individual differences within groups which enhances the study’s ability to detect differences between groups
outcome if the IV did not make a difference in the DV; there is not covariance between the two
null effect
experiments with two or more IV
complex/factorial designs
why would a researcher opt to use a complex design? benefits?
levels not manipulated; to test whether an IV affects different kinds of people in the same way
how do we interpret results from a complex design?
collect data on a variety of DV- graphs
how are marginal means calculated?
sample sizes are equal = simple average.
sample sizes unequal = computed using the weighted average counting larger sample more; hep you eye ball data to determine if there is a main effect (need to know inferential statistics to verify - ANOVA)
how are factorial designs used to test theories?
study how variables interact by combining them in a factorial and measure whether the results are consistent with the theory
one IV is manipulated as independent-groups and the other is manipulated as within-groups
mixed factorial
there are more complicated factorial designs…
ex. 2 x 2 x 2
what is the difference between a true experiment and a quasi-experimental design?
quasi-participants cannot randomly be assigned to an IV
what are three types of small N designs?
- stable baseline design
- multiple baseline design
- reversal design
what are the benefits of quasi-experiments?
real world applicability and external validity and excellent for things you can’t ethically vary an IV
change in behavior that emerges more/less over time
maturation threats
affects most members of treatment group at same time as treatment itself, making it unclear whether the change is caused by treatment received
history threats
group mean is usually extreme at one time then next time it is measured, it is less likely to be extreme
regression threats
when attrition is systematic, certain kind of participant drops out
attrition threats
change in participants as a result of taking a test more than once
testing threats
measuring instrument changes over time
instrumentation threats
a preexisting variable that is often a characteristic inherent to an individual, which differentiates the groups or conditions being compared in a research study. Because the levels of the variable are preexisting, it is not possible to randomly assign participants to groups
quasi-independent variable
