Chapter 14- Replication, Transparency, and Real World Importance Flashcards
Replicable
Whether researchers will get the same results if they conduct the same study again. This is part of interrogating statistical validity- we ask about effect size, confidence interval, and what else is known from other studies
Direct replication
Researchers repeat an original study as closely as they can to see whether the effect is the same in the newly collected data. If there are any threats to internal validity or other flaws in the original study, these flaws will be repeated. Also, the theory will not be tested in a new context
Conceptual replication
Researchers explore the same research question but use different procedures. The conceptual variables in the study are the same, but the procedures for operationalizing them are different.
Replication plus extension
Researchers replicate their original experiment and add variables to test additional questions
Why might a study not be replicable?
It could be an issue with the replication study- there are differences with sample, materials, etc even in direct replications. There could also have been a problem with the original study.
Scientific literature
Consists of a series of related studies conducted by various researchers that have tested similar variables. You might hear about the literature on alcohol and aggression, etc
Meta-analysis
A way of mathematically averaging the results of all the studies that have tested the same variables to see what conclusion that whole body of evidence supports
File drawer problem
The idea that a meta-analysis might be overestimating the true size of the effect because negligible or opposite effects have not been included in the collection process.
Harking
Creating an after the fact hypothesis about a research result, so it appears the results were predicted all along. This misleads readers about the strength of the evidence
p-hacking
Researchers might remove outliers from the data, compute scores several different ways, or run a few different types of statistics. This is called p hacking because the goal is to get the p value under .05. This might not be done intentionally, but biases creep in.
Open science
The practice of sharing one’s data and materials freely so others can collaborate, use, and verify the results
Open data
Psychologists provide their full data set, so other researchers can reproduce the statistical results or conduct new analyses on it.
Open materials
Psychologists provide their study’s full set of measures and manipulations so others can conduct replication studies more easily. All conditions and measured variables are reported, not just the ones that worked.
Preregistration
When scientists publish their study’s method, hypotheses, or statistical analyses in advance of data collection.
Which validity does replicability help interrogate?
External validity- especially when it’s a conceptual replication or replication plus extension study because they demonstrate that the results can be generalized to other settings
How is a study’s generalizability to other people assessed?
You would ask how the participants were obtained. If a convenience sample is used, you can’t be sure of generalizability. If intended to generalize to a certain population, the researchers must draw a probability sample from that population.
Ecological validity
A study’s similarity to real world contexts
Theory testing mode
When researchers are designing correlational or experimental research to investigate support for a theory. In theory testing mode, external validity often matters less than internal validity. An example is Harlow’s contact comfort experiment. The majority of studies in psychology are theory testing.
Generalization mode
When researchers want to generalize the findings from the sample in a previous study to a larger population. Generalization mode is more concerned with external validity, and researchers are careful to use probability samples.
Cultural psychology
A subdiscipline of psychology focusing on how cultural contexts shape the way a person thinks, feels, and behaves
Muller-Lyer illusion
Two lines are actually the same length, but one appears longer. Almost all North Americans and Europeans fall for it, but many people around the world don’t.
Field setting
When research takes place in the real world
Experimental realism
When lab experiments create situations in which people experience authentic emotions, motivations, and behaviors
