W3: Replication Crisis, Sampling Baises Flashcards
What is the ‘replication crisis’?
A phenomenon where many scientific studies cannot be replicated, raising concerns about the validity of research findings.
What percentage of studies replicated similar results in the replication crisis?
Between 40% and 50% of studies replicated similar results.
What was observed about the strength of effects in replicated studies?
The strength of the effects was generally smaller in the repeated tests.
What are two major method problems contributing to the replication crisis?
- No Replication
- No Continuity
What are some examples of Questionable Research Practices (QRPs) related to replication?
- Fraud
- Mistakes
- P-Hacking
What is p-hacking?
The practice of trying different options in data, method, or analysis until achieving a significant p-value.
What does the term ‘HARKing’ refer to?
Hypothesizing After the Results are Known; presenting exploratory findings as if they were hypothesized beforehand.
What is the ‘File-Drawer Problem’?
The tendency for researchers to not publish non-significant results, leading to a biased representation of research findings.
What is ‘pre-registration’ in research?
The process of planning and documenting the hypothesis, method, and analyses before collecting data.
What are the advantages of pre-registration?
- Extra planning with a checklist
- Evidence against p-hacking or HARKing
- Easier tracking of changes to the study
What is ‘open and transparent science’?
Making materials, data, and publications available for free public access.
Fill in the blank: Pre-registration and open/transparent science help reduce ______.
[Questionable Research Practices (QRPs)]
What does open science fulfill in terms of research ethics?
It facilitates replication and continuity in scientific research.
True or False: The replication crisis only affects psychology research.
False
What is meant by ‘no continuity’ in research?
The disruption of continuity in research results due to questionable practices.
What is the purpose of sampling?
To select individuals or groups representing the whole population.
What are the four main sampling strategies?
Probability/Random Sampling – Equal chance for all.
Stratified Sampling – Divide population into groups, then sample.
Purposive/Targeted Sampling – Specific individuals selected.
Convenience Sampling – Uses readily available individuals.
What is the purpose of sampling?
To select individuals or groups representing the whole population.
Probability/Random Sampling
Equal chance for all.
Stratified Sampling
Divide population into groups then sample.
Purposive/Targeted Sampling
Specific individuals selected.
Convenience Sampling
Uses readily available individuals.
What are common questions when considering sampling bias?
Who are we studying?
Who sees or agrees to participate?
Who completes or stays in the study?
Who are we not studying?
How can you use distributions to assess your sample?
By visualizing data distributions (e.g. negative or positive skew) to understand sample characteristics and biases.
Why is visualizing data important in sampling?
It provides insights into sample characteristics, biases, and the phenomenon studied.
What does “WEIRD populations” refer to?
Western, Educated, Industrialized, Rich, and Democratic populations.
How can sampling strategies be compared to recipes?
Sampling strategies are like choosing ingredients—know who/what is included and excluded.
What is a disadvantage of random sampling?
It may not perfectly represent subgroups within the population.
What is an example of using convenience sampling appropriately?
When studying phenomena relevant to a specific, easily available group.
What error might Instagram have made by surveying after showing ads?
Sampling bias—surveying only current users may exclude those who quit due to ads.
