QRP Flashcards
What are some of the Questionable research practices?
Fraud?
Unethical?
Conflict of Interest?
Plagiarism?
QRP?
What is The inappropriate manipulation of data analysis when interpreting and writing up data to enable a favoured result to be presented as statistically significant?
P-Hacking
What is the file draw effect?
Publish it only if it has good result effects= Publication bias
Researcher’s Selectivly publish studies which biases the literiture
Do jelly beans cause acne? example
Based on that 5% of false positive (95% confidence)
Maybe its the colour of jelly bean
do 5 tests
1-4 find nothing
5th test we find significance of green jelly beans causing acne
NEWS: green jelly beans linked to acne
not disclosing all the things that you did, but you did other correlations that got reported. Researchers should be more transparent on what they are recording.
Data fishing, data dredging, data snooping is known as?
P-hacking
p-Hacking:
You do lots of tests with the same data
You label insignificant numbers as outliers and not include them in ur data
Reporting the thing that comes out (being selective)
What is the term used to describe when you test results until significant?
Optional stopping
Sample size= too noisy for small samples, so the significance for smaller samples will be higher
What instances/ Acceptable practices can researchers use Optional stopping?
Assessing data quality
Sequential analyses—using a correction (and reporting as such) to statistic that incorporates consideration that decision was made at earlier point in data collection
E.g., patient or animal research
Optimal stopping
What is the term used to describe making a decision on data that’s independent to your research question?
E.g., if collecting EEG data along with a cognitive task, if EEG data is noisy then may need to collect more participants
Assessing data quality
Optimal stopping
What is the term used to describe using a correction
(and reporting as such) to statistic that incorporates consideration that decision was made at earlier point in data collection?
E.g., patient or animal research
Sequential analyses
when u correct for ur tests u these tests are more intercorrelated, u use the same data points and more added each time
adjusted it accordingly, then justify why you are doing that
What is the term used to describe using multiple measures and report the significant results, but don’t report all the measures/tests you used?
Uncorrected multiple testing
If u have a lower alpha value, look at the inflation
researcher degrees of freedom
- we can see how much variablility there is in ANOVA distribution flexibility the tresearcher has
E.g., predictors to include in a regression,criteria for outlier removal, adjustments for distribution properties, analysis method
-could change outliers description
but doing multiple of them could affect data
Study of different groups were given same data and research question and asked to analyse results: Some say that theres an effect some say there isn’t
Key point:
So dont exploit that by doing multiple tests which weakens our alpha
Researcher degrees of freedom
This table shows that researchers have different ways which inflate data
eg. multiple measures
not reporting all thing s you did
multiple things at the same time
This = higher likelihood of getting false positive
What is the term used to describe:
Positive (i.e., significant) results have, historically, been more likely to be published than negative results
A form of publication bias
File Drawer problem
