23 - Reproducibility

Question 1

T or F: Scientific research enriches and improves our lives

Answer 1

T

Question 2

Both ________ and _________ are public goods, and both must be protected

Answer 2

knowledge and health

Question 3

Why most studies are false: (6)

Answer 3

1) Large studies are more likely to yield true results, yet we publish studies with small sample sizes
2) Willingness to publish small effect sizes; with a smaller effect size, findings are less likely to be true
3) Flexibility in designs, definitions, outcomes and analysis reduce chances that results will be true
4) Fishing expeditions to generate new hypothesis or explore unlikely correlations
5) Financial and other interests and prejudices reduce the likelihood that results will be true
6) Competition in research to produce positive findings especially in hot fields

Question 4

Past 15 years, efforts to characterize genetic alterations in human cancers = increased understanding of molecular mechanisms, hope this would translate to more effective drugs yet research — clinical success is very low

Answer 4

Amgen Study

Question 5

Talk about the reform towards statistical standards?

Answer 5

• avoid making decisions solely on the p=value
• be more rigorous and use p less than 0.01 vs 0.05
• present confidence intervals to better convey the range in which a variable most likely falls

Question 6

The p-value was never intended to be a substitute for ________________

Answer 6

scientific reasoning

Question 7

• Researchers should make their data available

- Researchers should use born-open data

Answer 7

Data Handling

Question 8

• Researchers should pre-register their protocols, filing in advance with an appropriate organization
• Adopt standardized schemes to outline methods and materials

Answer 8

Research practices

Question 9

What is the main goal of pre-registering your study?

Answer 9

-control the researcher who changes midway in a study based on preliminary results

Question 10

• Fields that rely too heavily on statistics (ie. like ours) to draw conclusions should better educate on the mis-use and mis-understanding
• Teach more holistic approaches as well as reasoning approaches to analyze data

Answer 10

Pedagogy

Question 11

ASN stands for:

Answer 11

American society of nutrition

Question 12

SRB =

Answer 12

Statistical review boards

Question 13

WEF Code of Ethics:

Answer 13

World economic forum

Question 14

T or F: Industry should advocate practices that minimize irreproducible research

Answer 14

T

Question 15

Work with academic to formulate standard research practices and protocols to promote _______________

Answer 15

reproducible research

Question 16

• Fund scientists’ effort to replicate earlier findings
• Fund researchers who strive to develop better methods
• Funding university chairs in reproducibility studies
• Establish a prize for most significant negative result in various disciplines
• Improve journalism that continues to uncover reproducibility crises

Answer 16

Private philanthropy

Question 17

To get funding in the US you have to go through the _______________ program

Answer 17

rigor and reproducibility

Question 18

NIH:

Answer 18

National Institutes of Health

Question 19

NIH Principles:

Answer 19

• Rigorous statistical analysis
• Transparency in reporting
• Data and material sharing
• Consideration of refutation
• Establish best practice guidelines

Question 20

What are the 4 areas of focus of the NIH:

Answer 20

• scientific premise
• scientific rigor (design)
• biological variables
• authentification

Question 21

• What is the basis for your proposed research study?
• Strengths and weaknesses of past research
• Have you scanned the entire literature or had a myoptic biased view? Are you systematic? Are you critical? CRAAP?

Answer 21

Scientific premise

Question 22

• Foundation for achieving robust and unbiased results
• Strict application of scientific method to design, method, analyses, interpretation, and reporting of results
• Randomization? Blinding? Replicates?

Answer 22

Scientific rigor

Question 23

• Sex, age, weight, underlying health affect health and disease
• Variables often ignored = incomplete understanding
• key now is sex
• explain how relevant biological variables are factored into research

Answer 23

Biological variables

Question 24

• How do you ensure the identity and validity of your biological and chemical reagents
• Do they differ lab to lab? Over time?
• Are people properly trained, and is training documented?
• Are lab notebooks maintained and reviewed?

Answer 24

Authentication of resources