Guest Lecture: Resources for Reproducible Research

Question 1

What is the point of replication?

Answer 1

• strengthens findings

- reduces errors, omissions and fraud

Question 2

Is it always possible to replicate studies?

Answer 2

not always possible

• time
• money
• unique circumstances (starvation studies, war)

Question 3

What is the purpose of reproducibility?

Answer 3

• Provides a minimum standard bridging the gap between replication and blind trust of a paper’s findings
• Make analytic data and code available
• Allows others to attempt to reproduce the results from the data obtained

Question 4

Replicability vs Reproducibility

Answer 4

• Replicability
o New researchers
o Original methods
o New data
o Independent analysis

• Reproducibility
o New researcher
o Original data
o Original analysis

Question 5

Explain the Duke Clinical Trials

Answer 5

• Duke University announced that it had an advanced, experimental treatment that would match chemotherapy to a patient’s own genetic makeup -> the holy grail of cancer care.
• Dr. Anil Potti = head researcher
• Published in top medical journals including the New England Journal of Medicine
and Lancet Oncology between 2006 and 2009.
• Research generated huge computer files of data.
• Kevin Coombes and Keith Baggerly began analyzing Dr. Potti’s data to verify his
results-> Data analysis was heavily flawed

Question 6

What are some of the limitations of reproducible practices

Answer 6

• Requires increased understanding of different coding languages and software programs
• Not currently a standard part of training

Question 7

What are the tools to use to make your study transparent and reproducible?

Answer 7

• Version Control Systems
o Git

• Literate Programming
o Rmarkdown, Knitr
o Jupyter Notebook

Question 8

What are the advantages of version control?

Answer 8

• Help manage changes to files
• Organization/declutter
• Always able to revert to past versions of documents
• Ability to experiment with new ideas and analytical techniques
• Creates a log of all activity and changes

Question 9

What are the practical advantages of Git?

Answer 9

• Each version of the file can be committed to storage
• Ability to retrieve any version of your file
• More comfort in experimenting with new
ideas
• Creates a log of all activity and changes

Question 10

What is literate programming?

Answer 10

• An author’s data analysis can become very confusing

• Solution: Format data analysis to make it human readable and computer readable
o Computer readable-> Programming Language (Python, R, etc.)
o Human readable-> Documentation Language (Markdown,
LaTeX, etc.)

• Tools:
o R Studio: RMarkdown and Knitr
o Jupyter Notebook