Appendix C

Workshop Agenda

Enhancing Scientific Reproducibility in Biomedical Research Through
Transparent Reporting—A Workshop

September 25–26, 2019

National Academy of Sciences Building, Lecture Room
2101 Constitution Avenue, NW, Washington, DC 20418

DAY 1: September 25, 2019

Session Objectives:

• Highlight current efforts by researchers, institutions, funders, and journals to increase transparency in proposing and reporting preclinical biomedical research.

• Discuss the incentives, disincentives, challenges, and opportunities for researchers when it comes to transparent reporting of preclinical biomedical research (e.g., pressure to publish, institutional resources, training, funding).
• Discuss experience with implementation of policies to encourage transparent reporting across the biomedical research life cycle.
• Consider the role of stakeholders in supporting a cultural shift toward transparent reporting in preclinical biomedical research. For more information on cultural barriers as sources of non-reproducibility, see p. 58, p. 97, and p. 104 of the National Academies’ Reproducibility and Replicability in Science consensus study report.

Session Objectives:

• Consider lessons learned from institutional and/or field-specific best practices for increased transparency in reporting rigor elements (i.e., research design, methodology, analysis, interpretation, and reporting of results) that are generalizable across biomedical research domains.
• Consider available tools and best practices for increased transparent reporting that support researchers and are generalizable across biomedical research domains.
• Discuss the roles of educational institutions, professional societies, researchers, and funders in improving computational reproducibility (Reproducibility and Replicability in Science Recommendation 6-6).
• Discuss how funding agencies and organizations could invest in research and development of open source, usable tools, and infrastructure that support reproducibility for a broad range of studies across different domains in a seamless fashion, as well as in outreach to inform and train researchers on best practices (Reproducibility and Replicability in Science Recommendation 6-1).

Session Objectives:

• Discuss journal and funder assessments of researchers’ adherence to transparent reporting guidelines, including discussion of the effectiveness of checklists.
  • Highlight empirical assessments of checklist application from funders, journals, and researchers; and
  • Consider practical application and effectiveness of checklists and guidelines to encourage or require transparent reporting of preclinical biomedical research.
• Discuss how funders could require thoughtful discussion in grant applications of how uncertainties will be evaluated, along with any relevant issues regarding replicability and computational reproducibility (Reproducibility and Replicability in Science Recommendation 6-9).
• Discuss how journals and scientific societies could disclose their policies relevant to achieving reproducibility and replicability, and how journals could be encouraged to set and implement desired standards of reproducibility and replicability and adopt policies to reduce the likelihood of non-replicability (Reproducibility and Replicability in Science Recommendation 6-7).

Session Objective:

• Discuss opportunities for improving the consistency of reporting guidelines and requirements for rigor and transparency by journals, funders, and institutions across the biomedical research life cycle.

DAY 2: September 26, 2019

Session Objectives:

• Consider approaches to compare reporting of rigor elements proposed in grant applications to those included in publications.
• Suggest stakeholder actions to encourage transparent reporting and practical next steps toward establishing minimal reporting standards for preclinical biomedical research.