Mandatory Registration: Participants must register their intent to participate by 11:59 PM Eastern Time on March 3, 2025.  Register for the challenge here. 

This Challenge aims to incentivize and reward data sharing excellence, promoting a new metric for assessing how effectively researchers share valuable data, driving a culture of openness in science.

yet2 is pleased to partner with NIH as the Power Connector to support the S-Index Prize Challenge.  For questions or further information, please don’t hesitate to submit your questions in the challenge Clarification Board or reach out to amclaren@yet2.com.

The Data Sharing Index (S-index) Challenge, led by the National Eye Institute (NEI) with contributions from multiple National Institutes of Health (NIH) Institutes, Centers, and Offices (ICOs), aims to incentivize the creation, development, and validation of a quantitative data sharing index. This Challenge is inspired by the h-index, which was developed by J.E. Hirsch in 2005 to evaluate research performance based on the impact and number of publications rather than the journal prestige or authorship order. Its simplicity, inclusivity, and ease of calculation led to the widespread adoption and integration into databases like Scopus, Web of Science (WoS), and Google Scholar. The S-index Challenge draws on the key factors contributing to the h-index’s success - simplicity, early adoption, education, and accessibility - while striving to incentivize and reward effective data sharing.

The S-index aims to incentivize and reward effective data sharing practices by assessing how effective a researcher is in sharing their research data in a way that has utility for future study. This metric will incorporate factors such as adherence to Findable, Accessible, Interoperable, and Reusable (FAIR) standards, data timeliness, completeness of annotation, frequency of utilization in other studies, and related publications and patents. This metric seeks to recognize and incentivize the efforts of individuals and teams in sharing data, rather than just the value of the data itself.  

While the enthusiasm for a cultural shift towards data sharing holds promise for promoting rigorous and reproducible research, encouraging data reuse, and accelerating scientific progress, it’s important to recognize that simply sharing data does not guarantee its value as a resource. Effective incentives are essential to motivate researchers to share high quality data that is well-described, well-documented, and in machine-readable formats, accessible through easily navigable platforms. Relying solely on broad data sharing policies or researcher altruism may not suffice to ensure widespread adoption of these best practices.  

Introduced in January 2023, the NIH Data Management and Sharing (DMS) Policy encourages the dissemination of scientific data to enhance reproducibility, data reuse, and scientific progress. This policy positions the research community at a unique inflection point to elevate FAIR data practices, potentially transforming the sharing of research data to enhance reproducibility, promote data reuse, and spur scientific advances.

The DMS Policy defines scientific data as the recorded factual material commonly accepted in the scientific community as being of sufficient quality to validate and replicate research findings, regardless of whether the data are used to support scholarly publications. Importantly, scientific data do not include laboratory notebooks, preliminary analyses, completed case report forms, drafts of scientific papers, plans for future research, peer reviews, communications with colleagues, or physical objects like laboratory specimens. Additionally, the DMS Policy defines data sharing as the act of making scientific data available for use by others, including the larger research community, institutions, and the broader public. This is typically achieved through established repositories, ensuring that valuable data can be accessed and utilized for further research and analysis.

The DMS Policy mandates data sharing, but the scientific community would also benefit from additional incentives that align with the policy’s goals and support the cultural shift towards greater openness and collaboration. A significant challenge we face is that generating well-annotated, accessible data is time-consuming and may impact investigators’ ability to publish as frequently as their counterparts who are not sharing. Currently, there is no way to recognize this crucial contribution, particularly for early-stage investigators who rely on publication productivity for career advancement. An S-index could be disseminated and used by the NIH and, if accepted by the scientific community, incorporated into prominent scientific search engines and university promotion and tenure committees. This would incentivize and evaluate high-quality data sharing, effectively measuring researchers’ contributions to science. It would also contribute to a cultural shift by emphasizing that generating and sharing “high-quality” datasets, thoroughly annotated and contextualized for use by others, is a valuable standalone outcome of research studies rather than merely an adjunct to publications. Furthermore, such an index would enable the direct linking of downstream research and health outcomes to the original datasets, thereby more equitably acknowledging the community-driven aspect of scientific discovery.

To achieve this goal, NIH is employing an open innovation approach and utilizing challenge (a.k.a., prize competition) authority. Challenges allow a high degree of creativity in generating ideas and flexibility in integrating user-friendly requirements. Additionally, it enables agile testing and iterative revisions to develop a robust solution effectively. A Challenge fosters broad and diverse public participation by enabling individuals, teams, and entities to compete for prize awards alongside institutions and organizations. Unlike the substantial infrastructure necessary to apply for and receive federal grants or contracts, participants in a Challenge may face lower entry barriers. This approach also enables the NIH to tap into innovators nationwide with diverse skill sets and backgrounds, including academic, community, and industry partners who may not typically contribute to NIH-funded science. Through a phased approach and the potential for multiple winners, we can enhance adaptability and optimize opportunities for participation and success.  

Goal: The S-index Challenge aims to advance the NIH mission by actively promoting data sharing and developing a robust metric to reward exemplary data sharers. This initiative seeks to incentivize and recognize researchers who excel in making their data accessible, ultimately fostering a culture of openness and collaboration within the scientific community. Additionally, this effort could result in the creation of multiple S-index metrics, enabling and helping to evaluate the metrics across different data domains.

National Institutes of Health Office of the Director (Office of Behavioral and Social Science Research, Office of Data Science Strategy, Office of Extramural Research, Office of Portfolio Analysis, & Scientific Data Sharing Policy Division), BRAIN Initiative, National Cancer Institute, National Institute of Biomedical Imaging and Bioengineering, National Institute on Drug Abuse, Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institute of Environmental Health Sciences, National Institute of Mental Health, National Institute of Neurological Disorders and Stroke, and National Library of Medicine/National Center for Biotechnology Information.

This Challenge is open to the public, including researchers, healthcare professions, data scientists, informaticians, and anyone interested in expanding data sharing in research. The competition aims to stimulate innovation and advance open science principles.

The S-index Challenge will be conducted in two phases, with the first phase laying the groundwork for the second.  

Phase 1: Proof of Concept Participants will engage in brainstorming and idea generation, culminating in the submission of a Proof-of-Concept proposal. These proposals will compete in the first round, with up to 6 finalists selected to advance to Phase 2.  In addition, up to 4 honorable mentions will be awarded, but will not be eligible for Phase 2.  

Phase 2: Refinement and Implementation Finalists from Phase 1 will compete for the Grand Prize(s) by submitting a comprehensive proposal addressing specific criteria. This phase will determine the preferred S-metric at the NIH S-index Innovation Event.  

The National Eye Institute (NEI) is conducting this Challenge under the America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science (COMPETES) Reauthorization Act of 2010, as amended [15 U.S.C. § 3719].

NEI’s mission encompasses research, training, health information dissemination, and programs related to blinding eye diseases, visual disorders, and mechanisms of visual function. Through this Challenge, NEI aims to explore and implement innovative methods in data science and informatics to enhance and incentivize data sharing. This initiative aligns with NEI’s goal of fostering innovation and driving scientific progress by engaging the broader community. The Challenge will identify and promote effective data sharing approaches, contributing to improved health outcomes and guiding future initiatives and policies.

For questions or further information, please submit your questions in the challenge Clarification Board or via email at: amclaren@yet2.com.