NIH has been considering metrics to measure the value and the impact resulting from NIH research funding. In January 2017, NIH announced a preliminary index called Research Commitment Index, a measure of grant support that goes beyond measures of funding and number of grants. See: Research Commitment Index: A New Tool for Describing Grant Support.
This was followed in May 2017 by Implementing Limits on Grant Support to Strengthen the Biomedical Research Workforce, which announced plans to monitor the Grant Support Index (GSI) based on the Research Commitment Index. Francis S. Collins, Director, NIH, also released a statement that discussed the GSI: New NIH Approach to Grant Funding Aimed at Optimizing Stewardship of Taxpayer Dollars.
The GSI is a measure of grant support that does not solely focus on grant money, since differing areas of research inherently incur differing levels of cost. Instead, GSI assigns a point value to the various kinds of grants based on type, complexity, and size. Applications for NIH-funding that will support researchers who have GSIs over 21 (the equivalent of 3 single-PI R01 awards) will be expected to include a plan in their applications for how they would adjust those researchers’ existing grant load to be within the GSI limits if their application is awarded. While implementation of a GSI limit is estimated to affect only about 6 percent of NIH-funded investigators, we expect that, depending on the details of the implementation, it would free up about 1,600 new awards to broaden the pool of investigators conducting NIH research and improve the stability of the enterprise.
Dr. Collins also noted that the NIH will be seeking feedback from the scientific community on how to best implement the GSI.
On June 5, 2017, NIH announced that based on feedback received from council meetings and stakeholders, they will continue to explore a modified GSI and include additional measures for guiding NIH funding decisions. See: Following Up on Your Feedback on How to Strengthen the Biomedical Research Workforce.
An update about the modified GSI was presented at the June 8, 2017 meeting of the NIH Advisory Committee to the Director by Lawrence A. Tabak, Principal Deputy Director, NIH. See: Enhancing Stewardship: The Next Generation of Researchers Initiative. Among the points covered in the presentation included the need to develop short-term and long-term metrics for measuring the impact of NIH funding. One example of a short-term metric is a measure for grant support based on commitment, not dollars. Examples of long-term metrics include:
- Disruptions in prevailing paradigms
- Patents/licenses
- New technologies
- New medical interventions
- Changes to medical practice
- Improvements in public health
The update also included metrics that assess the influence of publications such as the Relative Citation Ratio (RCR) and the NIH tool to calculate the RCR: iCite, and discussed the need to develop additional approaches. Going forward, NIH will encourage independent analyses of metrics that can be used to assess the impact of the NIH portfolio and all actions will continue to be informed by stakeholder input.
Readings
NIH scraps plans for cap on research grants. Sara Reardon. Nature. June 8, 2017.
NIH abandons controversial plan to cap grants to big labs, creates new fund for younger scientists. Joceyln Kaiser. Science. June 8, 2017.
NIH scraps GSI, takes second swing at grant distribution plan. Julian Zhu. Biocentury. June 8, 2017.