EDITOR’S NOTE: This reporting is written by Jennie Spotila
Next week, the NIH will convene a peer review panel to evaluate and score the applications submitted under the Data Center and Collaborative Research Center RFAs. NIH’s management of conflict of interest is key to understanding who is – and who is not – on the panel.
Peer review is an essential step in NIH’s funding decisions. Subject matter experts are appointed to review grant proposals, and provide NIH with an assessment of the quality of each one. Frequently, the subject matter experts have connections to the researchers applying for funding, and that is where the conflict of interest policy comes in to play.
If you look at the roster for the RFA review panel, you will probably notice a couple things. First, many of the biggest names in ME research are missing. No Cook, Davis, Hanson, Hornig, Jason, Klimas, Lipkin, Montoya, Younger, etc. That’s because these researchers are reportedly among those who submitted funding applications.
NIH’s conflict of interest policy prohibits an individual from serving on a review panel if he/she has a “major professional role” on one of the applications. Why? Because an applicant for funding wants to succeed, and should not be in a position where they can influence that outcome. The personal interest of succeeding is in conflict with a grant reviewer’s ethical obligation to evaluate all applications fairly.
There is another conflict of interest restriction at NIH, called the “out of room” conflict of interest. In this situation, a grant reviewer can serve on the panel, but has to leave the room when a specific application is discussed because of a potential (or even the appearance of) conflict of interest.
An “out of room” conflict of interest can arise in a number of specific circumstances, including:

• A person with a major professional role on an application is from the reviewer’s institution
• The reviewer has collaborated or published in the last three years with a person with a major role on an application
• The reviewer provided a letter of support for an application

Any close professional relationship can create the appearance of a conflict of interest, such as between a PhD advisor and doctoral student. In these and other similar cases, the reviewer in question must leave the room before the application is discussed. However, the reviewer can still participate in scoring the other applications at the same meeting.
This is where things get complicated for the ME review panel at the end of July. Multiple members of the panel overlap with applicants in publications and collaborations (some outside that three-year window), and one is from the same institution as one of the applicants.
Stringent enforcement of the conflict of interest policy is important to NIH, according to Dr. Richard Nakamura, Director of the NIH Center for Scientific Review. He says, “Reviewers are regularly reminded of their legal and ethical responsibilities to identify potential conflicts of interest.  Our Scientific Review Officers spell out COI policy in pre-meeting briefings and at the beginning of every review meeting. Our Internet Assisted Review system forces them to declare whether they have a conflict before and after the meeting.  This system also notifies our Scientific Review Officers when it identifies potential conflicts. Nonetheless, any knowledge of a field or the researchers in it means that there could always be some biases, so we must be vigilant and our goal is to manage conflicts so they are minimized.”
The burden of enforcing this policy falls on both the Scientific Review Officers and the grant reviewers. In any field, the web of professional relationships can be quite complicated, and Review Officers may not be able to identify each potential conflict. This is especially true when there is a large panel, such as the twenty-six members of the RFA panel this month. Grant reviewers have an ethical obligation to self-identify when they have a potential conflict of interest, as well.
Is it problematic to allow the “out of room” conflicted reviewers to participate in the remainder of the meeting? What prevents a reviewer from tanking competitors’ applications, or supporting others? Dr. Nakamura says, “Our Scientific Review Officers and study section chairs are trained to identify instances where a single reviewer is out of step with the others, and the group will be driven to discuss the reasons driving the different scores.  In addition, reviewers are encouraged to speak up if they see another reviewer attempt to skew a review.”

NIH does not publicly disclose the outcome of any conflict of interest discussion, so we won’t know if any of the ME experts on the review panel will be required to leave the room during discussion of a specific application. We also will have no transparency into the fairness of the scoring and reviewing process, and whether there was positive or negative bias at play.
Peer review is not the only step in NIH funding decisions. After scoring by the peer review panel, program staff evaluate and present a grant funding plan to the Institute Council. The Council makes their own recommendations, and the Institute Director makes the final funding decisions. An unfair advantage given to a grant application in the peer review stage does not automatically mean it will be funded.
It is important to remember that the integrity of the peer review process is critical for NIH. Researchers may complain about scoring and the quality of peer review, but for the system to work there must be general trust in the fairness of the process. A specific application may not get funded, but every application deserves an equally fair shot.  Disease communities, especially one that has been as historically underfunded as ME, also depend on the fairness of peer review in order to ensure that the best research ideas are funded.
There is a great deal at stake next week. The RFA represents the largest single investment in ME research in the last decade. Ten different groups applied for the two or three research center grants. There is a great deal of pressure on the peer review panel  – and on NIH – to get this process right. The public should expect rigorous and fair peer review that succeeds in identifying the best grants for funding.