Skip to main content

Discerning the Difference Between Exceptional and Good - The Power of Rigorous Peer Review

By Lorraine W. Egan, Outgoing President and CEO of Damon Runyon Cancer Research Foundation

Earlier this month, I attended my last meeting of the Damon Runyon Fellowship Award Committee before I retire. This meeting reaffirmed my strong belief that rigorous peer review works in choosing the best scientists to fund.

Damon Runyon conducts peer review at the highest level with the most respected scientific leaders reviewing the proposals of our applicants to decide if they merit funding. The committee members represent the broad spectrum of scientific disciplines and approaches funded by Damon Runyon. They also span generations, from highly respected senior scientists to those who have more recently become leaders in their fields. Most importantly, they know excellence when they see it.

These committee members spend hours poring over the hundreds of applications we receive from the most promising young scientists in the nation. Two members are assigned to each application to determine if it is worthy of consideration by the full committee. The committee meets in person to review the finalists. They debate the merits of each application vigorously and intensely. Each committee member then numerically ranks the applications, the combined scores are calculated, and the committee decides where the “cut-off” will be between those who receive the Fellowship and those who don’t.

So what happened at our last meeting that gave me full confidence in this approach? The review of two separate applications.

The first was an applicant who had received a high score from one initial reviewer and a lower score from the other. The higher scoring reviewer made a passionate argument describing the precise details of why the candidate should be funded, convincing his co-reviewer and the entire committee. The candidate will never know that a single committee member was her best advocate, enabling her to get a Fellowship that will help ensure a successful scientific career.

The second was just as compelling. Traditionally, the caliber of a scientist is judged based on his or her track record of publishing articles in scientific journals, with the prestige of the journal used as a barometer of the quality and potential impact of the research. Many have argued that there is an overemphasis on an applicant’s publications in top tier journals during peer review.  This month, one of the candidates had a remarkable track record of research and exceptional references from her mentors. But her graduate work was not yet published in a leading journal (it was under review by the journal, often a lengthy, painstaking process). Instead of rejecting the application on this basis, which often happens in peer review, the committee chose to fund her. As one committee member said, “We are fully capable of judging her work irrespective of whether it is published in an elite journal.”

The power of our peer review process is in discerning the exceptional from the good. We seek future leaders who will break the mold. Our committees take this mandate very seriously, spending tremendous amounts of time and effort to ensure that new generations of exceptional scientists are given the opportunity to pursue their ideas and, hopefully, make the breakthroughs we need to save lives. It has been a privilege to watch them at work.

 

Blog