Getting a Fair Shake?Gail is an Associate Professor who has published extensively and rightly considers herself one of the leading experts in her field. She knows the drill for developing and submitting a proposal for funding. Her most recent proposal is based on considerable prior research that she and colleagues have conducted, and it employs methodology that she knows will produce valid findings.Gail is anxious about her proposal being reviewed and funded. Prior to submitting, she ran it by a network of capable colleagues with whom she works collaboratively to develop and critique ideas. They all share the same dedication to medical research with important long-term implications for public health, as well as short-term implications for federal policies and funding.When Gail finally receives the funder’s evaluation, she is happy to see that she received a high score on her proposal. However, the score is below the cut-off point at which it can be funded. Gail turns to see what objections and concerns the reviewers had with her proposal. She plans to revise and resubmit for the next funding cycle.Gail is flabbergasted to read what the reviewers said. In their feedback, they find that her methods are “flawed”. And they suggest changes to the study design that Gail is certain would significantly compromise the quality of the science as well as its impact on the development of her field. They clearly did not understand her approach, which is highly novel.Gail is considering re-submitting the proposal employing the reviewers’ suggested methodology, then actually doing the study as she proposed it in the first place, explaining that “pilot testing” indicated that her methodology would work best.However, Gail worries that this is deceptive. And who knows if the same reviewers would be evaluating her resubmission? Perhaps she is better off proposing her original “flawed” methodology—with a direct response to reviewers—in hopes that she will receive new reviewers or that her reviewers will be unusually open minded. Gail has two months before the revised proposal is due. She feels caught between a rock and a hard place.What should Gail do?
I agree with the previous post. If reviewers misunderstand me, it is almost always because I failed to describe my plans adequately. This case study seems to fall within this category because apparently all the reviewers misunderstood. Alternately, "Gail" could incorporate the reviewers' suggestions as an alternative if her approach fails. Who knows, they could be correct? In any event, the difference of opinion does need to be acknowledged, but to simply change her approach to appease the reviewers is likely to backfire if for no reason other than the same reviewers may not review the revision.
The situation is completely different if the reviewer or a reviewer understands but disagrees with the applicant. This situation can occur when two or more schools of thought about mechanism may exist, and the proposal has been reviewed by someone from the other camp. I believe the best way to handle this situation is to devise a head-to-head test of both competing theories, or at the least acknowledge the difference of opinion and state that although one is aware of different theories, the one being proposed is the favored, and if it is disproved, then at least it will be taken off the table.
Should You Always Be Responsive to Peer Review? ABSOLUTELY!
The ugly fact is that Reviewers often have an unsatisfactory base of knowledge to review groundbreaking applications, and basically lack the tools to understand the more innovative proposals.
However, these same reviewers have huge egos.
There are no benefits to arguing with the review panel, even when Reviewers are patently wrong. Over the years of submitting grants to the NIH, this one truth emerged rather quickly: Reviewers do not appreciate any contradiction. I am yet to hear of a single incident where the applicant successfully argued with the reviewers and was awarded the funds to conduct the disputed studies. It is far easier to acquiesce to the reviewers suggestions, and after receiving funding, conduct the suggested experiments, and armed with the empirical knowledge that the reviewer panel got the wrong end of the stick, go back to the correct originally proposed research design. The cost of this approach is some wasted time and money, however, on the plus side, you have funds to conduct important studies.
I agree with the first comment. Even if you think that you have been crystal clear, you need to take responsibility for a miscommunication. If the proposed methodology is the best approach, you should stick to your guns, but re-write until your reasoning is apparent to even a novice to your field. The original reviewers must have liked the proposal a great deal, or the score wouldn't have been as high as it was.
The above response is spot-on. As grant writers, we are teachers, self-advocates, and our own and only sales person. It is our task to convince the reader, period. If the reader doesn't "get it," we failed to convey our message. The reader can be less versed in the field, even less intelligent. Doesn't matter. Complaining about such is like saying that none of the students in our classes "get it" even though we are the leader.... My mentor taught me the magic phrase for response/resubmission, "We were unfortunately unclear in our [presentation/explanation/prior description]..." and from there to go on to add a figure, and make a table or bullet points or whatever it takes to make it clear to an eight grader, but never bark. The reader essentially holds the funding in his or her hands, so treat it like a job interview.
I agree with the previous comment. If the reviewers don't "get it," it is always the applicant's fault...she should explain it better instead of whining. Reviewers are almost always a bit outside of the specific field of the applicant's expertise and need more explanation than one might give a collaborator.
And, they are doing a lot of work for virtually no pay, so the applicant should make it easy for them, even give them some enjoyment in reading the application. I reviewed a grant once that said in the usual tedious section about resources that his office of so many square feet had a great view of a parking lot. Humor helps.
The worst approach would be the bait-and-switch: submit a proposal for method B and then actually use method A.
The reviewers "suggest changes to the study design that Gail is certain would significantly compromise the quality of the science as well as its impact on the development of her field."
I'm pleased Gail is proposing a novel approach. Still, I'd like her to be more open-minded to alternate points of view. Why can't she objectively address a professional critique from peers? Possible answers: she's under pressure to raise grant money, she has a healthy ego, and we live in a winner-take-all society.
If the root problem isn't her poor communication (see comment above) and instead rests on true scientific differences, perhaps there's value in trying both methods A and B and comparing the results. Based on the facts though, I wouldn't trust Gail to make the comparison since she seems too wedded to method A.
She may be barking up the wrong tree with her novel approach. Gail alone does not define quality science.
Even if "her" field (as if she owns it) eventually suffers negative impact, science overall may benefit.
You do have choices in numbers. Peer review is a good process in the developement in research. However, you are not confind to just one reviewer. Some studies may require more than one reviewer to truely help the researcher best with ethical or moral ideations.
Many doctoral programs use preparation of a grant proposal and response to critique as a qualifying examination. Students could encounter the same situation. In using this example, the relevance to the student in the near term and not just later in an academic career (and most students do no end up in academics) should be drawn.