When I am asked to review a grant proposal by a program director at NSF (doesn't matter which program), I almost always say yes. There have been a few cases that I can remember when I was sent a proposal quite far out of my field of expertise, and I said no to those, but it is rare that I am asked to review proposals that I have no reasonable basis to judge. This is not because I am so awesomely expert in so many things but because the program directors do an excellent job of targeting reviewers.
Of course, saying yes to review a proposal on a topic that you find somewhat-to-very interesting doesn't mean the proposal will be good (or very good). And, unless you have reviewed proposals and/or manuscripts by the PI/coPIs before, you may not know in advance whether it is likely to be well written or not.
So it can be hard to predict what the review-experience is going to be like, even if you have some basic information at the time of agreeing to the review. I am thinking in particular of a proposal that I reviewed not long ago. The proposal was on an interesting topic, the writing was mostly good in a technical sense, and the proposed work involved what seemed to be the appropriate approach (methods etc.), and yet.. it was extremely difficult for me to read more than a page without flinging it aside (or, in reality, the electronic equivalent of flinging, as I was reading it as a pdf).
Why? Because it was so so so so boring to read this proposal. It was very painful.
Is my attention span getting shorter as I get older and busier and more jaded? Do I need lots of insightful illustrations and formatted text telling me what is important? Do I have to be grabbed in the project summary and/or first paragraph of the project description or else I have trouble being interested in the proposal? Answers at the end of this post.
Eventually I did read the proposal all the way through, of course, and I even re-read what I thought were some critical parts, but it was a slog from start to finish. I had to work really hard to figure out that the proposed work was of some interest and significance. The PI did not highlight the significance, but it was in there, somewhere.
I have written about related issues before: How hard do/should we as reviewers work to see the good (or excellence) in a proposal that has some technical flaws? In many cases, the flaws are in the writing, but in this case it wasn't so much how the sentences were constructed (each one by itself was fine), but in how the proposal as a whole was framed. If I had to work so hard to extract the interesting essence of the proposal, does that mean I will/should rate the proposal lower than I would if the proposal were technically excellent (in addition to containing interesting proposed work)?
(I am ignoring, in this particular discussion, the issue of giving early-career people a bit of a break; i.e., not expecting proposal perfection for those who haven't written many or any before.)
I fear the answer is yes, in a way. There was no way I was giving this proposal a rating of Excellent (the highest rating for an NSF proposal). If the proposal itself had been really well put together, showing more vision in general about the work and how to explain it, it could possibly have been in the Excellent range, but given what I saw in the proposal: no. Ignoring the fact that some of the shortcomings in understanding the proposal may have been my own, can I assume that the dense, complicated, non-linear, dry presentation of the proposed work is an indication of how the research itself will be undertaken, understood, and communicated? I decided the answer to that was also yes, and hence my overall positive but not-Excellent review.
That might sound harsh, so let me say that my first impression of the proposal was even more negative, but my opinion improved with more careful reading and thought. The proposed work is good, even very good. In the end, my rating was quite positive, and (I hope) backed up by what I wrote in my review. So:
Is my attention span getting shorter as I get older and busier and more jaded? Yes
Do I need lots of insightful illustrations and formatted text telling me what is important? Maybe -- it would be nice, anyway. Of course, writing "The hypothesis to be tested is ..." in bold italics underlined and set off with an indented bullet, surrounded by spaces and maybe some nice subtle shading and prominent on page 1 of a proposal doesn't guarantee you won't get a reviewer who writes "It is not clear to me what hypothesis is being tested", but one can reduce the chances of getting comments like this.
Do I have to be grabbed in the project summary and/or first paragraph of the project description or else I have trouble being interested in the proposal? Not this time, but I think in general, yes. I eventually forced myself to sit down and read the whole proposal carefully, painful though it was to do so, and saw some interesting things buried in the dense text, but I wouldn't want to risk this myself when writing a proposal. Perhaps my review would have been more positive if I hadn't found it a huge chore to read the proposal. An annoyed, weary, bored reviewer is probably best avoided.
Of course, saying yes to review a proposal on a topic that you find somewhat-to-very interesting doesn't mean the proposal will be good (or very good). And, unless you have reviewed proposals and/or manuscripts by the PI/coPIs before, you may not know in advance whether it is likely to be well written or not.
So it can be hard to predict what the review-experience is going to be like, even if you have some basic information at the time of agreeing to the review. I am thinking in particular of a proposal that I reviewed not long ago. The proposal was on an interesting topic, the writing was mostly good in a technical sense, and the proposed work involved what seemed to be the appropriate approach (methods etc.), and yet.. it was extremely difficult for me to read more than a page without flinging it aside (or, in reality, the electronic equivalent of flinging, as I was reading it as a pdf).
Why? Because it was so so so so boring to read this proposal. It was very painful.
Is my attention span getting shorter as I get older and busier and more jaded? Do I need lots of insightful illustrations and formatted text telling me what is important? Do I have to be grabbed in the project summary and/or first paragraph of the project description or else I have trouble being interested in the proposal? Answers at the end of this post.
Eventually I did read the proposal all the way through, of course, and I even re-read what I thought were some critical parts, but it was a slog from start to finish. I had to work really hard to figure out that the proposed work was of some interest and significance. The PI did not highlight the significance, but it was in there, somewhere.
I have written about related issues before: How hard do/should we as reviewers work to see the good (or excellence) in a proposal that has some technical flaws? In many cases, the flaws are in the writing, but in this case it wasn't so much how the sentences were constructed (each one by itself was fine), but in how the proposal as a whole was framed. If I had to work so hard to extract the interesting essence of the proposal, does that mean I will/should rate the proposal lower than I would if the proposal were technically excellent (in addition to containing interesting proposed work)?
(I am ignoring, in this particular discussion, the issue of giving early-career people a bit of a break; i.e., not expecting proposal perfection for those who haven't written many or any before.)
I fear the answer is yes, in a way. There was no way I was giving this proposal a rating of Excellent (the highest rating for an NSF proposal). If the proposal itself had been really well put together, showing more vision in general about the work and how to explain it, it could possibly have been in the Excellent range, but given what I saw in the proposal: no. Ignoring the fact that some of the shortcomings in understanding the proposal may have been my own, can I assume that the dense, complicated, non-linear, dry presentation of the proposed work is an indication of how the research itself will be undertaken, understood, and communicated? I decided the answer to that was also yes, and hence my overall positive but not-Excellent review.
That might sound harsh, so let me say that my first impression of the proposal was even more negative, but my opinion improved with more careful reading and thought. The proposed work is good, even very good. In the end, my rating was quite positive, and (I hope) backed up by what I wrote in my review. So:
Is my attention span getting shorter as I get older and busier and more jaded? Yes
Do I need lots of insightful illustrations and formatted text telling me what is important? Maybe -- it would be nice, anyway. Of course, writing "The hypothesis to be tested is ..." in bold italics underlined and set off with an indented bullet, surrounded by spaces and maybe some nice subtle shading and prominent on page 1 of a proposal doesn't guarantee you won't get a reviewer who writes "It is not clear to me what hypothesis is being tested", but one can reduce the chances of getting comments like this.
Do I have to be grabbed in the project summary and/or first paragraph of the project description or else I have trouble being interested in the proposal? Not this time, but I think in general, yes. I eventually forced myself to sit down and read the whole proposal carefully, painful though it was to do so, and saw some interesting things buried in the dense text, but I wouldn't want to risk this myself when writing a proposal. Perhaps my review would have been more positive if I hadn't found it a huge chore to read the proposal. An annoyed, weary, bored reviewer is probably best avoided.