I recently returned from an NSF (National Science Foundation) review panel. (Of course, I'm not allowed to say when, or for what.) On the way back, at the airport, I ran into a more senior colleague (from an entirely different field -- not EECS) coming back on the same flight. It came up that he had never served on an NSF review panel. He admittedly seemed a bit sheepish about it, but offered the excuse that there were more important things to do, and I will give him the benefit of the doubt and assume that he meant other service activities, not research and such. Essentially, his argument seemed to be that NSF panels were a waste of his time.
Having served on a half dozen or so NSF review panels, I have some sympathy for this point of view. In particular, being anti-travel, I'm not sure why we have to fly in for a two-day meeting to make decisions; it's expensive for the NSF and time-consuming for me. (It's not clear to me that decisions are any better because of the face-to-face meeting. Indeed, arguably, they can be worse, depending on the group interaction. But government process is government process, so unlike for PCs, not having the face-to-face meeting is not currently an option...)
However, despite the time, I've tried to make myself available when the calls for NSF panels go out, because I've always figured that if the NSF is paying my summer salary (which, in large part, they do), they have the right to ask for some of my time. Indeed, my gut reaction (without working through the arguments) is that it's objectionable that they don't actually require people who have received funding to serve on a review panel at some point during the term of their grant, though I suppose the implementation of that rule could become unpleasant. In short, my moral interpretation is that by taking their money, I'm part of their system, so when they come calling, I should respond, even without an explicit quid pro quo.
Even if one does not subscribe to this implicit understanding, I think it's important for us individually and as a community (including non-academics who don't get NSF funding directly) to do our service for the NSF, particularly in the way of review panels. In general, citizens should be keeping an eye out on how the government uses their money, and in this specific case, as scientists, we should be paying especially close attention to how the government is distributing research money to scientists in our own and related fields, and we should be attempting to influence the process to move in the right directions as much as possible. The panels give us some opportunity to influence the process, and these otherwise near-useless face-to-face meetings give us some opportunity to influence the NSF administrators who shape and make the decisions that affect us all.
So, with all due respect to my senior colleague, and any of you other slackers out there, get over yourselves, and go serve on an NSF panel every other year or so.
Subscribe to:
Post Comments (Atom)
5 comments:
I think there are several even better and more "pragmatic" reasons the serve on the panel:
1) you see how decisions are made, which can help you in your own grant applications. Not necessarily to "game" the system (although this too), but just to see what gets funded and what does not.
2) you can try to make sure that some morons who know how to game the system don't get the money (instead of stronger, but more self-conscious researchers).
3) most importantly, and related to 2), theory program is not the only program open to theory people. When you serve on other panels (such as cynertrust, nets, GENI, etc.), you want to make sure that theory gets proper representation and respect. Ultimately, this is what will improve the funding situation in theory.
One small item to remember: Make sure that you aren't getting paid on a grant during the days when you serve on an NSF panel. That counts as double-dipping.
This post raises an issue with "service" in general. For example, there is little motivation for senior academics to serve on program committees, and even less for them to do journal reviewing (as opposed to being editors).
Tenure and promotion committees do pay lip service to "service", but it is interpreted very broadly and is, in general, much less important than good research.
In some cases, such as editorships and a few program committees, the visibility provided by a service position is a partial reward. However, NSF panels and reviewing definitely do not have visibility benefits.
How can we create an incentive to encourage low-visibility service activities?
This is a bit like creating an incentive for people to be nice. Studies (and experience) tell us that humans are nicest when somebody is watching, since we use kindness mainly as cement for social structures (families, communities, etc). This doesn't work at the NSF, though, if the program manager changes every three years (why create a bond with someone who is leaving?). And it doesn't work with journal reviewing if nobody except the editor knows you're doing it.
As the post said, requiring one panel membership per grant for people with NSF grants would be problematic. A hard constraint like that would drag the program manager into creating a panel schedule that uses everyone exactly once, while simultaneously maximizing the quality of reviewing in each sub-area... sounds NP-complete.
But you could imagine a looser coupling: the NSF could keep a record of invitations made to serve on panels vs invitations accepted, or simply panels served on per dollar/year/unit of grant money. The program director could then be allowed to explicitly take this institutional memory into account when deciding which grants to recommend. Of course, this system is open to abuses of different kinds, but one could imagine guidelines that would keep things reasonable ("if you've *never* served on a panel, and you have X or more grants...").
In the crypto community, there is an informal version of this that is used to help program chairs put together their committees. John Black at Colorado maintains a list of how many papers different authors have at CRYPTO/EUROCRYPT authors, together with how many times each author has served on a PC. Program chairs can look for underused researchers (and avoid abusing the nice ones who can't say no!). Perhaps the NSF could use something similar internally.
All this may be far too naive to work in reality. I have never served on an NSF panel myself (I just took up a faculty position) so my ignorance about the process is, um, significant.
-Adam Smith, Penn State
(NB: I don't know if John created his list with this in mind, but I know from talking to program chairs that it gets used this way, at least occasionally).
Your thoughts on serving on an NSF panel are interesting. I served on one last year for the first time, having finished graduate school several years ago. My motivation was essentially to gain insight into the whole process of how proposals are selected.
I found that the discussion there actually did drastically change the course of what would have happened to a significant fraction (1/4) of the proposals. In most of these cases, I doubt that the same result would have occurred if each reviewer just posted their comments online and read those from others (in which case I think people would be unlikely to alter their perspectives much).
You can telecommute to NSF meetings. I did it last year. They put a speakerphone in the middle of the table and I called in with skype. It was reasonably successful, although I had to often ask people there to speak more loudly.
Post a Comment