I'm serving this year, as I have every year I've been at Harvard, on the Hoopes Prize Committee for Harvard science theses. This is one committee I don't mind serving on at all. I get to read (or at least skim) a number of senior theses in a variety of areas -- mostly computer science, math, applied math, and engineering, but usually I have a couple in fields more removed from my natural interest, like psychology, astronomy, evolutionary biology, and even geology. (I admit, I eagerly avoid the very large number of theses in biology and biochemistry; I'm sure my colleagues in related fields can better deal with the jargon there.) The worst of them is generally quite interesting; the best are high-quality publishable research. The committee sends in the reviews, and meets for lunch to work out who gets the prizes.
The committee exemplifies all the possible problems one can imagine in committees (that have arisen in the context of discussing PC meetings on this blog).
1. Only 2 reviews per work.
2. No guarantee the reviewer is an "expert" in the specific subarea.
3. Widely varying interpretations on what the scores mean. (In past years we've been told to aim for an average of 3. Most submissions are quite good; novice reviewers tend to end up with an average closer to 4.)
4. Widely varying interpretations on what the prize should be for. It's a writing prize, for the sciences. How should one judge math theses, which, arguably, if written entirely correctly, nobody in the room might be able to properly read in the time available? How much weight should be given to scientific novelty versus the writing itself?
5. Increasing numbers of submissions per reviewer. (When I started, I think we each had six theses to read; this year, it has hit ten. The barrier to entry is low, so perhaps there's an increasing tendency to submit...)
And so on... in some sense, it's great training for program committee work.
Despite the potential problems, it generally works out well. Some years there are many disagreements that have to be discussed; most years, surprisingly few. Overall, there's always the feeling that the student work is quite amazing, and that we at some point have to draw a line and just pick what we think is best. I wish all committees left me feeling as positive when I left the room.
Subscribe to:
Post Comments (Atom)
4 comments:
Michael,
I'm trying to analyze why you had a better time in the Hoopes Committee (HC) than some of the PCs. This might not apply fully but anyway I think there are two factors:
1. The Hoopes prize carries, from the point of a CS PhD, less weight than say a top conference paper acceptance. If I were in the HC, the pressure on me would be less. This ties in with Matt Welsh's blog on devaluing conference papers.
2. HC members are detached from the submissions, in the sense that they are not colleagues of those who submitted, nor did they submit any of their own work. In other words, there is no (significant) personal gain to be made. I'm wondering, for conferences, whether the PCs should be restricted to those really senior folks with nothing more to lose or gain. Unfortunately, the load would be unthinkable since the pool is pretty small.
Anonymous :
I think both your points are relevant. There's definitely less pressure and more detachment than at a regular PC, although certainly people have opinions and argue for them. I wonder if we could capture a bit of the spirit of the HC in a PC somehow.
Any idea when this year's Hoopes winners will be announced? Thanks!
Any idea when this year's Hoopes winners will be announced? Thanks!
Post a Comment