Suppose you're the PC chair, and someone who has submitted a paper asks you NOT to have the paper reviewed by a specific person on the PC. Do you honor that request?It's an interesting question -- that I hope others will comment on -- though as a default my answer would be yes. I certainly have had run-ins of sufficient severity with various people through the years that I would not want (and would likely ask) for them not to review my papers if the issue came up. Looking at it from the other end, if I am on a PC and those people submit a paper, I make sure not to review them. (Usually it is sufficient simply to rank them low on my list of desired papers, but I have also told PC chairs in advance I would not review certain papers if they seemed likely to head my way.) It is not that I actually think I couldn't give a fair review; it's that I think it's inappropriate, in such a situation, for me to give a review in the first place. If as a PC member I have the right (actually, I would say, a responsibility) to refuse to review a paper under such circumstances, it seems fair that a submitter can ask for a specific PC member to not review a paper as well.
Context does matter, though. In the networking conferences I have served on, this is standard -- PC members and submitters are expected to list their conflicts. Indeed, one issue that seems to have arisen lately is that there is suspicion that some people submitting papers are abusing this right, listing people as conflicts when they are not because they are known to be "challenging" reviewers. While I'm skeptical this sort of gamesmanship gains anything (challenging reviewers are usually calibrated appropriately at the PC meeting), it is a concern that once you open the door to such requests, you may need to make sure the privilege isn't abused.
For theory conferences, where many people seem painfully unclear on what "conflict of interest" even means, I'd grant such a request as a matter of course.
7 comments:
Michael,
Regarding the abuse of right to list non-preferred reviewers: There are some reviewers that are known (e.g., by serving with them on other PCs) to trash papers without sufficient reasons.. IMO, even if there is no conflict of interest with these reviewers, one should list them as non-preferred reviewers.
Allowing people to list "known trashers" as non-preferred reviewers, even when there is no specific conflict with that reviewer, seems to be just one more way for the in crowd (who know about which reviewers to avoid) to get an edge over outsiders to the field (who don't). For that reason I think it should be discouraged.
Also, certain people may be known to like or dislike certain lines of research, and to rate submissions accordingly. If I'm writing about k-anonymization and reviewer X is known to hate k-anonymization (but I've had no past personal interactions with X) then I don't think it would be appropriate for me to request that my paper not go to X.
But that's not the same as a situation where research teams have developed an animosity towards each other that extends to a personal level rather than being only about an evaluation of the research. Requests to avoid reviewers for that sort of reason should be honored.
I haven't had a request about a particular PC member but as a PC chair I have had a request not to send something to a particular sub-referee. I think that it is much simpler in this case. If one is talking about the core PC then it is hard to avoid letting the PC member know that there is a conflict of some sort with the paper so this can't be something secret.
BTW: In a related domain, in the bio section of NSF proposals one can indicate an exclusion list on potential reviewers over and above the official conflicts of interests for co-authors and such.
I have a different view on this. My view is, if an author asks me NOT to assign his/her paper to a particular reviewer on the program committee, the first step is that I ask the author why. My default starting position would be "no", but given a sufficiently good reason (e.g., a true conflict not disclosed by the reviewer) I would grant it.
I got one of these requests at one PC I co-chaired. When I asked the authors why, the response was "this PC member has repeatedly trashed our paper, often for bad reasons", and went on to list one or two examples. I thought their request was not out of line, but in the end I decided not to honor their request regarding reviewer assignment, but instead to monitor very closely that reviewer's reviews and actions. I did not notify the reviewer. In that particular case, the reviewer behaved fine and there was no need to adjust the reviewer assignments.
The new HotCRP system allows authors to nominate conflicts. Some program chairs don't review those at all, and just take them for granted. I have seen one or two instances where authors listed a PC member as a conflict, yet where none of us were able to figure out the conflict and we were fairly certain that this was a case of gamesmanship. (That PC member was also a member of a prior PC where the paper had been previously submitted and rejected.) I find this kind of gamesmanship highly distasteful, and I think it is incumbent on program chairs to take steps to prevent it.
I have served as a secondary reviewer for a PI, where we had a specific mandate to reject all papers on a hot topic that conflicted with PIs traditional way of doing things. I really thnk there should be a thorough data analysis of decision patterns, and these patterns should be used to make fair recommendations of reviewers. With all the stuff we do, very little analytic information is used or logged in peer review.
Unless the conlict is purely personal, I don't think the PC should respect the request.
11011110,
RE:
Also, certain people may be known to like or dislike certain lines of research, ... then I don't think it would be appropriate for me to request that my paper not go to X.
Now replace "conference submission" with "grant proposal", how would that change your position? For me, I don't care about this issue for conference submissions so much, there is always the next conference, but going without summer or travel money for a year, that is significant.
Post a Comment