I'm currently going through papers for NSDI, and the PC chairs had to send out a post saying how the papers where they are co-authors would be specially handled. NSDI not only allows PC members to submit papers, but the chairs as well!
In the theory community, we generally don't do this. PC members can't submit papers, apparently to avoid any conflict of interest. This is, from what I've seen, unusual, even for CS, which is already unusual in the competitiveness and importance of conferences. Most networking conferences, for instance, allow PC members to submit.
Does conflict of interest really exist in these situations? I don't think so; generally, from what I've seen, it gets handled, and handled appropriately. People on the PC realize it's a competitive process, and understand when their papers aren't accepted. Often when PC members can contribute papers the process is double-blind, so nobody "knows" the PC-authored papers. While I understand allowing PC-authored papers is a risk, I don't think it's much of one. As a comparison point, theory conferences almost never use double-blind reviewing, and I think just the name of a "prestige author" on the paper has a significant effect on the odds of acceptance.
What's the upside? The biggest is that it makes it easier to have larger PCs. I have 20 papers to read for NSDI. I can't remember the last time I had less than 40 papers to review as a PC member for a significant theory conference.
Subscribe to:
Post Comments (Atom)
9 comments:
And the advantage of having larger PCs is of course better quality reviews (or, in case of theory conferences, having any reviews at all !)
Of course, of course, better quality reviews.
(OK, I admit, I was just thinking of less work for myself, but "better quality" does sound like a better reason...)
Crypto seems to be the outlier here: not only do Crypto/Eurocrypt (and many others) have anonymous submissions, but PC members are allowed to submit a bounded number of papers.
Another argument for allowing PC-authored papers is that it doesn't force someone to choose between not joining the PC or waiting 6 months for the next appropriate conference to which to submit their paper.
"I have 20 papers to read for NSDI. I can't remember the last time I had less than 40 papers to review as a PC member for a significant theory conference. "
I think for systems conferences, you must have at least a minimal implementation and some results ... this takes time :( . And this is what limits the number of submissions.
10-15 papers per PC is usual in most systems conferences ... our group gets that number of papers every conference in which my advisor is a PC member.
If the submission is double-blind, is there any problem at all for PC members to submit? Most of the systems conferences (except for VLDB and some DB confs) seem to have double blind submissions. Also, most systems such as EasyChair or CMT can be configured not to let a PC member see any part of review process for a paper he/she authored or have a conflict of interest with an author. In case the PC chair (who can see everything) is an author, I've seen them delegate the decision making to another PC member.
There is the conflict-of-interest.
And then there is the appearance of conflict-of-interest. When people get up-in-arms when their papers don't get into conferences, it may be better to avoid the latter, not just the former.
At many CS systems conferences, even top tier, the PC *chair* can submit papers! I'm glad the theory community is different. (I still wish they had double-blind.)
The STOC/FOCS model is to have a physical PC meeting with all PC members discussing a large fraction (if not) all submissions. The idea is that PC members will discuss not just the papers they were assigned but will also see the "bigger picture" of the entire conference.
I think allowing PC submission will make such a meeting very cumbersome with people constantly having to leave the room. (There are conflicts of interest even without PC submissions but in this case there will be much more.)
Another problem is that if you do allow PC submissions, and have a large PC composed (hopefully) of strong researchers, then you're likely to have a very significant fraction of the program comprised of PC authored papers. This doesn't look very good.
Anon 7: Having been at (multiple) PC meetings where people leave the room for conflicts (including PC members), I can state that it's functional.
As for having a PC filled with strong researchers, and then having many of their papers in the conference, why is that a bad thing? (Again, you could argue that that's how SIGCOMM works, but people think very highly of SIGCOMM...)
To add in to the comment on SIGCOMM. I think the perception is that it is a good conference but exceptionally cliquish. If you are in the group and can publish successfully, it likely seems great but for folks outside of it, it is incredibly difficult to break into. Case in point, take a walk back of last N years of SIGCOMM and examine how many authors are not members of the PC or students of members of the PC. The double blind process for SIGCOMM is largely regarded as a joke whenever I have seen the suggestion brought up with regards to reviewing outside of the the core SIGCOMM group.
Certainly, SIGCOMM papers are written quite well and the top papers are quite good (the XORs in the Wild being one of my all time favorites). The conference is often nicely run with a consistent quality of presentations (very few horrible presentations) which is nice from an attendee perspective. They also do a fantastic job shepherding students (only second to USENIX in my opinion) which is to be commended.
However, there are quite a few papers from familiar faces that are the yearly paper for group X's work which would have been rightfully thrashed at many other conferences by expert reviewers. My general perception from various PMs (DARPA) and industry folks is that the "SIGCOMM" crowd rates itself at a far higher degree of importance and quality than in reality.
The folks on the PC are not bad people and I know they mean well but if there was ever a conference that would benefit from PC members not being able to submit papers, SIGCOMM would be a poster child for that.
Post a Comment