Consider your favorite conference(s). Do you think they're accepting too many papers, too few, or just right? (With SIGCOMM decisions out, STOC/EC/CCC coming up, etc., it seems like a timely question to ask.) Keep in mind if you think conferences should be accepting more papers, it means longer conferences or more parallel sessions. When thinking about adding papers, there are different levels you could imagine -- 10% bigger, or 50% bigger.
I'm sure with the SIGCOMM acceptances/rejections out some people think that SIGCOMM could accept more papers. (33 were accepted; there was a nominal target of around 36.) I'm sure some other people like the bar to be kept high. Increasing paper acceptances by about 10% would get us to the nominal target, a minimal change. Increasing paper acceptances by about 50% might change the nature of the conference non-trivially, but I'm sure there are arguments on both sides about whether that would be a good or bad thing. 45-50 papers is still not a huge conference.
Since we're talking across multiple conferences, rather than do a blog poll, please comment in the comments.
Subscribe to:
Post Comments (Atom)
19 comments:
ISIT could probably stand to have a slightly lower acceptance rate, in general. 8 parallel sessions gets pretty tiring pretty quickly.
Why not have conferences nominally "publish" all papers on the web, and have the PC members select candidates for presentation? (The scarce resource is no longer pages in printed proceedings.) Such an approach would substantially reduce reviewing load by preventing authors from submitting half-baked work, and would greatly speed the dissemination of knowledge, which is ostensibly the point of writing research papers in the first place. What's the downside?
More papers doesn't necessarily mean longer conferences or more parallel sessions: you could just have (more) poster sessions. In some conferences (the machine learning conference NIPS is a good example) *most* of the work is presented in poster sessions.
I agree - I would like to see ISIT have an acceptance rate of around 30%, and fewer sessions (maybe 5).
I'd also like a rule that each author can have a maximum of 4 submissions. A few people have 10+ papers/year nowadays!
> What's the downside?
The downside is that, for better or worse, publications at top-tier conferences are "recognition" of good research. People pay more attention to these papers, because the PC has put a stamp of approval that "these are good/important/novel pieces of research." Hiring and promotions decisions are impacted by such publication records.
If acceptance ratios trend much higher, then conferences are deemed to be non-selective, as they are in most areas of science and engineering.
All this does is then push this selectivity criteria to journals, which will begin the debate anew. In fact, there are fewer good reason for journals to be selective than conferences, in my opinion, as page limits mean naught with electronic publications. Presentation slots, on the other hand, are not infinite.
You might be asking a difference question -- should we fundamentally change how research is recognized -- but that's a much larger question.
And as people have said before to paraphrase Churchill, this peer-reviewed publication game is the worst system we might have, except for everything else we have tried.
(As an aside, I'm sometimes concerned that NIPS' approach is actually detrimental to other areas of computer science when it comes to hiring/promotion, as it weakens our argument about the primacy of the conference system in computer science.)
> Why not have conferences nominally "publish" all papers on the web, and have the PC members select candidates for presentation?
Perhaps I actually just misinterpreted your suggestion. I think it's a fine proposal to stick all "submitted" work on the web. However, there should be a clear demarcation between submitted and "accepted" work, so that authors cannot cite their "submitted" work as appearing at the conference.
That said, I'm not convinced this would reduce reviewing load in any way (without sacrificing the quality of the review process).
My criticism of NIPS was that, as far as I've been told by my ML colleagues, citations explicitly do *not* differentiate between those papers accepted for presentation and those accepted as posters. They responded that the community "remembers" which were the full vs. poster papers. I somewhat doubt this institutional knowledge persists over time, however, and it's certainly not known to people in other CS communities. Can anybody from NIPS verify if this is the case, and why this policy arose as it did (e.g., as opposed to citing something as "Title (Poster)" ...)?
Certainly, STOC/FOCS should accept much more papers. The current situation is harmful to science. It is evident almost to anyone in the field that too much energy is put on promotional stuff, packaging and exaggerated self praise in order to get most papers accepted.
If papers nowadays start with sentences like "We prove the surprising result..", or "we refute it on the strongest terms possible..", or "we dramatically improve...", then something is wrong.
Mike, for someone who's tenure track and who's got 2 papers into sigcomm this year, it is certainly expected that you will strongly support that conference and will refute many non-compliant models.. why am I surprised?
It has become a norm these days for young tenure track faculty (who in turn train their grad students) to chase, collect, and proudly display brand names. As a PhD student, one wonders which of these two is right (no, the two are not equivalent):
- think and worry about doing good, solid, and useful research?
- think about what/how I would go about for the next tier-1 deadline?
Of course, in utopia, I would pursue the former, and call the latter as rubbish. In today's society, my post-PhD opportunities are strongly dependent on the number of brand names I display on my resume, so I should be and am worried about the next sigcomm deadline. Will I make it? Will I not?
All that aside, I wouldn't think great science has been done so far because someone chose to worry about sigcomms, stocs, focs', and other branding nonsense. (and chose to keep writing about it on their blogs.)
All that aside, I wouldn't think great science has been done so far because someone chose to worry about sigcomms, stocs, focs', and other branding nonsense.
Recognition in the form of papers in a conference is often a driving force for good research. It can also be a driving force for overselling introductions and other things we don't like about conferences. But without big brand name conferences, a lot of research that currently gets done and gets written up may not.
Surely researchers who submit to tier-1 conferences know what they are getting into and the lure of recognition is the reason they submit in the first place.
Recognition in the form of papers in a conference is often a driving force for good research. It can also be a driving force for overselling introductions and other things we don't like about conferences. But without big brand name conferences, a lot of research that currently gets done and gets written up may not.
Of course. Hence, we are talking here on changing the balance (i.e., higher acceptance rates), and not abolishing STOC/FOCS altogether. This will lower the need to waste time on promotional stuff, and on doing research that is directed solely to get accepted to some conferences in the first place; while, on the other hand, still maintaining a certain level of fruitful competition.
The current situation is not good, and it only benefits few who get accepted regularly into these conferences.
Mike, I agree with your general points. Your concern about NIPS is somewhat misplaced, though. It's true that the model is different (and to the best of my knowledge emerged because of the original nature of NIPS as a combined neuro/cogsci/cs conference), but in recent years, for example, from 1000-1100 submissions, typically about 250 are accepted in total. Of these, only 22 or 23 get an oral presentation. It's just a totally different model for a conference. Lots more invited talks, poster "spotlight" sessions, etc. But getting a paper into NIPS, whether as a poster or oral presentation, is as competitive as getting a paper into major conferences in other areas.
See here for a long discussion of the same issue by members of the POPL community.
Thanks to all TPC members for the constructive comments on my half baked papers. You can be sure there will be more next year!
Thanks to all TPC members for the constructive comments on my half baked papers. You can be sure there will be more next year!
Thank you, you are starting to gain a reputation as a sloppy researcher and by submitting more of the same sloppy dough next year, you will helps us dispel the remaining doubts.
At least I don't have to suck dough to get my papers accepted at SIGCOMM.
In my opinion, there is no point in increasing the acceptance rate. Already there are too many conferences and too many papers coming up.
About poster sessions, well, I think it is very difficult to convey something in a noisy, crowded hall by the means of a poster - especially in theory, where we don't have pictures to show.
No matter which criteria we choose as "recognition of good research", it can not be completely fair. Then why not live with the current setup?
The downside is that, for better or worse, publications at top-tier conferences are "recognition" of good research.
"Good" research is subjective, especially in networking as opposed to say theory. If the attempt to keep these "top-tier" conferences clean isn't working, why keep pretending it is? If we increase the acceptance rates, then it no longer becomes "exclusive" to different clans, and so, in a way, we solve that problem. And, for young faculty on tenure-tracks, rather than focus on the number of papers accepted, they can then focus on important work, work that the rest of the world truly cares about.
I've personally talked to a number of the chairs and PCs in this Sigcomm, frankly speaking, my hopes that this conference will be fair is rather dim. From the results, I see a bias towards a different clan, one that has been largely hidden by the dominant one. I guess that's how we know the true character of a person: we give him power, and see what he does.
Post a Comment