People in various have been noting the SODA accepts, but nobody has been talking about the numbers. I count 138 papers accepted. I can't find now how many submissions there were; I recall it was over 600 abstracts, but I think it cut down to 520-540 or so. So this seems like just over 25%.
Is that a good number? Too many? Too few?
I admit, I have trouble believing that there weren't at least 40, 50, maybe 100 of those rejected submitted papers that were of sufficient quality that it would have been just fine to have them in. I could be wrong -- I wasn't on the committee -- but I'd guess there was more good stuff out there.
For those papers that were rejected, where will they go? ICALP and ESA deadlines are pretty far off; there aren't a lot of good homes for algorithmic papers until then. (STOC/FOCS may not be appropriate; other conferences and workshops have, I think, weaker reputations that may make them less desirable, especially for up-and-coming students and younger faculty.) It seems like there's a hole in our schedule. Rather than fill it with yet another conference, wouldn't the community be better off accepting more?
Subscribe to:
Post Comments (Atom)
22 comments:
An alternative to accepting more is to let papers "overflow" to more specialized workshops (with the authors' permission). This is done for example at the ICDM conference, where the reviews for the main conference are passed on to a chosen workshop PC, allowing them to make a rather fast decision. Sure, this introduces "A+" and "A-" papers, but the current system already does that, with the drawback that A- papers may take a long time to get out.
Maybe go to the journals. But in computer science, the review process of journals is so long!!!
Anonymous is right, when is CS going to journals, as all other serious fields do?
Going to journal submissions is certainly the right way to go, and in any case, this is what eventually will happen.
Also, the review process of journals is very efficient from my experience, and certainly much more efficient than the conference submission process, since, as this post mentions, many good and deserving papers are being rejected from conference based on arbitrary acceptance rates. It seems that encouraging direct submissions to journals, avoiding the long conference submit-reject-re-submit process, will actually help in faster dissemination and a more serious culture.
Roughly 135 papers has been quite typical for SODA for quite a few years and there is a simple reason - one can handle at most that number with triply parallel sessions in a 3-day conference. There were been 491 submissions 5 years ago (albeit a mix of long and short) so the number is not so different.
Would the venue handle a 4th parallel session? Adding a 4th day probably would not be possible at this late date and could increase costs a lot.
Adding a 4th session might dilute audiences a bit because it probably would not substantially increase attendance. However, with enough advance warning maybe a PC could make such a determination.
Maybe a good revenge of not getting one's paper into a conference would be to polish it quickly and get it accepted it to a good journal.
(I, too, would enjoy larger conferences or a publication culture that makes directly-to-journal publication viable.)
Let me point out that STACS (http://stacs2012.lip6.fr/) has the correct time slot for submitting manuscripts that were rejected from SODA. If STACS introduced an algorithms track (like ICALP did in the 1990s) and dramatically increased its volume, it would fulfil exactly the role Michael is soliciting.
Yuval Rabin shared numbers and insights into the reviewing process of SODA 12 (https://plus.google.com/107399564684143990874/posts): 523 submissions
STACS and LATIN are natural catch-alls for SODA rejects. STACS 2012 has a rather limited number of slots for algorithms papers though.
From PC member Yuval: The competition this year was fierce. We had to reject papers that received good reviews and are clearly SODA quality papers.
I've heard this from other PC members as well. This begs the question. Was it really that hard to find an extra room in Japan for a fourth parallel session?
I'd like to thank Yuval for discussing how the PC meeting went (it's useful information, and recommended reading).
I can understand why the PC would be hesitant to decide "on the fly" to introduce a 4th parallel session; how do they know what the hotel can accommodate? I think the issue is that WELL BEFORE THE PC MEETING there needs to be a decision of the form -- hey, we'd rather err on the side of accepting more papers rather than accepting too few. I think this is true for many theory conferences, and I've stated it before. But we seem to be in a mode where we've decided, "This is what we've always accepted, this is what we should accept again."
With regard to Paul/anonymous: I'm happily tenured, and if a paper of mine gets rejected, I'm fine with sending it to a journal or to a "less prestigious" conference and figure people will find the paper and realize how wonderful and important it is down the line. But how can you explain to a student that their "SODA-quality paper" [From PC member Yuval: The competition this year was fierce. We had to reject papers that received good reviews and are clearly SODA quality papers.] is going to have to go to STACS or LATIN? They're rightly concerned about careers, jobs, etc.; THEY care where their paper ends up, and if we're rejecting SODA-quality papers for SODA, aren't we doing something wrong that we should fix?
I think it really does not make much sense to reject "SODA"-quality papers because of slots. Many conferences allot different time slots: 30 minutes presentations and 15 minutes presentations. SODA could have done that too.
To confirm the guess work, yes, I am frustrated about my SODA reject which I think is a better paper than some of my previous SODA accepts. It's more frustrating since the reviews are not out yet!
I think it is beyond belief that the program committee considers the more or less random historical number of accepts more important than the principle of academic quality.
We have lost sight what we are supposed to be about.
I understand an increase in the number of accepted papers wasn't even discussed. For all we know the cost of an extra session in Kyoto could have been minimal!
While Yuval's comment that SODA quality papers were rejected seems to have drawn a fair bit of attention, I don't think there's anything unusual about this year's SODA in that respect. I'd argue that this is true every year, and the reasons are almost always because of capacity constraints (and yes, my paper got rejected so go ahead and label me a sore loser :))
I think Rasmus' suggestion goes in the right direction. It is a small change that has a chance to be accepted by the community.
I don't really understand Michaels and related comments. Of course you can say that something is wrong "if we're rejecting SODA-quality papers for SODA". I would argue that the quality of a conference is a function of time (that has always been the case) and there is nothing particularly wrong about this.*
*I know that people will say that the real problem is that the decision is arbitrary because you cannot accurately rank papers. This is true to some extent, but an economic reality of the world is that we have to rank things that cannot be accurately ranked. We do it all the time. You accept all the papers to a conference in order to level the playing field for people's careers? Great, but you still need to rank all these candidates applying for that faculty position somehow.
Suresh wrote: I don't think there's anything unusual about this year's SODA in that respect.
I've heard several PC members talk about this, and their take is that, contrary to what you say, this was an exceptional year.
They felt that they could have easily accepted 20-30 more papers and still be above the historical SODA quality norm.
Maybe we could set up some sort of virtual Salon des Refusés for all the good papers that have been rejected.
Actually, similar situations in the past have been the basis for the foundation of new conferences, CCCG for instance.
Rasmus suggestion makes a lot of sense. It would be really easy to have an ACM Conference on Algorithms (acronym COLA) and accept the top 30-40 SODA rejects there.
Authors would then have a chance to accept or withdraw their paper from that venue.
As Rasmus pointed out ICDM already does this. So does INFOCOM.
In fact, I would be happier to have a chance of rebuttle of one high school probablistic argument that one review clearly does not understand. :(
Of course, I am not saying my paper would be accepted if the reviewer is convinced. Still it is frustrating to receive such comment and you cannot reply.
Unsubstantiated rumor has it that many rejects are traditionally in geometry (where STOC/FOCS is not really an option), which is why SoCG was created.
Mihai: SoCG started in 1985, SODA in 1990. It may well be that SoCG started because too many geometry papers were being rejected from STOC/FOCS (I don't really know, it was before my time) but it's not because of SODA rejections.
I like Rasmus' idea also for a different reason: It reduces the total reviewing load a bit, as good papers that are "above the threshold" don't have to get reviewed multiple times.
- Philipp
I was not being facetious in suggesting polishing a rejected paper for journal publication. I may be wrong, but my sense is that having a paper in one of the top few theory journals is as good as, or even better than, a paper in a second tier conference and J.ACM is more selective than any of the conferences (though the selection metrics may be somewhat different because of J. ACM's scope).
Special issues are indicative. The FOCS/STOC best paper award papers end up in J.ACM and other top FOCS/STOC papers end up in SICOMP special issues. SODA ones end up in TALG, CCC ones end up in Computational Complexity.
After the top half dozen or so theory journals, things do get murky and the quality "floor" for journal papers can get quite low, despite the existence of many good papers in such journals, a function of the distributed nature of journal refereeing and something that may be more reliable from conferences.
My impression is that the field's problem in this area is not one of improperly ranking conferences over journals but it is one of laziness in (1) citations and in (2) polishing papers that causes people not to cite or even produce journal versions if a conference version is available.
Post a Comment