I just got my set of NSDI papers to review, and have been looking them over.
One thing that immediately strikes me as I give them a first quick pass is how nice it is that the submissions are 14 double-column pages. The authors have space to present a meaningful introduction and a reasonably full description of and comparison with related work. They can include (detailed) pseudocode as well as description of their algorithms. They have space for a full page (or more) of graphs for their experimental results, and even more space to actually explain them. The papers actually make sense, in that they're written in sequential order without having to flip over to "appendices" to find results. The phrase "this will appear in the full paper" appears rarely -- not at all in most papers. The papers are, as a consequence, a pleasure to read. (Well, I can't vouch for the actual content yet, but you get what I mean.)
As a reviewer, it's also nice that if I tell the authors they've left out something that I think is important, I'll generally have confidence they'll have space to put it in if the paper is accepted, and that it's a reasonable complaint to make, in that they ostensibly had space to cover my issue. (There are some papers which fill the 14 pages and perhaps won't have something obvious that could be removed, but experience suggests they'll be rare.)
So I wonder again why theory conferences have 10-page single-column submission formats ("appendices allowed"!), or, even worse, for conferences like ICALP and ESA, they have final page counts of 10-12 pages in the over-1/2-blank-page LNCS format. (Really, it's just about enough space for an abstract, introduction, and pointer to your arxiv version.) Interestingly, for my accepted SODA papers this year -- which went with 10 page submissions, "full versions" attached on the back, but had 20 pages for accepted papers -- both sets of co-authors didn't want to bother when the final submission deadline came around to filling the 20 pages, figuring people could just be pointed to the arxiv (or eventual final journal) version. Why create yet another version of the paper according to arbitrary page limitations? I certainly couldn't suggest a good reason.
On the theory side, as I've maintained for years, we're doing something wrong with our submissions, with artificial page limits creating more mindless work for authors and making decisions more arbitrary than they need to be.
Subscribe to:
Post Comments (Atom)
11 comments:
Hi Michael, Thanks for the post. Can you please share how much is the paper load on TPC members for a conference such as NSDI. Thanks.
Do you see an advantage in 2-column over 1-column format?
Although you can compress information into fewer pages using the typical 2-column format, I find it harder to format things in that style (especially if there are lots of math equations), and I find it less enjoyable to read in that format.
For final version, the 2-column format has one advantage: it's more readable in standard 6-inch-screen e-readers.
I'm agnostic as to 2-column vs. 1-column in terms of "readability". They key is to allow enough space for a suitable presentation of results whichever format you choose.
I fully agree with you, but we get into the problem of what's the difference between a conference paper and a journal paper. If only one outlet should be used then I would prefer the journal version with its more rigorous review process (at least in theory).
I asked the PC chairs (Steven Gribble and Dina Katabi) of NSDI, and they said I could discuss the issue of "paper load". (In general, however, I'll try to err on the side of not revealing information about PCs that are currently ongoing!)
I was initially given 20 papers to review. NSDI works with a multi-round system, so I expect I'd have 5-10+ more to read at a later date.
This is typical for such conferences. As Steven suggested, I'll offer some context (his words):
"...the typical TPC load for top-tier systems conferences is between 20 and 30 papers, over a one to two month period -- this is what I've seen with NSDI, SOSP, OSDI, Eurosys, etc. This is a high workload, especially since the reviews tend to be quite careful and thorough; so, accepting a TPC position is a pretty major commitment."
I'd agree with this assessment. The major theory conferences have higher loads in terms of numbers of papers; however, there is almost never "subrefereeing" in systems conferences, and reviews are generally significantly more detailed than reviews in theory conferences.
@Anon1, Sylvia and I posted stats about last year's NSDI submissions and review workload in the message from the chairs; you can read it here:
http://www.usenix.org/event/nsdi11/tech/nsdi11_message.pdf
700 reviews across 26 PC members in thee rounds of reviewing.
I don't understand the point about SODA. This year you had to submit the "long-form" 20 page version together with the shorter "for review" version. So you must've had it prepared. Also why isn't the arxiv version easy to submit for the SODA proceedings? Is it longer than 20 pages? Significantly so? After all, if the conference allows 20 page submissions, isn't submitting the (almost?) full paper for the proceedings less confusing for the readers?
Michael and Dave: Thanks a lot for your replies regarding the TPC load of these conferences. Such information is hard to find. So, really appreciate your replies.
-Anon1
Sasho:
One of the papers was indeed longer than 20 pages for the full version.
For the other, there's probably some more stuff we'd like to add in to the "long version" (eventual journal version). Rather than figure it all for the SODA deadline, it seemed easier to turn in something closer to the submission version than the "arxiv version".
Unfortunately, there's another reason to think about whether you want to send all 20 pages of your paper to SODA, or just point to arxiv, and it has nothing to do with time. In one of the communities I am part of (lying outside theory), there has some movement to reject journal submissions when the paper is substantially the same as the conference version. When the conference version was only 10 pages, this wasn't an issue; it was rare that the whole content fit in 10 pages. When the conference version can be 20 pages, it becomes an issue. So then the question becomes whether I want to submit a full version of the paper to the SODA proceedings, and worry about whether a journal will later agree to a review process for the same paper, or just submit the same abstract we sent to the program committee.
Post a Comment