Joan has permitted/encouraged me to post on the STOC PC process this year. So far, all seems to be running smoothly.
Despite pre-conference suggestions by some naysayers that the formatting rules would be a problem, it doesn't seem to have been, unsurprisingly confirming that STOC submitters can in fact (with high probability) follow basic directions.
The hardest part, I think, has been the assignment process. This always seems to be the hardest part on the PC end. Having a large committee both helps and hurts -- more flexibility, but more individuals to deal with. In particular, some people didn't seem to enter their paper preferences in a timely fashion, causing issues downstream. Joan has been handling assignment issues as they arise, and I don't expect any significant problems in the end.
As usual, I'm thrilled to be using HotCRP over the alternatives. I'm by no means an expert in HotCRP usage - I don't go around throwing down searches using Boolean expressions, or that check what the Overall Merit score fields are. Still, nice to know that they're there. I'm just thrilled that I can easily get the key overall view on my assigned papers, at a glance seeing how many reviews are in and the scores for everything. Also, being able to tag papers in arbitrary ways just seems like key functionality.
First-round reviews are trickling in. To all of you on the PC, thank you - and please get those reviews in! It's actually helpful to us if you submit reviews as you do them as opposed to waiting to put them all in the system in bulk at the deadline, so please post them as you can.
Subscribe to:
Post Comments (Atom)
17 comments:
The formatting rules were a problem for me AND I can follow directions. In my opinion, the requirement was a mistake and you're attempting to fabricate the narrative without any real thought or consideration. An unwillingness to admit one's mistakes is not helpful to the community.
Anon #1: Perhaps I should have been clearer. The problems suggested in other blogs regarding the formatting rules was that we'd have large-scale numbers of people who would turn things in in the wrong format and subsequently have their paper rejected out of hand, angering significant numbers in the community. As far as I'm aware, that didn't happen.
Perhaps you can clarify why the formatting rules were a problem for you. If what you mean is that you just found it a hassle, that wasn't the sort of problem I was talking about. Currently, though, I don't think there's a "mistake" that I have to admit (or, really, the executive committee has to admit -- it wasn't my individual choice). Of course I (and, I imagine, the executive committee) welcomes more useful feedback and information.
As a (non-executive) PC member, I am only looking at the full version of the submitted papers and ignoring the submission version, unless the full version isn't clear. I really don't see why both versions were requested; the
submission format is harder to read.
Russell Impagliazzo
Russell --
I think you've explained why we asked for both; because some people like the full version to read, some like the submission version. In particular, some people like to stop at some point when reviewing.
I imagine from your comment that you were equally unhappy back when people had to submit a 10-page single column version and a full version, which I find inferior to what is currently going on.
You appear to be proposing we do away with page limits for submissions, which I'm loathe to do, for reasons I've already explained elsewhere and multiple times. I believe most people agree that that approach is untenable.
Anon #1: Perhaps I should have been clearer. The problems suggested in other blogs regarding the formatting rules was that we'd have large-scale numbers of people who would turn things in in the wrong format and subsequently have their paper rejected out of hand, angering significant numbers in the community. As far as I'm aware, that didn't happen.
I didn't see anyone make this objection. I didn't read a single comment to this effect, and it certainly wasn't the main thrust of the complaints. I did see many commenters who thought it was a bad idea based on more reasoned grounds.
http://valis.cs.uiuc.edu/blog/?p=6070#comments
Anon #5: I appear to be recalling a Google+ conversation (under Jeff Erickson) where Sariel was stating, for instance:
"I don't care but clueless people that get their papers rejected because of minor technicality will be very upset. This is going to be good... "
as part of the discussion.
Again, I don't understand the context of what you're saying. Anon#1 said the formatting rules were a problem. If all they (and you) are saying is that they don't like them, that's fine, they can have their own opinion. If they're saying it was a problem in that it has caused some systematic problem with the running of the conference (in particular, for example having a number of "clueless people that get their papers rejected because of minor technicality" -- Sariel's words, not mine!), I haven't seen it.
Anon #5 again.
Again, I don't understand the context of what you're saying. Anon#1 said the formatting rules were a problem. If all they (and you) are saying is that they don't like them, that's fine, they can have their own opinion.
The context is well-reflected in the discussion on Sariel's blog about the many downsides of the required submission format.
The comment you mention from a Google+ interaction appears to be a joke. Credit goes to the executive committee for getting the word out on the newly instituted requirement well in advance of the deadline.
The only issue I took with your post is that you dismiss the "naysayers" as having unfounded concerns while I don't see any evidence to this effect.
From my limited perspective, people were upset about the (multiple) hours it took to get their submissions into proper format. It is especially frustrating if you feel like the original format was significantly more readable.
As an external referee, I have twice had to request the "expanded version" when the PC initially sent me only the two-column format.
Since there is such a large PC, there is an opportunity for a statistically significant sample. One could simply ask the PC members which format they prefer.
> The comment you mention from a Google+ interaction appears to be a joke.
I didn't (and still don't) interpret it that way; you'd have to see the full conversation, but I don't think it was a joke. Sariel can comment, of course.
> The only issue I took with your post is that you dismiss the "naysayers" as having unfounded concerns while I don't see any evidence to this effect.
As I've tried to clarify, in my initial post, I meant this in regard to the comment above, where the "problem" that was concerning was that people would not submit in the required format; not that some wouldn't like it. Again, I've seen no evidence that the submission requirements were a problem in the regard I intended and have clarified.
> From my limited perspective, people were upset about the (multiple) hours it took to get their submissions into proper format. It is especially frustrating if you feel like the original format was significantly more readable.
Now I feel we're going in circles, as I've commented extensively on this before (on Sariel's blog, for instance), but I'll repeat my arguments.
For years I've been upset about having to meet the 10-page single column format, throwing random things into appendices. This takes me multiple hours to put into this bizarre format. Then, if my paper is accepted, I get to spend more hours revising my paper, not just putting it in the ACM format (that the submission could have been in in the first place...), but creating a whole new version of the paper because now suddenly my page limit is different and I should restructure accordingly. I interpret you saying that this old process was less time-consuming and annoying to you. I simply disagree.
Similarly, for years as a reviewer I've been upset with the 10 page single-column format, which makes me go at random places to an appendix to see what's going on. And no, I don't necessarily want to read the 30+ page full version some people submit. I'd like to read the conference submission, please.
Also recall I would have been in favor of using an equivalent single column format (say, 16 pages, or whatever the right translation is) for readability. But it was decided that the submissions, for consistency, should look like the final paper as much as possible. I understand and accept that decision. I can view it as highly reasonable even if it wasn't necessarily my first choice.
> Since there is such a large PC, there is an opportunity for a statistically significant sample. One could simply ask the PC members which format they prefer.
To be clear, the executive PC discussed this extensively. While not as large as the full PC, I think you might consider that we're a reasonable sample as well.
Anon #5, again.
For years I've been upset about having to meet the 10-page single column format, throwing random things into appendices. This takes me multiple hours to put into this bizarre format. Then, if my paper is accepted, I get to spend more hours revising my paper, not just putting it in the ACM format (that the submission could have been in in the first place...), but creating a whole new version of the paper because now suddenly my page limit is different and I should restructure accordingly.
This is not an acceptable model for the majority of FOCS/STOC papers. Their content cannot adequately (i.e., formally and rigorously) be expressed in a 10-page extended abstract. Thus most of these conference abstracts _must_ be thought of as a summary and introduction to the "real work" which exists independently.
In other words, the standard model should be that there exists an idea or set of ideas expressed in a paper. One submits the paper to the PC who then decides whether they will anoint their blessing upon an extended abstract. Upon approval, an extended abstract is written and submitted for publication. You seem to indicate that the abstract itself is the primary goal. I disagree.
From a purely functional standpoint, I believe strongly that (i) the majority of FOCS/STOC authors prefer not to submit in two-column 10-page format and (ii) the majority of FOCS/STOC reviewers prefer not to referee in two-column 10-page format. If those assessments are true, I do not see how the requirement can be defended.
Hi all,
Regarding the submission format, I have to agree with Anon 5 and Russell - I definitely prefer the usual single-column format. From the submission perspective, my co-authors and I had to spend several extra hours formatting the paper. Certainly, we would have preferred to spend this time polishing the content. From the reviewer perspective, the two-column format is harder to read.
I am a (big) fan of most of the changes introduced this STOC (two-tiered committee, itemized and well-defined coi rules, etc). But as far as the submission format goes, I'd vote for going back to single columns.
Hi Piotr. Thanks for using your name.
Two questions.
1) Again, as I've said, I'm pretty agnostic as to whether submission should be 1 or 2 column; my concern was that it shouldn't be "10 single-column pages with appendices", it should be the length of the final paper. That is, your submission should look like your final paper as much as possible, in particular according to length. (You want to keep it formatted at 1 column for submission, OK, as I've previously said, I wouldn't mind that.) Is that what you mean? Saying you prefer 1-column doesn't address this point, which is where my issue is.
2) Similarly, you don't address my point as to whether "the several hours" you spent formatting was really a loss of time compared to previous rules. Of course you don't want to spend time formatting. Who does? But as I've expressed, I always find it's MORE of my time to put things in 10-page single-column format with random appendices (as 10 single-column pages is not enough space), and then rewrite the paper again for the final version with the new space requirements. Do you not find this? (Nobody has seemed to address this point directly, so I'll ask you.)
HI Michael,
Regarding 1: indeed, my main issue is the two-column format. My experience was that a fair amount of formatting was needed to fit the equations, pseudocode, etc into the two columns. Other than that, I am OK with 10 single-column pages with appendices, or the full paper with the assumption that the reviewers read only the first 10 pages, or other permutations.
Regarding 2: I certainly felt that the reformatting to fit the two-column format took much more time than creating the appendices. It is hard to say how much time this will save in the future. However, one should note that:
- only a fraction of the submissions will get to the final version stage, at least at this conference. If another submission is necessary, the paper will likely change substantially enough to warrant another round of reformatting (that's what the reviewers are for :) . Also, some publishers (e.g, Springer) use the single column format.
- even if a paper is accepted, it can often change enough before the final version to require reformatting. Often the algorithms get improved or proofs simplified (reviewers again). Also, all of the"hardwired" break points to avoid overfulls will come back with a vengeance.
I agree with Piotr. Putting things into 2 column ACM format perhaps is not a waste of time for people whose papers are accepted (although it does make it harder to propagate reviewer changes). However, it makes life significantly harder for people whose papers are rejected and resubmitted to (say) SODA. In this case, the awful formatting changes have to be undone. And it doesn't make life -easier- for people whose papers are accepted, it just moves the formatting earlier. Given that only ~ 25% of the papers will be accepted, this change seems to be imposing significantly more drudge work on the community than the previous rules did.
Let me add my two nickels:
Whatever work is put toward moving the paper into the submission format should be wasted.
Step 1) Write a paper.
Step 2) Create another document formatted for submission.
Step 3) Get reviews (hopefully get accepted, otherwise back to step 1).
Step 4) Revise main document based on reviews.
Step 5) Post document on arXiv
Step 6) Create another document formatted for camera-ready.
In these steps, the step 2) document is disposed of. Otherwise, the author needs to make every revision twice--once in the main thread and once in the extended abstract.
There is also the possibility that the main document is written after the extended abstract. However, James (and everyone, I believe) thinks this is a bad idea and rarely happens anyhow.
We could make the submitted document the same as the main document; however, this is hard on the reviewers.
Thus making authors submit a 2 column format is doubly harmful. They will have to do it twice anyhow, and it is harder for the reviewers to read. I think we all agree that 17 single column pages is not necessarily the best choice, but is at least is good as the current choice. I hope that we adopt this in the future.
Getting things to work in double column format is more of a hassle than the typical situation wrt formatting for submitted papers is that authors either
1. have written a full version and then move and edit it down to a conference submission version which
either
(a) didn't get accepted or
(b) is accepted and then is not useful for the 2-column final format, which must be redone at a later date, or
2. have only written the conference submission version.
In situation 1(b), the new system will save a step later (though the re-examination needed for later reformatting can be useful for improving a paper) but in the other situations, the new system forces an earlier investment of extra time into formatting that isn't repaid later.
A bit of news: ACM is going to be coming up with a standard single-column proceedings format for sometime next year which might make the extra effort of doing 2-column formatting a thing of the past for ACM conferences that choose to adopt it.
After reading more of my assigned papers, or at least attempting to read them, I am finding myself really struggling with the format. The submission versions are frequently in too small a font for me to actually read, and some of the supposed full paper versions are actually appendices to the submission version rather than a full presentation. I've been resorting to searching for arxiv versions of the papers, and hoping the submission is actually the same.
My preference is to have no page limits for submissions. I had never heard of asking for submissions in two different formats, but Michael's reply to my comment suggests this was used in the past.
Russell Impagliazzo
Post a Comment