I had a funny conversation with Madhu Sudan yesterday, with him relaying an idea he said he heard from Umesh Vazirani (and perhaps the trail goes on further from there) -- roughly that FOCS should double in size and STOC should halve in size. Or, I guess vice versa -- the point is that right now the two are pretty symmetric, and it's not clear that's the best setup.
The idea (or my interpretation of it) is that in theory we could use a more selective "top" conference -- one that people felt they should really try to go to, even if they didn't have a paper in it, because it would have the major results from the year. Hence we halve one of the conferences and make it more selective (and, naturally, make it single-session, maybe have some special tutorials or other activities). At the same time, we don't want to lessen the number of papers that currently are taken in FOCS/STOC -- indeed, since the community (or at least the number of papers being written) has expanded, we should probably accept more. (So maybe people wouldn't feel the need to start yet more conferences, like ICS.) So we double the other. Again, this would be a conference that, ideally, more people would attend, because more people would have papers in it. Indeed, this could help get papers out of the system faster (instead of papers being resubmitted quite so frequently). By introducing asymmetry, perhaps we could make both conferences more appealing and better attended.
I pointed out that one community I know of already does this -- this is very similar to SIGCOMM and INFOCOM in networking. I think that model works, though there are certainly tensions and problems with it -- as you can see in the comments on my recent post on Ranking Networking Conferences. (Bigger conferences are more variable in quality, primarily; also, they require large-scale parallel sessions.) Again, we'd have asymmetry -- the larger conference might become perceived as "weaker", but it would play the important role of bringing the community together and being an outlet for more papers.
Interesting as though the idea is, I have trouble imagining the theory community moving in that direction. Big changes are always hard to get moving, and it's not clear how many people really think the current system is broken -- though the ICS movement clearly seemed to think something was wrong. I'd be willing to try it, myself, but of course I also like the "two-tiered" (or maybe 1.5-tiered) SIGCOMM/INFOCOM system.
Subscribe to:
Post Comments (Atom)
17 comments:
This has the disadvantage that one must potentially sit on a very good result for up to a year, whereas with STOC/FOCS considered equal, one need only wait at most 6 months.
Of course, since one may submit to a journal at any time of the year, this problem, and the problems you mention, and lots of other problems with conferences, would all be solved if we would take Lance Fortnow's advice and grow up.
Hi, Michael - this probably belongs in your conference ranking thread, but I'll point out that there's a small error in your comparison of your proposed scheme to the SIGCOMM / INFOCOM split: Most people I know -- this is a biased sample of more systems-y and sigcomm-style networking people -- will attend SIGCOMM if they don't have a paper, but will never attend INFOCOM if they don't have a paper there. Many would just send the student to present the paper without also attending. I do have a strong bias here: I find INFOCOM approximately useless as a filtering function for discovering papers I want to read: Of the nearly 300 papers (!!), there are probably about 5 that I'd find interesting. I wait for other people to tell me about the cool papers so I don't have to slog through the other 295. This does not make for a conference I want to attend. In contrast, sigcomm probably also gets ~5-10 papers that really draw my interest, but I only have to discard 25 others. :)
I find it interesting that the two conferences are able to maintain a stable equilibrium at relative parity. I would have thought there would be a runaway divergence effect at some point, with a slight increase in perception of the quality of one conference resulting in better papers submitted there, resulting in a feedback loop.
I think that STOC often gets better quality submissions than FOCS, and this is probably because of the deadline placements. The STOC deadline is in November, and the FOCS deadline is in April. Between those five months, much less research is typically done (due to teaching, etc.) compared to the seven months from April to November. Maybe it's just me but I already have the impression that STOC is at least a little bit better.
To Arvind: Given a state where the conferences are considered equal, it is a bit hard to change: the initial (random) perturbation needed to initiate such a feedback loop is necessary to be sustained for at least 3-4 years so that the two conferences can get different reputations. Unless there's some external factor to help in this, I think it is hard to happen.
I thought at first this post might have been a joke.
Given all the griping about biases (both toward areas and author names), insularity of the theory community, reduction of travel budgets (making it harder to attend a conference when you don't have a paper), difficulty for students to break in to the community, etc., do you really think the best way to encourage people to attend a conference is to reduce the number of papers accepted?!
How about increasing the number of papers accepted? This would not necessarily have to reduce the prestige of the conference -- it could instead simply increase representation of areas currently neglected, or you could have a special "student track". And if it does reduce the "prestige" a bit, so what? Isn't it about the science, not about the prestige?
On another note, it seems to me that STOC has an easier time attracting attendees than FOCS simply because STOC is during the summer rather than during the academic year.
An easy way to achieve asymmetry without major overhaul of the selectivity of conferences:
In each STOC or FOCS, split the three days as follows, assuming 5 hours of "speaker time" per day: the first day is single-session with 10 30-min presentations, and the next two days have (three) parallel sessions with 3 x 15 20-min presentations. This way, we can have 10 + 45 + 45 = 100 papers at the conference that is both more selective and more inclusive at the same time.
I hope one of the near-future STOC/FOCS PC Chairs will implement something like this.
When I have a good result, I would usually submit it to the next deadline between FOCS/STOC/SODA. They all have about the same visibility, which I think should be the primary concern. Reputation should be built by the quality of the papers. The perceived quality of the venue is a poor proxy for that. Given the high visibility of ICS this year at least, it could be added to the above set if it wasn't so hard to get to.
This approach has the huge advantage that results get out soon. I agree with anon1 that creating a significant asymmetry may compel people to wait, slowing things down.
To anon1: I don't think journals can be good substitute. Firstly, we are a deadline driven community. I'm sure most of us have essentially proved a result at some point, but really proved (i.e. worked out the details, and handled the issues that invariably arise when you do that) it a week or two before the deadline. It is much harder to give yourself a deadline for a journal submission
For one, if it's going to take 5 years after I submit it, 5 years and 1 month is not going to make a big difference.
I am a big fan of putting things on the arxiv. Maybe once we all start doing that, this issue gets resolved somewhat. But my impression is that people are fairly possesive about there partial results (and their own chances of improving on them), and only the chance of having a paper in a top-conference compels them to share the result. It is hard to tell how creating an asymmetry will affect these incentives. They may get better on average if the top conference accept more paper.
On the other hand, I would be concerned about the quality of the selection process if size of focs is doubled. If the number of submissions also doubles as a result of a lower percieved bar, the work of the committee gets significantly harder, and as a result more random.
Regarding comment by InclusionExclusionPrinciple:
or a self-serve model - 4 or 5 talks per day in single session format, and 20-30 poster presentations per day - we can select which posters we want to pay closer attention to, and can have a real live dialogue with the authors...
Of course, since one may submit to a journal at any time of the year, this problem, and the problems you mention, and lots of other problems with conferences, would all be solved if we would take Lance Fortnow's advice and grow up.
I've submitted extended versions of each of my conference papers to journals, and I have yet to achieve a single journal publication (or even a rejection notice!). I would love to submit to journals instead of conferences, because it is so financially difficult for me to travel. However, if I did that, I might graduate with a single publication to my name -- if that -- and end up slaughtered on the job market.
In a real sense, I felt as though I had to submit to ICS in September, because my travel and lodging expenses would be covered if I got in. I literally could not afford to take a chance by submitting to STOC instead.
Please understand that I also submitted to ICS because I thought the PC was incredible, and being on the same program as the "derivatives" paper is both exciting and intimidating. So I'm not trying to take any position about whether STOC/FOCS is "better" than ICS, or vice/versa. But I'd like to point out that the #1 constraint faced by up-and-coming researchers is often financial, and any proposal to improve attendance somewhere, or increase journal submission rates, needs to address (1) providing support for potential conference attendees, and (2) ensuring journal turnaround time that is rapid enough so a graduate student can put publications on the CV before looking for a job.
2 + 0.5 is not equal to 1 + 1.
Computer scientists...
Hi Prof,
Why are you so obsessed with ranking things?
Attendance at STOC and FOCS has been very similar over the years. FOCS last year had more attendees than STOC. It was reversed this year. Because it is during the academic year, a less convenient location for FOCS will have more of an impact than for STOC so the minimums are lower for FOCS than for STOC.
I agree with InclusionExclusionPrinciple that it would be better to have tracks of different prestige within a conference rather than putting different prestige works in completely different conferences. Separating less prestigious works into separate conferences (as happens today) risks several undesirable effects:
1. Increased delay in sharing results from waiting for the correct deadline.
2. PCs have to reevaluate papers many times as they are submitted to and rejected from a string of conferences of decreasing prestige.
3. Potentially harder for people doing less than stellar work, e.g. students or profs at teaching universities, to keep up with the best work. (Attendance at conferences by non-presenters doesn't seem to happen much in practice.)
There should be general conferences and specialized conferences, with the general ones naturally more prestigious since they have more to pick from. But I see no reason for there to be several conferences with the same scope that are differentiated by prestige.
The biggest downside to having multiple tracks that I see is people from other areas may not realize that multiple tracks exist and therefore the prestige of the venue may suffer. To remedy this, I propose that the two tracks be given completely unrelated names.
Here's my counter-proposal: let's make FOCS small and STOC large as you suggest, *but* hold both conferences twice a year and collocate them, with perhaps FOCS talks in the morning and STOC talks in the afternoon.
Clarification to my counter-proposal: the FOCS talks would be single session and the STOC talks 3-5 sessions, just like Michael's proposal.
This asymmetry may have the following good consequence. As people wait for a year to submit to the prestigious conference, they need to put there results out there so that they don't get scooped. Thus arxiv is used more, and results are actually disseminated faster. (By making things slower, they get faster.) Journals are slow, but it seems like a good idea to put stuff on Arxiv and then give a talk to anyone who will listen to get your stuff out.
I don’t have a good sense for who doesn’t have money to go to conferences. What percent of our community struggles with this? I don't know students that struggle with that at my school, but perhaps it is not representative. I hope that in the future (as at FOCS) the talks are taped, so people that can’t come, do not miss out on that. However, there is no substitute for meeting the best minds in our field. STOC seems to give out $500 grants for students, which covers a lot of the expense (if you stay 4 to a hotel room and don’t eat out at the most ritzy places).
What percent of our community struggles with this? I don't know students that struggle with that at my school, but perhaps it is not representative.
I don't know a percentage, but, whatever it is, I am certain it's rising. I'm at a public university in a state with a major budget shortfall. My twelve months of support has been cut to nine, and it was just a living stipend before that cut. Further, I received an email this summer informing me that, along with other grad students, my tuition would no longer be covered; the department reversed itself on that decision for the current academic year. Next year, who knows.
Also, I strongly suspect from your comment that you attend school in the United States (as do I). It costs well over $500 for a European or Latin American to attend STOC, FOCS or SODA (to say nothing of faculty members in India or China who want to attend). We are presumably engaged in a worldwide scientific enterprise, and it's provincial to consider the needs only of students in U.S. institutions.
Another thing that concerns me is this: I'm not the brightest star in the sky, but my CV is above average for a third-year graduate student. As a result, I've received travel grants to attend workshops that might not otherwise be available to me. What about all the people who are "only" doing good, solid work, and publishing a paper every 18 months in reputable-but-not-premier venues? Should they be frozen out?
Post a Comment