A commenter asked me a while back to say what I thought of the SIGCOMM submissions this year.
I'm finally getting around to reading and reviewing. (First round reviews aren't due for at least a week!) And so far, by and large, the papers I'm getting are pretty terrible.
This generally seems to happen to me on the first round, but this year is extreme. My first several papers just don't belong at this conference. (Arguably, they don't belong at any conference...) There's some number of papers submitted at every conference that are just not serious submissions, and apparently I got more than my fair share on the first round.
This makes it harder to judge the other papers -- it's hard to calibrate when you start with a lot of junk. Many of my other papers are theoretically oriented, and I'm not too optimistic about them. There's room for theory papers at SIGCOMM, but I think the bar is, rightly, pretty high. When I read a theoretical paper for SIGCOMM, I look for one of two things. First, it could be the paper has a nice theoretical idea that's actually useful. The problem there is that it's incumbent on the paper to clearly demonstrate the utility, and most fall down in that regard. I quote the SIGCOMM call: "SIGCOMM is a highly selective conference where full papers typically report novel results firmly substantiated by experimentation, simulation, or analysis." A mathematical analysis alone generally does not count as a firm substantiation. [Such papers generally have a better chance at INFOCOM -- which I think is a good, and very important, thing! There needs to be an outlet for more theoretical networking work, and perhaps it's just better suited for a big conference. Many such papers will end up having minimal impact, but once in a while, a good idea gets built on and has a real impact.]
Second, it could be the paper really challenges our way of thinking, introducing a new framework that seems a clearly important guide for future work. Such papers are rare, but important. I seem to have a number of economics-networking papers that are very high-level, and I'm really trying to understand if any of them have that character. Again, because SIGCOMM is so selective, I think the bar is very high for such papers. I'm really looking for something that enhances our fundamental understanding of the network.
That's it for now. If you have a submission, don't let my comments make you antsy -- the meeting is still a long way away, and I'm quite sure I'm not reading your paper anyway.
9 comments:
Why bother reading? Don't TPC members simply put their mate's papers on one pile, and count the number of 'junks', and then rightly report that SIGCOMM is a HIGHLY selective conference?
Well said!! SIGCOMM is only good for soliciting free advice. Every year I ask my students to submit 'junk' papers and see whether the 'gods' think any of them are worth pursuing.
I'm with Michael here - I'm on the PC as well and of the papers I've reviewed so far, only 2 have been any good (but they were VERY good). I sit on a lot of PCs and was surprised at how bad the quality was for SIGCOMM this year.
Anyone that reviews paper for good conferences knows that part of the submissions are junky, mostly coming from random chinese/indian institutions.
Could it be that because sigcomm is in india this year, this has just intensified?
interesting, anyway, sigcomm is famous for its private club style. reviewers can easily figure out the authors.
Sigh. Whenever I post about SIGCOMM, some random anonymous people come out of the woodwork and gripe about SIGCOMM being some sort of private club, etc. I'm tempted to start blocking them. Either state a legitimate, fact-based complaint, and/or state your name for the record if you have a problem. I do. It's not so hard.
For the record, I don't think we can figure out the authors. (I don't even try.) I'm glad to hear that Matt agrees with my quality assessment. Anon #4 may have a point; there's always a significant amount of junk submitted to the top conferences, and maybe this year it is exacerbated by other considerations.
For the record, I don't think we can figure out the authors.
You are wrong, it has been well established (particularly by people who have never been double blind reviewers) that one can easily figure who the authors are. Just read any theory blog when the topic comes up if you doubt me.
I say, what do you have to offer against this other than your piddling data and actual experience with double blind submissions, which can easily be discarded as they contradict the starting assumption that double blind reviewing is bad?
well, I think sometimes it's easy to figure out authors because they have done consistent works in several years and those works are well known in sigcomm community. also why somebody may think it is like a private club lies in that there are not so many researchers/groups can afford those type of works which require tremendous data or system development. it's tough. a good idea without substantial implementation seems can not be accepted. correct me if I am wrong.
"part of the submissions are junky, mostly coming from random chinese/indian institutions."
Can't stop laughing (not at the statement but at your ignorance)! You can't be serious saying this ... pretty serious claim to make I would say!
Post a Comment