tag:blogger.com,1999:blog-8890204.post8980389113353372375..comments2024-03-10T05:26:42.148-04:00Comments on My Biased Coin: A Note to Networking/Systems PeopleMichael Mitzenmacherhttp://www.blogger.com/profile/06738274256402616703noreply@blogger.comBlogger10125tag:blogger.com,1999:blog-8890204.post-28559458931869162222009-03-10T09:38:00.000-04:002009-03-10T09:38:00.000-04:00Great!! Great!!! Great!!. The last mobicom was red...Great!! Great!!! Great!!. The last mobicom was reduced to implementation only conference. Finally a post.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8890204.post-55304769970425181062009-03-09T11:32:00.000-04:002009-03-09T11:32:00.000-04:00Sorry, Adam -- I don't think I (or Matt) meant to ...Sorry, Adam -- I don't think I (or Matt) meant to make it sound like pulling teeth -- it just wasn't in the first draft, and Matt did push (politely) to say it really should be in there. So you did it. The end, no exciting arguments or drama or anything. :)<BR/><BR/>And we agree it turned out great since it really was useful for that DIMACS chapter (which everyone should read!).Michael Mitzenmacherhttps://www.blogger.com/profile/02161161032642563814noreply@blogger.comtag:blogger.com,1999:blog-8890204.post-32955586166341873232009-03-09T11:25:00.000-04:002009-03-09T11:25:00.000-04:00Michael and Matt,It's no secret that I'm the forme...Michael and Matt,<BR/><BR/>It's no secret that I'm the former student in question. If I recall correctly, the issue with my thesis was that Michael and I both originally thought that the connections to concrete applications were basically obvious given a few citations. Matt politely disagreed, and we eventually concluded that a brief exposition was in order. I did that, and the whole thing worked out nicely. (And that review became the foundation for a DIMACS book chapter that Michael plugged here a few months ago.)<BR/><BR/>You guys make it sound like you had to pull teeth from me :-) It was just a simple misunderstanding...<BR/><BR/>As for getting this work accepted by the networking community, I agree with Michael's frustration. In particular, we've encountered a lot of bizarre pushback (e.g., a conference paper is accepted to a high-profile networking conference with high reviews, but the journal version is initially immediately rejected as out-of-scope; adding some more introductory material in a revision passes the "out-of-scope" test, but despite persistent effort, the journal has incredible trouble getting reviewers to even read the paper). Arg.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8890204.post-83990418292947350322009-03-06T18:45:00.000-05:002009-03-06T18:45:00.000-05:00Contrary to what Stefan says, I find that citing h...<I>Contrary to what Stefan says, I find that citing half a dozen papers that use hash tables doesn't seem to satisfy some reviewers;</I><BR/><BR/>To be clear, I wasn't suggesting that this would be sufficient. Rather, if there are papers demonstrating that the property of hashing that you address (e.g. space or time performance in the context of some input environment) are <I>dominant</I> or even <I>large</I> parts of a well-known systems or networking problem, then I think citation can get you a long way. If many systems simply use hashing and don't specifically make that point about hashing's importance then the onus tends to fall on you.<BR/><BR/>Think about this from the standpoint of the systems reviewer. They actually may not <I>know</I> how important hashing is to application X. However, over time they do end up reading lots of papers that optimize components of systems that, when considered holisticly, really didn't matter that much. There are cannon examples of this in caching, where countless papers optimized eviction or replacement in settings where the most important component was compulsory misses (heavy tail). Now sure, <I>you</I> know to pick problems in which the data structures you are optimizing are the ones that matter, but the reviewer is frequently not that sophisticated unless the bottleneck has already been well established. For example, you wouldn't have a problem arguing for why space-time optimization for DFA/NFA representations of classifiers are important for network security because there's a significant literature demonstrating that already. If you wanted to argue that your modified splay tree is 5x faster than Sleator and Tarjan's then it probably wouldn't suffice to say that splay trees are used in Windows NT... people would want to be convinced that the performance mattered. <BR/><BR/>That said, sometimes this can be done by fiat (i.e. Personal Communication with someone at Cisco saying that this matters) and I personally think its fine to do a measurement on an existing system to demonstrate the importance and then just do microbenchmarks on your solution (although I'll admit that the systems community certainly has its hardliners who want the "full monty" as you point out :-( )Stefan Savagehttps://www.blogger.com/profile/04899968055883156907noreply@blogger.comtag:blogger.com,1999:blog-8890204.post-84103569426352817002009-03-06T08:55:00.000-05:002009-03-06T08:55:00.000-05:00Hey Matt. As we've discussed before, I totally ag...Hey Matt. As we've discussed before, I totally agree that it was appropriate in the setting of a thesis to ask for a more concrete use scenario. As you point out, there's no page limit, and moreover one can expect a wide range of readership (to the extent people read theses!), and so one should try to give a clearer picture of the use story. (And it ended up helping us greatly that you made the student do this.)<BR/><BR/>For a conference (or page-limited journal -- like IEEE/ACM Transactions on Networking) I expect people to understand that you can't always have EVERYTHING in the paper. I admit I also expect the readers to be more knowledgable of fundamentals -- I find it troublesome to think that I need to explain to networking folks why a better hash table (that tries to take into account standard hardware issues) is something that should squarely into a networking publication! Contrary to what Stefan says, I find that citing half a dozen papers that use hash tables doesn't seem to satisfy some reviewers; indeed, I find some reviewers aren't happy even if you list and describe some standard applications unless you specifically test your structure in a specific setting. (It also doesn't help, I suppose, that things like how Cisco uses hash tables in their routers is generally proprietary and not documented in a citeable way.)Michael Mitzenmacherhttps://www.blogger.com/profile/02161161032642563814noreply@blogger.comtag:blogger.com,1999:blog-8890204.post-40290005475371731562009-03-06T08:30:00.000-05:002009-03-06T08:30:00.000-05:00Having served on a thesis committee for one of Mic...Having served on a thesis committee for one of Michael's students, I can understand his frustration at someone taking a "systems" view of "theory" work. (In this case, I was the source of the frustration.) The thesis in question had a very hand-wavy description of how the particular data structure could be used in all kinds of networking settings, but did not link the technique to a particular use case to demonstrate why the idea was so compelling.<BR/><BR/>In this case, there was no page limit (it's a dissertation after all), so I pushed pretty hard to get a more concrete use scenario. And I got it: the final thesis did a nice job of making tha connection.<BR/><BR/>I can imagine a SIGCOMM or NSDI reviewer having the same problem I did. It's great to do things that are general but Stefan is right that it is deeply in our DNA to look at things from a "real problem" perspective.Matt Welshhttps://www.blogger.com/profile/04255792550910131960noreply@blogger.comtag:blogger.com,1999:blog-8890204.post-71565130698514457752009-03-05T23:33:00.000-05:002009-03-05T23:33:00.000-05:00Hi Ranjit, Too much time in airports I fear. Tha...Hi Ranjit,<BR/> Too much time in airports I fear. That said, I think its far far better to hang out on other people's blogs.... no rent.Stefan Savagehttps://www.blogger.com/profile/04899968055883156907noreply@blogger.comtag:blogger.com,1999:blog-8890204.post-55778605959094350392009-03-05T23:24:00.000-05:002009-03-05T23:24:00.000-05:00Stefan -- From the size and frequency of your post...Stefan -- <BR/><BR/>From the size and frequency of your posts, I think its time to start your own blog ;) <BR/><BR/>-Ranjit.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8890204.post-34736671667882630132009-03-05T19:00:00.000-05:002009-03-05T19:00:00.000-05:00Another push back on real application settings. Fa...Another push back on real application settings. Far too often, "principled systems building" is a euphemism for premature optimization.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8890204.post-53236378042748569462009-03-05T15:41:00.000-05:002009-03-05T15:41:00.000-05:00Bravo! I'll generalize further that the broader s...Bravo! I'll generalize further that the broader systems communities frequently have elaborate dogma about both the topics that are "in scope" and the particulars of "how" one describes a contribution. Some of this is inherent -- communities build culture and as a submitter you have some responsibility to grok that culture to become part of that community (e.g., there are various unintuitive "sacred cow" issues in various communities; the evil of packet reordering at SIGCOMM or the preeminence of privacy interests in the security community). <BR/><BR/>However, in my opinion there is far too much orthodoxy present -- and, in particular, a bias in service to dominant commercial mindset (i.e., the "this idea isn't practically deployable because...") In addition to discouraging bolder papers it also puts the community at risk of having decreasing relevance. <BR/><BR/>There was a period of time when the core systems community (i.e. SOSP/OSDI) was moving towards defining itself out of existence (in the mid-to-late 90s there was a constituency who was arguing that systems papers needed to be about kernels or file-systems or something similarly gung-ho... when such topics were being overshadowed by the impact the Internet).<BR/><BR/>That said, one point I'll push back on is the need to effectively argue for or demonstrate the value in a real application setting. This issue is deep in the DNA of systems people -- the need to understand how a real problem is solved. If its also ready been established that hash performance is key to application X, Y and Z, then life is easy and you can just cite prior work. If its not clear then I think that -- for a systems conference -- this is a responsibility to convince the reviewer that this is so.<BR/><BR/>. That said, undo orthodoxy isStefan Savagehttps://www.blogger.com/profile/04899968055883156907noreply@blogger.com