tag:blogger.com,1999:blog-8890204.post4380983918723748610..comments2024-03-10T05:26:42.148-04:00Comments on My Biased Coin: The Reviewers' PledgesMichael Mitzenmacherhttp://www.blogger.com/profile/06738274256402616703noreply@blogger.comBlogger9125tag:blogger.com,1999:blog-8890204.post-39174617778819957012007-08-06T07:17:00.000-04:002007-08-06T07:17:00.000-04:00Michael, I do sympathise with your point of view ...Michael,<BR/><BR/> I do sympathise with your point of view - whenever I ask someone for a review, and he/she responds immediately, it makes me a very happy man... However, this is the best case scenario. Often people are busy with other stuff, and relegate dealing with reviews etc to less busy times of the week. Given that you seem to be doing the same, I think you can appreciate this perspective.<BR/><BR/> Of course, this can cause a delay. However, I think that, in your example, the proper accounting is that the reviewer is responsible for 1 week of the delay, while the editor is responsible for the remainder (i.e., 3 weeks). <BR/><BR/>PiotrAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-8890204.post-26339578078115210712007-08-03T18:52:00.000-04:002007-08-03T18:52:00.000-04:00Two comments, one specific and one general.First, ...Two comments, one specific and one general.<BR/><BR/>First, as Piotr indicates, sometimes the review interface is part of the problem. I just today went to try to review a paper and the system forced me to spend 5 minutes filling out a biography first. <BR/><BR/>Second, I don't know about the rest of you, but I have 15-20 journal publications and get <EM>asked</EM> to do, I would say, about 4 times as many reviews. I try to say yes most of the time, but this ratio is ridiculous.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8890204.post-1661101295283709322007-08-03T13:05:00.000-04:002007-08-03T13:05:00.000-04:00Piotr --I'm also not always thrilled about having ...Piotr --<BR/><BR/>I'm also not always thrilled about having to respond to a review, but consider my own point of view as an editor. I might set aside 1/2 a day every two weeks or 1 day a month to deal with editor duties, specifically finding reviewers. If I send you a note and don't hear from you, you're delaying the reviewing process on that paper by as much as a month. If you send me a quick no while it's still fresh in my mind, that month might be avoided.Michael Mitzenmacherhttps://www.blogger.com/profile/02161161032642563814noreply@blogger.comtag:blogger.com,1999:blog-8890204.post-81525178143326951742007-08-03T11:20:00.000-04:002007-08-03T11:20:00.000-04:00I support most of the pledges, but I do have issu...I support most of the pledges, but I do have issue with pledge #1. I think that the issue here can be better resolved by imposing, say, a one week timeout. If the response is negative or none, the editor can assume that the review has been declined. <BR/><BR/>Unfortunately, often the editors do the opposite: if the review is not declined, they assume this implies acceptance. Sometimes, declining can be quite time consuming in itself, if you have to login somewhere and answer a bunch of questions.<BR/><BR/>I can see that this is a way to get reviews, but I dont think the reviews obtained in this manner are particularly good.<BR/><BR/>PiotrAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-8890204.post-19364691437265554232007-08-02T23:36:00.000-04:002007-08-02T23:36:00.000-04:00Paul --The difference in impression may be that I ...Paul --<BR/><BR/>The difference in impression may be that I didn't start publishing until the early-mid 1990's, when electronic systems started. I recall getting fairly detailed reviews at that time. These days, it's not surprising for me to get 3 conference reviews, with 1 being empty, one being 1-2 sentences long, and one actually being somewhat useful.Michael Mitzenmacherhttps://www.blogger.com/profile/06738274256402616703noreply@blogger.comtag:blogger.com,1999:blog-8890204.post-29127342589490665302007-08-02T13:38:00.000-04:002007-08-02T13:38:00.000-04:00I think conference reviews have grown increasingly...<I>I think conference reviews have grown increasingly less useful over the years for major CS conferences </I><BR/><BR/>If you focus on theory conferences I think that the exact opposite is true. Until electronic review systems were introduced in the mid-90's, there were virtually no conference reviews for major theory conferences, just conference decisions. Moreover, my impression is that reviews communicated to authors are now more detailed than they ever have been for major theory conferences.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8890204.post-7965079958549103742007-08-02T10:21:00.000-04:002007-08-02T10:21:00.000-04:00Aaron - that's an interesting point about how conf...Aaron - that's an interesting point about how conference reviews may have spoiled our "reviewing culture" in CS. I think conference reviews have grown increasingly less useful over the years for major CS conferences -- enough that people are starting to pay attention to the issue. But I think our whole culture could use a tune-up. (1 month! I'm asking editors to send you my submissions from now on...)<BR/><BR/>Gilad -- most CS papers have more than one author. I'm making the handwaving estimate of approximately 2-3 authors per paper on average, and 2-3 reviewers on average. Though I agree that 1 review per paper should be a lower bound...<BR/><BR/>In general, it's an interesting question how much "service", such as reviewing, is credited by our community. This is something I've been mulling over for a future post.Michael Mitzenmacherhttps://www.blogger.com/profile/02161161032642563814noreply@blogger.comtag:blogger.com,1999:blog-8890204.post-58288325494877613872007-08-02T09:06:00.000-04:002007-08-02T09:06:00.000-04:00It seems that if you want more than one person to ...It seems that if you want more than one person to review your paper (well, if someone wants each paper reviewed by more than one person), you should review more than one paper for each paper you submit (or have your students review more than one paper).<BR/>The truth of the matter is that different people have different styles of thinking, and I suspect that reviewing is easier for some than for others just like publishing is. Perhaps reviewing articles (combined, say, with writing reviews of topics) should give people some academic credit, just like publication. The ability to write good reviews and have a wide knowledge of the field is an important contribution to the community, and a great asset for a teacher.Unknownhttps://www.blogger.com/profile/04842369931350847129noreply@blogger.comtag:blogger.com,1999:blog-8890204.post-43697990067278322632007-08-02T04:40:00.000-04:002007-08-02T04:40:00.000-04:00These are, obviously, great rules to try to live b...These are, obviously, great rules to try to live by. As someone who typically publishes more often in physics journals, however, I'm a bit spoiled with relatively fast reviews. And, as such, I strive to fulfill my own reviewing obligations in a timely fashion (I try to get things done within a month of receipt). The fact that the APS journals have an automated system that bugs you every month is also a great help (but also a great annoyance; at least they let you set "away" status on the website so that they won't send you new papers to review during that time).<BR/><BR/>This topic makes me wonder whether the state of reviewing in CS and ECE is at least partially due to the emphasis on conference proceedings rather than journal publications. In my experience, conference reviews are short and often unhelpful. Is this because people in CS and ECE generally aren't schooled (acculturated) in how to write a good review?Anonymousnoreply@blogger.com