Some colleagues and I are working on extending a certain known result. There's a journal paper with a proof of the result that seemed like it would allow a reasonably straightforward extension. Except that, now as we go through it carefully to figure out how to prove our extension, we have found what seem to be at least one and possibly two fatal bugs in the argument. The result is almost certainly true -- there's an earlier paper with a much more complicated proof that seems much less amenable to our extension -- but it seems unlikely that the approach taken for the simpler proof works without some very major changes.
How best to handle such a situation? The authors are good researchers, not hacks. It may be that they already know there's a problem, although we haven't seen any sort of retraction or explanatory note anywhere. At some point, I imagine we send a polite note to them, explaining what we found, although for now we'd like to try to fix the argument (and prove our extension!) ourselves. It seemed like it must be a common enough situation -- in the past I've found major and even unfixable bugs in papers -- that people might have an opinion on how to best handle the problem.
Subscribe to:
Post Comments (Atom)
7 comments:
It would be great to hear constructive suggestions on how to deal with this. For online papers (e.g., the arxiv and the ECCC) I know that corrective revisions are fairly common. But for journals--- or even many textbooks, there are no posted errata.
In fact, in the past I have found a bug, which led to an opportunity for collaboration, but the bug in the earlier paper still remains (in this case, the proof is not correct, although the result is true, and a simple proof exists). What's standard here? Should we only post in the ECCC and arxiv?
- A Grad Student
But for journals--- or even many textbooks, there are no posted errata.
That is not quite right. Several journals in TCS and Mathematics post errata.
If you find a purported error in a (journal) paper, I would encourage you to contact the author(s) and ask them for clarifications. If they confirm that the paper has an error then they ought to publish an erratum or a retraction, depending on the nature of the error, in the journal where their paper was first published as well as on the web sites where the paper is posted.
Sometimes fixing the error requires a considerable amount of new research and may spawn off a collaboration between the original author(s) and the researcher who spotted the error. I always find this development satisfying. A lot of good work may arise from a previous error, and the whole community gains from it.
What if... You find an error, you contact the authors, they acknowledge the error and say they will issue a correction. Several years later, they have not done so. The result is wrong, no collaboration will result from this because it cannot be corrected. The incorrect paper is still being cited, although probably not in essential ways.
I'd say that if you have read the paper closely enough to find an error that nobody else had spotted before, this is because you are using the result in some way in your own work. In that case, if the authors have acknowledged the error, you might wish to add a remark to your paper to the effect that result xy in paper z (or its proof) is incorrect and provide a counterexample.
This way whoever reads your paper will know that a result they might want to build on in their research is wrong.
I am afraid that there is no best way to handle these situations. Everything depends very much on the people involved.
In one of Knuth's Computer Musings he says something like this: "This theorem has a nice published proof but unfortunately I found that it is wrong when I was trying to turn it into a program. I found a fix, which is ugly, and emailed the authors. I believe they should have the opportunity to publish first a correct proof. Unfortunately I got no reply. May they no longer use that email address. There's nothing I can do."
On a related note, I was recently in a similar situation. I reviewed a good paper that had a small technical error. The paper got published without the error corrected. I'm pretty sure I'm right but apparently I failed to convince the authors in the review. What should I do?
Personally I'd like to see journals and conferences start to allow for online comments. Papers usually already have a webpage devoted to their abstract, why not blog style comments?
Comments would allow readers to discuss the paper, answer each others questions, and of course point out any errors. If someone incorrectly points out a mistake, the authors could correct them.
Obviously paper errors should be corrected, but online comments offer a simple partial solution, especially when authors and journals are unresponsive.
Pointing out bugs in journal versions is tricky. Of course one should alert the authors but the only way for them to correct it is via an erratum page. There is no way to hold the authors to publicly retracting a serious bug. By our usual etiquette the only way to point it out publicly yourself is as a side-issue in writing a follow-on paper, but you only get to do this if you've got new results. Unfortunately in this case the new results seem to depend on the bug...
The bug issue is obviously more prevalent in conference papers. Too few people write errata pages for fatal flaws in conference papers or journal papers, myself included. (The case of k-uniform hypergraphs for k>3 in my 1990 SODA paper was incorrect, a fact that was announced in the talk. Pierre Kelsen's 1992 STOC paper fixed it, but without reading that paper one might not know.) It is a kind of 'tipping point' issue - if more people did it then more of us would feel comfortable doing it. It is one thing when errors get fixed in journal versions but when the journal version won't ever appear because of the error, it is a problem.
This is also a problem for online preprint servers. I agree with the previous poster who suggested that online comment features are perfect for this context. The lack of a comment feature is the major problem with the Arxiv (particularly the Computational Complexity section which seems to have a high percentage of bogus papers posted). That's a major reason to prefer ECCC to the Arxiv.
Post a Comment