I recently received a request to review an article submission from the editor of a proceedings volume. The experience eventually led me to ask the question: what is peer review?
I wasn’t an expert in the paper’s subject matter, but after expressing this reservation and receiving encouragement in return, I agreed to read it. The paper was a pleasant read, and in the end I wrote six pages of non-compulsory stylistic recommendations, a few short questions, and a couple of typographic corrections. I sent it in, and that’s when I received a puzzling reply:
I have to ask you to send it again [...] could you please state that the results are correct?
I answered that I could not possibly say with certainty that the results were correct, since for instance, the paper relies on other papers which I haven’t read. Moreover, I don’t think I could ever be sure that any complex statement is 100% correct. In any case, if I had found an error, I obviously would have mentioned this in my report. So, as a compromise, I offered to state (somewhat redundantly) that I did not notice any errors. Here was his response:
That’s good enough although I usually think that a referee report includes truth verification based on the assumption that results from other papers that are used are correct.
This view highly questionable, and not the least important reason for this is that it’s self-contradictory. Indeed, if we are merely assuming that the other papers are correct, then clearly their referees haven’t lived up to this standard. Moreover, from an objective standpoint this view is false: the existence of a single incorrect published paper already proves that! But even viewed as a qualitative or hoped-for statement, it has implications which are troublesome to me. For one thing, it implies that incorrect articles are worthless. But surely there are counterexamples to this; it’s just embarrassing for journals obsessed with “standards”. What’s more, it places the blame (and there must be blame) for incorrect articles squarely on an ambiguous cloud of anonymity. It lets mathematicians stand up and joke “well, the reviewer didn’t notice it!” as if they’ve gotten away with something.
They haven’t gotten away with anything. As long as the error is found, even if this comes after publication, then everything is in its right place. In fact, the publication-as-perfect culture is a real detriment here, because it makes it almost impossible to let readers know there is a correction.
Why not regard peer review as a chance to help mathematicians improve their exposition? Most errors come out in this process anyway. And for that matter, why not let it be open and transparent? The more voices are able to participate in the discussion, the more successful it will be.
To drive my point home, I refer as I always to to Wikipedia, which as of this date states:
Publication of incorrect results does not in itself indicate a peer review failure.
What an appealing line! I like it because the old guard will certainly object: “Citation needed?” To which I would counter: “Even if we find one, who will verify it’s correct?”