propagating changes; comment on "Time To Rethink Retractions And Corrections?"

A comment on Amending Published Articles: Time To Rethink Retractions And Corrections? (doi: 10.1101/118356) which asks for making it easier and less of a matter of guilt to change published articles.

Update: Leonid Schneider from forbetterscience notes that there’s a whole dungeon of misconduct which might be facilitated by “living papers”. We need investigate problems in depth before changing established processes. Scientific communication is a complex process. Publication is an important part of it.

Firstoff: The underlying problem which makes it so hard to differenciate between honest errors and fraud is that publications are kind of a currency in science. It is not possible to make them serve a dual function — not only scientific communication but also the main currency to get a job in science — without also getting Fraud. If you want a short quotation for that, you can take Goodheart's law:

When a measure becomes a target, it ceases to be a good measure. — Goodheart’s Law

We cannot reach the best possible level of scientific communication while publications are part of the currency of science. And there is no metric which can fix this.

That said, I’m happy to see you take up changes to scientific articles! It ties into concepts I wrote a two years ago with concepts for propagating corrections: Information challenges section 2.3.3: Propagating corrections (this is a section in a larger article about information challenges for scientific publishing)

Note however that if you have living documents and only the latest version of the document is treated as authoritative, then scientific information propagation becomes orders of magnitude more expensive. There must be a clear distinction between changes which invalidate anything others might have built upon and changes which keep all the citable information the same. As I showed in the article I linked to, there are technical measures which could reduce the cost of propagating corrections. If you make corrections easier, then these measures will become essential.

Guilt should not be the problem (and should not be part of making a change). The actual problem is that a change to a published paper incurs a cost on everyone who cited it.

Keep in mind that when you change an article, you need to inform everyone who cited it.

Journals could reduce this cost on authors by checking where the article was cited and whether the change is relevant to the reliability of the citing article. If it is, then the author of the citing article must take action. With highly cited articles, a single amendment could require hundreds of scientists to take action and amend their articles as well, if it affected the core message of the article, this could cause ripples of ever more articles to amend. There are two core ways to minimize this: Amend quickly, while the article has few citations, and ensure high quality and consequently a low rate of invalidating changes for published articles.

In the article I posted,¹ I suggested using microformats to mark amendments. Their important attribute is that they can be parsed automatically, that anyone with access to the source of a publication can automate checking for the region in which a given reference was used, and that they are not tied to any given platform. Any other method which has these properties works as well.

Keep in mind, however, that while anyone can search through those updates, someone must do it. To make the system reliable that someone will have to be paid.