The central question of science is: how can we improve our scientific theories?

Post-publication review aims to collectively reflect on what has been published, in order to inform other researchers about what has just been done, but also about what remains to be done to advance scientific theories.

But where does post-publication review take place? Answering this question makes us aware that researchers are an integral part of society. I remember a time when a very large part of the research community was on Twitter (and, today, to some extent, on Bluesky). It was a time when Twitter was a vast network where people could explain a paper, sometimes say things that could not be said in the paper itself, exchange ideas, ask questions about a paper, and sometimes challenge researchers discussing it. At the time, Twitter was a huge colloquium, a place for discussing what had been done and what remained to be done. And I believe that, ultimately, post-publication review is about our capacity to hold a colloquium, that is, etymologically, to keep a conversation going.

But since researchers are part of the network of a social field, they also abide by the conventions of that field. In a field where researchers are asked to produce more with fewer resources, and with resources becoming increasingly centralized, researchers will try to survive by becoming part of the central core that receives the financial means to produce more research. And doing so involves taking shortcuts with the realities of the field, with results, and with their interpretation. It also involves taking shortcuts with the conventional peer-review process.

These shortcuts, whether they involve questionable research practices or genuine problems of data fabrication or manipulation, are difficult to discuss in the public sphere. The reason is that researchers who investigate problems in published work are often themselves accused or targeted.

Indeed, many researchers specializing in error detection receive threats, and sometimes even legal complaints, from other researchers. It is therefore not surprising that post-publication review has developed dedicated channels that allow, among other things, for the protection of sources, such as PubPeer.

While PubPeer presents itself as a discussion forum, in practice it is often a place where researchers can criticize research while taking relatively little risk to their own reputation, notably because they can remain anonymous. The downside is that PubPeer has no direct power to pressure editors or authors.

It’s the editors, and editors alone, who decide whether a paper should be modified, amended, or retracted. There is, therefore, an asymmetry between discussions among researchers, which can take place relatively freely on social media, and research criticism, which sometimes has to be conducted anonymously because those who uncover problems are often vulnerable within a system that rewards the production of papers, where time spent identifying errors is time not spent being productive oneself. To improve science, we need to spend time finding other people’s mistakes, and more time to deal with the editors that are too often reluctant to act. Yet the scientific system primarily rewards those who produce new results.

There is also an asymmetry between the time of publication, which is treated as sacred, with peer review supposedly taking place only before papers are accepted, and the logic of post-publication review, which calls for research to be discussed after it has been made publicly available. There is, incidentally, an interesting question to ask about whether peer review can become post-publication review when the paper is a preprint.

APCs, or article processing charges, can also reinforce the asymmetry between researchers and publishers, since a financial transaction has taken place between them to put a paper online. Retracting the paper then raises a financial issue: why pay to publish a paper that may subsequently be removed from the internet? Researchers’ efforts to uphold research ethics thus come up against a market logic in which what is being purchased is, in effect, the right to publish a PDF.

The problem of post-publication review is therefore not simply a matter of how to discuss papers, but of who can criticize them, under what conditions, and with what means of action.

Post-publication review therefore raises epistemological, ethical, and financial problems. How can we discuss scientific results respectfully if there is a risk of discovering that our research practices are not the best possible? How can we criticize scientific results without the energy spent improving already published work becoming a waste of time in the race to publish? And how can researchers influence scientific publications once they have been approved (most often without transparency) by an editor, without having any means of exerting pressure?

Like a colloquium, post-publication review makes it possible to discuss a piece of work once it has been made public, to ask questions, propose alternative interpretations, point out errors, and collectively think about what remains to be done. The difference is that this colloquium has neither a room, nor a program, nor an organizing committee: it takes place on social media, in comment sections, on PubPeer, on blogs, or in responses to published papers.

But this lack of institutional structure may not be without consequences. Science needs these discussions after publication, without having built the institutions needed to recognize, protect, and value those who sustain them. Post-publication review is therefore an essential space for scientific discussion, without any corresponding recognition of the value of these discussions.

The question, then, is how we can build a genuine digital colloquium: a space where criticism can be public, adversarial, and collective, while also ensuring that those who devote their time to advancing science can be protected and recognized for this work. Finding an answer is one of the keys to effectively improving our scientific theories.


About the author:
Adrien Fillon | @adrienfillon.bsky.social | Linktree