I was first introduced to the concept of formal post-publication critiques as an honours student (Australian 4th year undergraduate) in 2023. I had been assigned to the MetaMelb lab, under supervision of Simine Vazire and Tom Hardwicke, after expressing interest in undertaking a meta-research project. I was excited to be learning more about how psychology journals supported self-correction mechanisms, however, I soon realised that most of them do not. Scrolling through page after page of journal policies opened my eyes just how inconsistent and unorganized the scientific publication process could be. Every journal presented differing standards and levels of detail. Some explicitly outlined every accepted article type with descriptions, restrictions, and expectations while others only provided a vague list of what they might publish with no further explanation provided. Few were transparent about their acceptance of post-publication critiques.

Science is supposed to be self-correcting but the resistance to dissenting voices seems more like censorship at times. This was especially surprising to see in the wake of the COVID-19 pandemic. The pressing need to understand this deadly novel virus led to an unprecedented level of preprints and fast-tracked publications. An impressive feat showing that, when necessary for the public good, the system can adapt. However, criticisms were often left off the academic record and relegated to the often misinformed and messy battleground of social media sites. Surely this should have set an example underscoring the importance of curated and substantiated criticism? Apparently the opposite.

To me, the most interesting discovery from this research did not make it to publication; an informal policy comparison between my own results and Tom Hardwicke’s previous work on the topic. Between the handful of journals included in both studies, and another handful used for pilot testing coder accuracy, I noticed something unexpected. Since 2019, Current Biology had changed ‘correspondence’ to invite only, Science removed ‘technical comments’ format, and the publishers APA, Cell Press, and Nature had all discontinued web comments. Only one case arose where post-publication critique was improved, an additional option provided in a journal where they were already accepted. We emailed APA and Elsevier to ask about the changes and were met with silence from the former and an excuse that not enough people used them from the latter. This opaqueness from big publishers, whose profit margins often exceed those of major tech companies, actively disempowers the very community they rely on to function.

Thankfully, the scientific community refuses to be silenced. Maybe it’s just my confirmation bias feeding me false hope, but every day I see a rising awareness of research integrity issues, both within the scientific community and general public discourse. Post-publication review is alive and well on platforms like PubPeer, a nonprofit website where researchers can comment on articles with the option of remaining anonymous. Growing interest in initiatives like ReproducibiliTea, where critical engagement with the literature is actively encouraged, shows what I can only hope is the beginning of a systemic shift in attitudes towards scientific critique. One where open critical discussion is welcomed, where corrections and retractions are seen as signs of successful self-correction and not scientific failure. In truly open scientific discourse constructive criticism should be welcomed rather than shunned. Maybe, one day, the journals will catch up.


About the author:
Annie Whamond | @anniewham.bsky.social


See also the video recording of the presentation by Annie Whamond - Post-publication critiques in psychology journals