The main focus of our ReproducibiliTea club has been increasing early career researchers’ confidence by putting open research principles into practice. When the idea of adding preprint reviews to our meetings was first brought up, it seemed like a natural fit. Making open reviews the norm in research can increase opportunities for diverse feedback, might encourage positive reviewer behaviour, and would improve transparency around how research rigour is evaluated. Although it was daunting, with a club mostly made up of PhD and Masters students without reviewing experience, we’ve found it an interesting and helpful experience to review preprints ourselves.

Along with publishing preprints, making our work globally accessible on a faster timescale, post-publication reviewing seemed like a great step to engage with in the publish-review-curate model. This model summarises a method of research dissemination where preprinted or open work is reviewed publicly by people with relevant insight, and validated in context by others to improve discoverability and trust. With support from ASAPbio’s preprint review club program to share snacks with attendees while we discussed papers, we set about using some of our ReproducibiliTea meetings to review preprints together on a regular basis. When we did some research for guidance on reviewing, we realised that it was very common to have minimal help when starting out. ASAPbio’s resources on the FAST principles for preprint feedback formed the backbone of our approach; mainly, that reviews should be focused, appropriate, specific, and transparent. Focusing on being reflective and specific, we aimed to provide clear, actionable, and useful suggestions for the authors.

There were understandable concerns about being open in critiques of research. In particular, as early career researchers there were specific worries about being judged negatively for engaging in public criticism or leaving inexpert feedback. Making it optional for journal club attendees to sign the reviews and acknowledging our expertise candidly made it a more accessible process. The important thing for us was to have peer-to-peer feedback during the process, which helped to shape the tone and clarity of our comments to better contextualise the research findings. We also discussed the ways that open review may give readers of the feedback the impression that reviewed work is less robust than preprints without public feedback, but agreed that a collegial and engaged review remains an optional, useful contribution to authors and colleagues, should they choose to read it.

The process has given us an opportunity to talk about why open reviews can be a positive contribution to journal clubs and the research community. Reviewing preprints openly provides an alternative way of engaging with research in the public domain, beyond journals that allow for hosted post-publication critiques, such as letters to the editor or validation reports. There are often specific restrictions to these forms of feedback. They also may not be linked directly to the original article, or published at all. By using the PREreview platform, we have a DOI for each review and location linked to preprint servers on which to share our feedback freely. Open reviews, credited to reviewers, may be a more incentivising practice than closed reviewing for for-profit journal entities, and could motivate the understanding that reviews are pieces of scholarly work in and of themselves.

We have now completed several reviews, and it’s been empowering to think together about how to convey positive, constructive feedback in an open way. Taking preprints seriously as a valuable dissemination practice has fostered discussions about their value as a more equitable, self-determined approach to sharing research than through traditional routes. Hopefully, we have also contributed useful insights to the preprints we have reviewed! We now review preprints on a regular basis, and look forward to continuing in our ReproducibiliTea meetings into the future.

About the author: Ze Freeman | @zefreeman.bsky.social | King’s College London ReproducibiliTea