Jekyll2024-03-28T09:30:00+00:00/feed.xmlReproducibiliTeaReproducibiliTea - Journal Clubs for Open ScienceReproducibiliTea in 20242024-02-12T14:00:00+00:002024-02-12T14:00:00+00:00/jc/2024/02/12/ReproducibiliTeaIn2024<p>The <a href="/jc/2021/01/28/A-new-year,-a-new-ReproducibiliTea-Steering-Committee.html">ReproducibiliTea Steering Community</a> is expanding and changing yet again to make sure that we keep the early career researcher (ECR) perspective to building open science communities as well as allowing for more diverse perspectives. This year, we are welcoming two new committee members who will bring fresh impulses to our work while saying thank you and goodbye to our fantastic former committee chair Jan Vornhagen.</p>
<p>So without further ado,</p>
<p><strong>Please join us in welcoming our new committee members:</strong></p>
<p>Michael Muhoozi <a href="https://twitter.com/MUHOOZIMICHAEL">@MUHOOZIMICHAEL</a></p>
<p>Michael is a researcher at Makerere University, Center for Health and Population Research, Uganda. He has experience starting and organizing ReproducibiliTea in Makerere University, Uganda. Michael is keen to promote research reproducibility, particularly within the realm of healthcare policy.</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/Michael.png">
<object data="/assets/images/blog/Michael-small.png" type="image/png">
<img src="/assets/images/blog/Michael.png" alt="Michael Muhoozi" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p>Lianne Wolsink <a href="https://bsky.app/profile/liannewolsink.bsky.social">@liannewolsink.bsky.social</a></p>
<p>Lianne is a PhD student at the Department of Cognitive Psychology at Ruhr University Bochum, Germany. She is originally from the Netherlands, where she studied Psychology and Neuroscience in Nijmegen and Maastricht. Lianne started and co-organises the ReproducibiliTea in Bochum.</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/Lianne.png">
<object data="/assets/images/blog/Lianne-small.png" type="image/png">
<img src="/assets/images/blog/Lianne.png" alt="Lianne Wolsink" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p><strong>And, our new committee chair:</strong></p>
<p>Sophia Crüwell <a href="https://bsky.app/profile/cruwelli.bsky.social">@cruwelli.bsky.social</a></p>
<p>Sophia is a PhD student in Philosophy of Science at the University of Cambridge, working on conceptual issues surrounding the replication crisis; she has also worked on empirical metaresearch projects. Sophia started the original ReproducibiliTea journal club at Oxford (with Sam Parsons & Amy Orben), as well as those at Amsterdam (with Angelika Stefan & Florian Wanders), and Berlin (with Corinna Hartling).</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/sophia.png">
<object data="/assets/images/blog/sophia-small.png" type="image/png">
<img src="/assets/images/blog/sophia.png" alt="Sophia Crüwell" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p><strong>And, give a warm ‘welcome back’ to our returning members:</strong></p>
<p>Helena Gellersen</p>
<p>Helena received her PhD in psychology from the University of Cambridge where she was also a co-organiser of the ReproducibiliTea journal club. She is now a postdoc at the German Center for Neurodegenerative Diseases. She is also a Research Fellow-elect at St John’s College Cambridge.</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/Helena.png">
<object data="/assets/images/blog/Helena-small.png" type="image/png">
<img src="/assets/images/blog/Helena.png" alt="Helena Gellersen" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p>Matt Jaquiery <a href="https://twitter.com/MJaquiery">@MJaquiery</a></p>
<p>Matt is a Research Software Engineer at the University of Oxford. He completed a DPhil in Experimental Psychology in January 2023.
Matt manged the Oxford Experimental Psychology ReproducibiliTea while a DPhil student there, and has contributed to various open science projects with the UKRN, FORRT, and other organisations. He does a bunch of the technical stuff for the core ReproducibiliTea organisation.</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/matt.png">
<object data="/assets/images/blog/matt-small.png" type="image/png">
<img src="/assets/images/blog/matt.png" alt="Matt Jaquiery" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p>Paulina Manduch <a href="https://www.linkedin.com/in/paulinamanduch/">(on LinkedIn)</a></p>
<p>Paulina is a psychology student at the SWPS University in Warsaw, Poland, working on her master’s thesis on creativity and memory. Paulina is the chair of the Open Science Society at her university and she co-organises online ReproducibiliTea meetings in the Polish language.</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/Paulina.png">
<object data="/assets/images/blog/Paulina-small.png" type="image/png">
<img src="/assets/images/blog/Paulina.png" alt="Paulina Manduch" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p>William Ngiam <a href="https://bsky.app/profile/willngiam.bsky.social">@willngiam.bsky.social</a></p>
<p>William is currently a postdoc in the Department of Psychology at the University of Chicago, where he started and co-organizes the ReproducibiliTea journal club! He will soon be a Lecturer at the University of Adelaide, where he hopes to help early-career researchers from community with a new ReproducibiliTea journal club.</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/William2.png">
<object data="/assets/images/blog/William2-small.png" type="image/png">
<img src="/assets/images/blog/William2.png" alt="William Ngiam" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<h1 id="plans-for-2024">Plans for 2024</h1>
<p>We have a lot of exciting projects and we are always looking for ideas. If you want to get involved, have a cool idea you think we could tackle, or have some critique for us, please <a href="mailto: ReproducibilITea@gmail.com">get in touch</a>.</p>ReproducibiliTea Steering CommitteeThe ReproducibiliTea Steering Community is expanding and changing yet again to make sure that we keep the early career researcher (ECR) perspective to building open science communities as well as allowing for more diverse perspectives. This year, we are welcoming two new committee members who will bring fresh impulses to our work while saying thank you and goodbye to our fantastic former committee chair Jan Vornhagen.ReproducibiliTea in 20232023-01-25T14:00:00+00:002023-01-25T14:00:00+00:00/jc/2023/01/25/ReproducibiliTeaIn2023<p>In late 2020, we decided to expand the <a href="/jc/2021/01/28/A-new-year,-a-new-ReproducibiliTea-Steering-Committee.html">ReproducibiliTea Steering Community</a> in order to make sure, we keep the early career researcher (ECR) perspective to building open science communities as well as allowing for more diverse perspectives.
Consequently last year we started a new call for applications to again bring fresh impulses to our work.</p>
<p>So without further ado:</p>
<p><strong>Please join us in welcoming our new committee members:</strong></p>
<p>Helena Gellersen</p>
<p>Helena received her PhD in psychology from the University of Cambridge where she was also a co-organiser of the ReproducibiliTea journal club. She is now a postdoc at the German Center for Neurodegenerative Diseases.</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/Helena.png">
<object data="/assets/images/blog/Helena-small.png" type="image/png">
<img src="/assets/images/blog/Helena.png" alt="Helena Gellersen" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p>Paulina Manduch <a href="https://www.linkedin.com/in/paulinamanduch/">(on LinkedIn)</a></p>
<p>Paulina is a psychology student at the SWPS University in Warsaw, Poland, working on her master’s thesis on creativity and memory. Paulina is the chair of the Open Science Society at her university and she co-organises online ReproducibiliTea meetings in the Polish language.</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/Paulina.png">
<object data="/assets/images/blog/Paulina-small.png" type="image/png">
<img src="/assets/images/blog/Paulina.png" alt="Helena Gellersen" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p><strong>And, our new committee chair:</strong></p>
<p>Jan Vornhagen <a href="https://hci.social/@VornhagenJB">@VornhagenJB@hci.social</a></p>
<p>Jan will follow Sam as the chair of the steering committee for 2023.</p>
<p>Jan is a cognitive Psychologist and PhD fellow at the IT university Copenhagen, Denmark. He joined the ReproducibiliTea Steering Committee in 2021.</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/jan.png">
<object data="/assets/images/blog/jan-small.png" type="image/png">
<img src="/assets/images/blog/jan.png" alt="Jan Vornhagen" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p><strong>And, give a warm ‘welcome back’ to our returning members:</strong></p>
<p>Sophia Crüwell <a href="https://twitter.com/cruwelli">@cruwelli</a></p>
<p>Sophia is a PhD student in Philosophy of Science at the University of Cambridge, working on conceptual issues of the replication crisis and also working on empirical metaresearch projects with the QUEST Center in Berlin.
Sophia started the ReproducibiliTea journal clubs at Oxford (with Sam & Amy), Amsterdam (with Angelika Stefan & Florian Wanders), and Berlin (with Corinna Hartling).</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/sophia.png">
<object data="/assets/images/blog/sophia-small.png" type="image/png">
<img src="/assets/images/blog/sophia.png" alt="Sophia Crüwell" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p>Matt Jaquiery <a href="https://twitter.com/MJaquiery">@MJaquiery</a></p>
<p>Matt is a Research Software Engineer at the University of Oxford. He completed a DPhil in Experimental Psychology in January 2023.
Matt manged the Oxford Experimental Psychology ReproducibiliTea while a DPhil student there, and has contributed to various open science projects with the UKRN, FORRT, and other organisations. He does a bunch of the technical stuff for the core ReproducibiliTea organisation.</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/matt.png">
<object data="/assets/images/blog/matt-small.png" type="image/png">
<img src="/assets/images/blog/matt.png" alt="Matt Jaquiery" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p>William Ngiam <a href="https://twitter.com/will_ngiam">@will_ngiam</a></p>
<p>William is a postdoc in the Department of Psychology at the University of Chicago, where he started and co-organizes the ReproducibiliTea journal club!</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/william.png">
<object data="/assets/images/blog/william-small.png" type="image/png">
<img src="/assets/images/blog/william.png" alt="William Ngiam" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p>Sam Parsons <a href="https://twitter.com/Sam_D_Parsons">@sam_d_parsons</a></p>
<p>Sam is a Postdoctoral Research Fellow at the Donders Institute for Brain, Cognition and Behaviour. He started ReproducibiliTea with Sophia and Amy, and also founded the Framework for Open and Reproducible Research Training (FORRT) project.</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/sam.png">
<object data="/assets/images/blog/sam-small.png" type="image/png">
<img src="/assets/images/blog/sam.png" alt="Sam Parsons" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<h1 id="the-coming-year">The coming year</h1>
<p>We have a lot of exciting things planned for next year.</p>
<ul>
<li>There will be a new season of the <a href="https://soundcloud.com/reproducibilitea">ReproducibiliTea Podcast</a></li>
<li>We will expand our offer of <a href="https://rpt-rl.netlify.app/">Reading Lists</a></li>
<li>and we are looking to rework our community outreach.</li>
</ul>
<p>Like last year, we will also reflect on our recruiting process. We will add that soon.
In the mean time, we are always looking for ideas. If you yourself want to get involved, have a cool idea you think we could tackle, or have some critique for us, please <a href="mailto: ReproducibilITea@gmail.com">get in touch</a></p>
<p>In conclusion: A new year, a new steering committee and a lot of things to do.</p>
<p>We are looking forward to it.</p>Jan B. VornhagenIn late 2020, we decided to expand the ReproducibiliTea Steering Community in order to make sure, we keep the early career researcher (ECR) perspective to building open science communities as well as allowing for more diverse perspectives. Consequently last year we started a new call for applications to again bring fresh impulses to our work.A new year, a new ReproducibiliTea Steering Committee!2021-01-28T14:00:27+00:002021-01-28T14:00:27+00:00/jc/2021/01/28/A-new-year,-a-new-ReproducibiliTea-Steering-Committee<p>ReproducibiliTea is a grassroots initiative, largely led by ECRs.
ReproducibiliTea is evolving worldwide, and our steering committee needs to as well.
In 2020, we set out plans for rotating the steering committee and posted a <a href="/posts/#ReproducibiliTea%20Steering%20Committee%20Application%20Announcement">call for applications</a> in December 2020.
Now, we are excited to introduce you all to our updated steering committee!</p>
<p><strong>Please join us in welcoming our new committee members:</strong></p>
<p>Jan Vornhagen <a href="https://twitter.com/VornhagenJB">@VornhagenJB</a></p>
<p>Jan is a cognitive Psychologist and Doctoral Candidate in Computer Science at the AERIS Lab - Aalto University, Finland - where he also runs the local, HCI-focused RTea chapter</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/jan.png">
<object data="/assets/images/blog/jan-small.png" type="image/png">
<img src="/assets/images/blog/jan.png" alt="Jan Vornhagen" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p>Alexa von Hagen <a href="https://twitter.com/alexavonhagen">@alexavonhagen</a></p>
<p>Alexa is currently a postdoc in the field of school psychology at Goethe University Frankfurt in Germany. Alexa first started with ReproT Singapore last year, but am now contributing to Frankfurt ReproT.</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/alexa.png">
<object data="/assets/images/blog/alexa-small.png" type="image/png">
<img src="/assets/images/blog/alexa.png" alt="Alexa von Hagen" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p>William Ngiam <a href="https://twitter.com/will_ngiam">@will_ngiam</a></p>
<p>William is a postdoc in the Department of Psychology at the University of Chicago, where he started and co-organizes the ReproducibiliTea journal club!</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/william.png">
<object data="/assets/images/blog/william-small.png" type="image/png">
<img src="/assets/images/blog/william.png" alt="William Ngiam" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p><strong>And, our new committee chair:</strong></p>
<p>Sam Parsons <a href="https://twitter.com/Sam_D_Parsons">@sam_d_parsons</a></p>
<p>Sam replaces Amy Orben as the chair of the steering committee.
He is a Postdoctoral Research Associate in Experimental Psychology at the University of Oxford.
He started ReproducibiliTea with Sophia and Amy, and also founded the Framework for Open and Reproducible Research Training (FORRT) project.</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/sam.png">
<object data="/assets/images/blog/sam-small.png" type="image/png">
<img src="/assets/images/blog/sam.png" alt="Sam Parsons" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p><strong>And, give a ‘welcome back’ to our returning members:</strong></p>
<p>Matt Jaquiery <a href="https://twitter.com/MJaquiery">@MJaquiery</a></p>
<p>Matt is an Oxford DPhil student on secondment to the UKRN as Coordinator for Open Research Training.
He manages the Oxford Experimental Psychology ReproducibiliTea and does a bunch of technical stuff for the core organisation.</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/matt.png">
<object data="/assets/images/blog/matt-small.png" type="image/png">
<img src="/assets/images/blog/matt.png" alt="Matt Jaquiery" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p>Sophia Crüwell <a href="https://twitter.com/cruwelli">@cruwelli</a></p>
<p>Sophia is a PhD student in Philosophy of Science at the University of Cambridge, working on conceptual issues of the replication crisis and also working on empirical metaresearch projects with METRIC-B in Berlin.
Sophia started the ReproducibiliTea journal clubs at Oxford (with Sam & Amy), Amsterdam (with Angelika Stefan & Florian Wanders), and Berlin (with Corinna Hartling).</p>
<figure class="image">
<div class="fig">
<a href="/assets/images/blog/sophia.png">
<object data="/assets/images/blog/sophia-small.png" type="image/png">
<img src="/assets/images/blog/sophia.png" alt="Sophia Crüwell" />
</object>
</a>
</div>
<figcaption></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p>Katie</p>
<p>Katie is a PhD student in metaresearch at the University of Bristol.
She is deputy lead at the University of Bristol Reproducibility, and is responsible for the ReproducibiliTea blog.</p>
<h1 id="reflections">Reflections</h1>
<p>In our <a href="/posts/#ReproducibiliTea%20Steering%20Committee%20Application%20Announcement">call for applications</a>, We asked applicants to submit their CV, a cover letter and 100 words on how they thought ReproducibiliTea could improve in the next two years. We received six applications for steering committee positions, and one for the steering committee chair position.</p>
<p>The selection process looked like this:</p>
<ol>
<li>We separately read each application and marked whether we felt the candidate demonstrated each of our selection criteria - the applications were not blinded.</li>
<li>We then met as a group and collated these ratings.
We discussed any differences until we came to a consensus.</li>
<li>We discussed each applicant’s 100 word statement on how ReproducibiliTea can be improved.
We felt it was the most useful as an insight into each applicant’s unique perspective on where ReproducibiliTea can grow and best support our members.</li>
<li>Finally, we collated the scores and notes on each candidate and decided to invite three candidates to fill positions on the steering committee.</li>
</ol>
<p>Every application was impressive, and we are not merely paying lip service to say that the selection process was difficult.
It gave us all cause to reflect on how we can improve the process.</p>
<p>Selecting new committee members is not something we undertook lightly.
From developing the selection process and posting the <a href="/posts/#ReproducibiliTea%20Steering%20Committee%20Application%20Announcement">call for applications</a>, we were aware that we had the responsibility to ensure fairness.
At the same time, we knew that the process would be imperfect.
In the interests of transparency, and to ensure that our selection process improves, we each reflected on the process.</p>
<h2 id="pros-and-cons-of-the-100-word-statement-on-improving-reproducibilitea">Pros and cons of the 100 word statement on improving ReproducibiliTea</h2>
<ul>
<li>Give applicants more detail, for example we could have been clearer about what we wanted from the 100 words statement on improving ReproducibiliTea.
We found that the statements we were more positive about were those that included more specific ideas or plans that the candidate wanted to enact.
Next time, we can rephrase this item along the lines of “100 words on how you plan to help ReproducibiliTea improve”.</li>
<li>It’s also worth mentioning a reflection from one of our applicants on the 100 word summary.
<em>“I definitely found this 100 word document the hardest to prepare.
On one hand, I had lots of ideas I wanted to share and I felt 100 words was not enough for this.
On the other hand, I felt like I needed to know more about the broad next steps you have in mind for ReproT to be able to suggest ideas that could connect to that and build on all your previous work.”</em></li>
<li>We also reflected that we gained more insight from the 100 word statement than other application materials.
We found CVs the least useful in our decision making process, which might reflect how tailored CVs are to academic or research positions, rather than the steering committee requirements.
We might consider asking several short answer questions like this in the future.</li>
</ul>
<h2 id="selection-criteria">Selection criteria</h2>
<ul>
<li>We should use more concrete and direct questions.
Steering committee members marked some of the selection criteria differently.
A strength of our discussion was that we were able to resolve these and come to an agreement.
We should agree how each criteria will be met and judged before advertising positions.
We must be clearer in the call for applications how applicants can demonstrate that they fulfil the selection criteria.</li>
<li>One of the selection criteria was ‘early career status’.
Next time, we might consider specifically asking for graduate student applicants and postdoc applicants separately.
Committee members will academically ‘age’ quickly, and we want to encourage applications from as early as possible.
This will also help us navigate some of the differences we observed between candidates pre- and post- phd, insofar as experiences with application processes will equip those later in their academic career to write strong applications.</li>
<li>We will remove “communication skills” from the criteria.
This criteria opens the possibility for the committee to judge applications from e.g. people with english as a second language or neurodiverse individuals.</li>
<li>We may need to update selection criteria as the committee roles evolve.
For example, positions such as treasurer and communications officer may require different skills and experiences.</li>
</ul>
<h2 id="representation">Representation</h2>
<ul>
<li>We need to think more about diversity and how people who do have different backgrounds from us will be able to meet the selection criteria.
We should not expect people to need to self-disclose things such as being neurodiverse, and should strive to create criteria that include everyone.
Some of the above reflections speak to this point, however we should keep this firmly in mind.</li>
<li>Two of our selection criteria were “Connection to academia in parts of the world underrepresented by ReproducibiliTea” and “Connection to academic disciplines underrepresented by ReproducibiliTea”.
On reflection, these statements are quite vague and we will need to consider how to better phrase these and ask applicants to highlight these aspects.</li>
<li>We cannot escape having previous experience with applicants.
ReproducibiliTea is well connected via social media, and as we pursue a more integrated community and links between journal clubs we should expect more connections between the steering committee and applicants.
There is no easy solution to this as it offers some benefits (e.g. a demonstration of more widespread efforts to improve ReproducibiliTea or reach wider groups) as well as drawbacks (e.g. failing to see an applicant’s full potential or understanding their perspective).
In any case, we need to continue to reflect on how this subjectivity interacts with our assessments, particularly with respect to diversity of applicants.</li>
</ul>Sam ParsonsReproducibiliTea is a grassroots initiative, largely led by ECRs. ReproducibiliTea is evolving worldwide, and our steering committee needs to as well. In 2020, we set out plans for rotating the steering committee and posted a call for applications in December 2020. Now, we are excited to introduce you all to our updated steering committee!ReproducibiliTea Steering Committee Application Announcement2020-12-10T16:33:03+00:002020-12-10T16:33:03+00:00/sc/2020/12/10/Steering-committee-applications<p>We are extremely excited to announce that we are opening applications to join the ReproducibiliTea Steering Committee at the beginning of 2021!
Having recently reached our 100th Journal Club, we are looking for people with commitment, energy and creativity to join our team.
By joining the ReproducibiliTea Steering Committee you have the opportunity to shape one of the most exciting and dynamic Open Science groups over the coming years.</p>
<p>We are looking for 2-3 committee members and 1 chair.
Both committee members and chair are appointed for a stint of two years; however they can leave their position after one year.
More details of the application procedure and roles can be found below.</p>
<h2 id="application-procedure">Application Procedure</h2>
<p>Please send the following 3 documents (as 3 separate PDFs) to <a href="mailto:reproducibilitea@gmail.com?subject=Application_">reproducibilitea@gmail.com</a> with the email subject line “Application_YOURLASTNAME” (e.g. Application_ORBEN).</p>
<ol>
<li>A 1-2 page CV</li>
<li>
<p>A 1 page cover letter (200-400 words) that notes a) which position you are applying for and b) why you think you will be a good fit for the position you are applying for.</p>
</li>
<li>100 words on how you think ReproducibiliTea could improve in the next 2 years</li>
</ol>
<p>Applications are open until January 10th 2021 (midnight GMT).</p>
<p>In your cover letter, please make sure you address the “essential skills” outlined below in the <a href="#role-outlines">Role Outlines</a> section.
If you have any of the “desirable skills” please also tell us that in your cover letter.</p>
<p>Applications will be evaluated against the essential and desirable criteria by current Steering Committee Members.
If the members cannot meaningfully discriminate between two applications which are equally good, the position will be decided by lottery.
The committee members also reserve the right to ask applicants to attend a short interview if that is deemed necessary.</p>
<h2 id="role-outlines">Role Outlines</h2>
<h3 id="steering-committee">Steering Committee</h3>
<h4 id="currently-s-crüwell-s-parsons-j-pickering-k-drax-m-jaquiery">Currently: S Crüwell, S Parsons, J Pickering, K Drax, M Jaquiery</h4>
<p>The Steering Committee is the managing body of ReproducibiliTea, who make collective decisions about the organization’s structure, aim, membership and activities.
The Chair of the Steering Committee can decide whether the group be divided into separate roles (secretary, treasurer etc.) or whether roles are freely shared.
The Group engages in regular meetings to discuss any issues pertaining to the organisation and how to best support the diverse and large ReproducibiliTea membership.
The Steering Committee members commit to attending team meetings, and to dedicate ~3 hours a month (this can fluctuate) to helping run the organisation.</p>
<h4 id="essential-skills">Essential skills:</h4>
<ul>
<li>great communication skills</li>
<li>great teamwork skills</li>
<li>clear passion for Open Science/Open Research/Reproducibility</li>
<li>enthusiasm to improve ReproducibiliTea</li>
<li>experience attending a ReproducibiliTea journal club</li>
</ul>
<h4 id="desirable-skills">Desirable skills:</h4>
<ul>
<li>experience organizing a ReproducibiliTea journal club and/or contributing to ReproducibiliTea</li>
<li>leadership experience</li>
<li>connection to academia in parts of the world currently underrepresented by ReproducibiliTea</li>
<li>connection to academic disciplines currently underrepresented by ReproducibiliTea</li>
<li>self-identified early career researcher (or equivalent) status</li>
</ul>
<h3 id="chair-of-the-steering-committee">Chair of the Steering Committee</h3>
<h4 id="currently-a-orben">Currently: A Orben</h4>
<p>The role of Chair is to manage and guide the Steering Committee, ensuring that the committee can work effectively and is addressing important issues and activities that will benefit the ReproducibiliTea membership.
The Chair also serves as a spokesperson for the ReproducibiliTea organisation, meeting and liaising with stakeholders and representing the interests of the members.
The Chair commits to dedicate ~1-2 hours a week to ensuring the successful administration of the ReproducibiliTea organisation.</p>
<p>If you are applying for Steering Committee Chair, we will consider your application for Steering Committee Member positions if you are unsuccessful in securing the Chair position.
If you do not want to be considered as a Steering Committee Member, please make that explicit in your cover letter as well.</p>
<h4 id="essential-skills-1">Essential skills:</h4>
<ul>
<li>proven leadership experience</li>
<li>great communication skills</li>
<li>great teamwork skills</li>
<li>clear passion for Open Science/Open Research/Reproducibility</li>
<li>enthusiasm to improve ReproducibiliTea</li>
<li>experience attending a ReproducibiliTea journal club</li>
</ul>
<h4 id="desirable-skills-1">Desirable skills:</h4>
<ul>
<li>experience organizing a ReproducibiliTea journal club and/or contributing the ReproducibiliTea</li>
<li>connection to academia in parts of the world currently underrepresented by ReproducibiliTea</li>
<li>connection to academic disciplines currently underrepresented by ReproducibiliTea</li>
<li>self-identified early career researcher (or equivalent) status</li>
</ul>
<p><em>Note: The current ReproducibiliTea Steering Group acknowledges that the current ReproducibiliTea leadership fails to be representative of our growing membership around the world.
We are committed to try to ensure that we become more representative over time.
As we currently still have a limited pool of possible Steering Group members we are not yet in a position to explicitly designate certain positions of the committee for certain groups of our membership.
In the first application round 2020/21 we will therefore reach out directly to a wide range of ReproducibiliTea journal club organisers to encourage more diverse application, with the hope that we can take stronger action in the coming years.</em></p>Steering CommitteeWe are extremely excited to announce that we are opening applications to join the ReproducibiliTea Steering Committee at the beginning of 2021! Having recently reached our 100th Journal Club, we are looking for people with commitment, energy and creativity to join our team. By joining the ReproducibiliTea Steering Committee you have the opportunity to shape one of the most exciting and dynamic Open Science groups over the coming years. We are looking for 2-3 committee members and 1 chair. Both committee members and chair are appointed for a stint of two years; however they can leave their position after one year. More details of the application procedure and roles can be found below.What can open science do for COVID-19?2020-04-02T20:00:00+00:002020-04-02T20:00:00+00:00/jc/2020/04/02/What-can-open-science-do-for-COVID-19<h1 id="open-covid-19-research">Open COVID-19 research</h1>
<p>On the 24 March 2020, I organised the first online <a href="https://reproducibiliTea.org/journal-clubs/#Bristol">ReproducibiliTea University of Bristol</a> meeting. We had 12 attendees, 6 from the University of Bristol and 6 from other institutions who saw the event advertised on my Twitter account and the <a href="https://reproducibiliTea.org/calendar">ReproducibiliTea Online calendar</a>.</p>
<p>Amazingly, we started on time with no technical issues and after a short introduction we got stuck into the main topic of debate: “What can open science do for COVID-19?”.</p>
<p>There was concern that more shared data, code and materials may mean more misinformation: Twitter is exploding with dodgy, over-simplified analyses using open COVID-19 data posted by people with no apparent research experience or credentials. More regression plots and visualisations of death counts, case counts and comparisons by country appear every day.</p>
<p>Some of us questioned if openness is really the problem. Many people do bad science with less transparent research practices, and open research practices allow us to identify problems and call them out. If people misuse open research materials we will know, because they are open.</p>
<p>The issue of misinformation may also stem from how people use, or abuse, open research. Why people access COVID-19 data and what problem they try to answer with it may determine if their answers harm or help others. For example, some people who post regression plots on Twitter maybe chasing likes, followers and clicks, rather than trying to inform effective public policy.</p>
<p>Ultimately, there is a “ton of garbage” but in amongst it is well-designed treasures, which are only possible on a large scale with shared data. Finding a way to sift through the rubbish and evaluate the helpful stuff could be very valuable.</p>
<p>We knew of many examples of free, publicly available COVID-19 data, one of the most famous being the <a href="https://github.com/CSSEGISandData/COVID-19">dataset</a> underlying the 2019 Novel Coronavirus Visual Dashboard operated by the Johns Hopkins University.
We were less aware of COVID-19 simulation studies with available code.
This may be concerning since some simulation studies have had major impacts on public policy during the pandemic, such as the <a href="https://www.imperial.ac.uk/media/imperial-college/medicine/sph/ide/gida-fellowships/Imperial-College-COVID19-NPI-modelling-16-03-2020.pdf">“COVID-19 reports”</a> from Imperial College London.
It is particularly important to share the simulation code of these powerful papers, otherwise we will struggle to precisely replicate the methods and evaluate the model’s validity.
For example, the code underlying a <a href="https://www.imperial.ac.uk/media/imperial-college/medicine/sph/ide/gida-fellowships/Imperial-College-COVID19-NPI-modelling-16-03-2020.pdf">particularly influential “COVID-19 report”</a> is not currently available. The authors are now trying to share their code but <a href="https://www.imperial.ac.uk/media/imperial-college/medicine/sph/ide/gida-fellowships/Imperial-College-COVID19-NPI-modelling-16-03-2020.pdf">are finding it difficult</a>.</p>
<p>Overall, we all agreed that sharing accessible and reuseable code, data and materials benefitted COVID-19 research but differed on who should be allowed to access these materials and how easily.</p>
<h1 id="student-covid-19-projects">Student COVID-19 projects</h1>
<p>We then moved onto a new discussion: “Should we get students to analyse COVID-19 data as part of a student assignment?”.</p>
<p>People thought this idea was exciting. Unlike many student projects, the research question would be incredibly relevant, so it offers students the chance to apply their skills to a real-world problem. This will probably make the project very engaging and interesting for students. The availability of open COVID-19 datasets also makes such a project quite feasible, although some datasets are missing metadata.</p>
<p>On the other hand, the topic’s relevance may be a problem. It is an anxious time for many and allowing students to think about other things may benefit their wellbeing. Lecturers could offer an alternative assignment for students who do not want to work with COVID-19 data, but this may be tricky to manage. The potential impact of students’ work has positive and negative aspects. If students share their work, they could contribute to the public’s understanding of COVID-19 and this may spread more misinformation. However, learning to think about the impact your work may have and what it means to disseminate research is an important lesson in itself. Sharing their work more widely may also encourage students to double-check their work and reflect on its ethical aspects and the uncertainties of the (meta) data. Lecturers could also try to minimise the risk of students spreading misinformation by limiting how informative or controversial their interpretations could be, perhaps by controlling how much data students get.</p>
<p>Overall, the greater potential impact of a project increases both the potential benefits and risks. Lecturers would need to weigh up the risks of a COVID-19 project, such as students spreading misinformation or overloading anxious students, with the benefits, such as a highly relevant research question.</p>
<div style="display: flex">
<object data="/assets/images/blog/prereg-badge-small.original-small.png" type="image/png">
<img src="/assets/images/blog/prereg-badge.original.png" alt="Preregistration thumbnail. Photo credit preregistered_small_color.png by Open Science Collaboration is licensed under CC BY 4.0" style="height: 40px; width: 40px; margin-right: 20px;" />
</object>
<h1 id="preregistering-covid-19-research">Preregistering COVID-19 research</h1>
</div>
<p>I then put a slightly different question to the group: did we think preregistration should be used for COVID-19 research, and if so how?”.</p>
<p>This question slightly divided us. For some, preregistration is what you make it. You control how much time and detail you put into your preregistration, so it could be fast. Even though we may be doing a lot of exploratory work in the context of this world-wide pandemic, it always makes sense to write down what you plan to do, regardless of how long it takes you.</p>
<p>Two attendees working in the computer and data sciences disagreed. Preregistration may be beneficial, but it is both hard to do and to discuss in computer science. One researcher wanted to pre-register their studies but found it difficult to do. The second researcher saw two different worlds within computer science: one building methods, the other applying methods. Preregistration felt particularly difficult for people who work on theoretical methods, who try many different things, throw away lots of stuff that doesn’t work, and don’t remember what they threw away. This researcher thought that preregistration is more suitable for applied computer scientists; for example, machine learning and language processing could gain something with pre-registration.</p>
<h1 id="registered-reports">Registered Reports</h1>
<p>One attendee asked if they should submit a <a href="https://web.archive.org/web/20200402210750/https://cos.io/rr/">Registered Report</a> now, given that the timescale for collecting data is so uncertain. In response, someone argued that COVID-19 shouldn’t hold you back from submitting a Registered Report because COVID-19 will likely affect everyone conducting similar studies and journals will hopefully understand the situation and be flexible. Now may also be a great time to think more about exploratory open research initiatives, for example, the Exploratory Reports format currently offered by the <a href="https://web.archive.org/web/20200327155633/https:/www.rips-irsp.com/about/exploratory-reports/">International Review of Social Psychology</a> and <a href="https://web.archive.org/web/20190610122655/https:/www.elsevier.com/__data/promis_misc/Exploratory_Reports_Guidelines.pdf">Cortex</a>.</p>
<h1 id="final-thoughts">Final thoughts</h1>
<p>Overall we had a lively, relaxed discussion focusing on the roles that sharing research, Registered Reports, student projects and preregistration can play in COVID-19 research. I encourage others to hold relaxed, virtual discussions, and we’ve pooled our experience at ReproducibiliTea to offer some <a href="https://reproducibiliTea.org/online">guidelines on hosting</a> these discussions.</p>
<p><em><a href="https://twitter.com/katiedrax">Katie Drax</a> is a PhD student at the University of Bristol studying meta-research. She founded and organises the ReproducibiliTea University of Bristol branch.</em></p>Kate DraxOpen COVID-19 researchReproducibiliTea does RepliCATS2020-02-22T11:51:00+00:002020-02-22T11:51:00+00:00/jc/2020/02/22/ReproducibiliTea-does-RepliCATS<figure class="image">
<div class="fig">
<a href="/assets/images/blog/replicats-oxford.jpg">
<object data="/assets/images/blog/replicats-oxford-small.jpg" type="image/jpg">
<img src="/assets/images/blog/replicats-oxford.jpg" alt="ReproducibiliTea Oxford does RepliCATS" />
</object>
</a>
</div>
<figcaption><a href="/journal-clubs/#Oxford">ReproducibiliTea Oxford</a> RepliCATS session. Photo credit <a href="https://twitter.com/IlsePit">@IlsePitt</a></figcaption>
</figure>
<!-- Credit https://stackoverflow.com/a/19360305 -->
<p>Last week’s ReproducibiliTea at <a href="/journal-clubs/#Oxford">Oxford Experimental Psychology</a> (honourable mention to our Anthropology regulars!) went on a little longer than our usual one-hour of snacks and journal articles. We, like a few other ReproducibiliTea clubs, teamed up with the <a href="https://replicats.research.unimelb.edu.au/">RepliCATS</a> project and estimated the replicability of research studies for the benefit of science (and pizza).</p>
<p>repliCATS (<a href="https://twitter.com/replicats">@replicats</a>) is a project aiming to understand the replicability of social and behavioural sciences and how well replicability can be predicted. There are three arms to the project: gathering researchers’ assessments of replicability of specific claims in papers across a broad range of disciplines, running replications of a subset of the assessed claims and attempting to predict replicability using machine learning techniques. The ReproducibiliTea sessions help with the first of these arms.</p>
<p>We spent around 30 minutes signing up to the repliCATS platform, which involves completing an interesting questionnaire about areas of expertise, metascience knowledge, and statistical knowledge, and watching the introduction video. Over the next couple of hours, we went over four claims using the platform’s <a href="https://replicats.research.unimelb.edu.au/sample-page/faqs/">IDEA approach</a>. The IDEA protocol grounds itself on group decision-making literature which pleased me, as a graduate student studying group decision-making. Individuals make their judgements about the claim, then they have an opportunity to discuss those judgements with others. After discussion, each person makes a final decision in private. This allows for both a diversity of estimates and reasons as well as minimising groupthink effects, at least in principle. As a group, many of us were quite effusive (especially me) so there may have been some cross-pollination of ‘individual judgements’ due to facial expressions, quizzical sounds, or irrepressible comments.</p>
<p>Individuals judge how readily each claim can be understood, whether it appears plausible, and how many of 100 direct replications of the claim are likely to show an effect in the same direction (with alpha = .05). This seems pretty straightforward but, in practice, we often found it quite tricky…</p>
<p>For some of the claims, we found that the claim extracted by the platform for assessment bore little resemblance to the inferential test put forward to test it. For example, one claim was that people in condition A would out-perform those in conditions B and C. This claim was ‘tested’ by a t-test of condition A performance vs chance. These discrepancies sometimes happened because the authors didn’t do a good job of testing the claim, but in others the platform selected the wrong test. Also, in some cases, the N supplied for the test didn’t match the actual N reported in the paper or the test’s degrees of freedom. These issues made it difficult to work out whether we should assess the replicability of the claim or the inferential test.<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup></p>
<p>For other claims, the tests seemed entirely misguided. The authors of one paper seemed to divide teachers into two groups based on whether their teaching was improving. They then used an inferential test to show that the ‘improving’ group improved relative to the non-improving group. That test seemed pretty pointless in terms of a meaningful claim, and it wasn’t very clear what we should assess. We could assess the probability that for any random set of 6 teachers you’ll be able to assign them to improving vs stable groups, or we could assess the claim that tautological t-tests should come up significant.</p>
<p>Of the four claims we investigated as a group, the most sensible one was that juvenile offenders are more likely to be placed in young offenders’ institutions if their family assesses as dysfunctional. We were pretty happy with the inferential test for that claim and it seemed intuitive. The only reason we came up with to doubt its replicability related more to the claim’s generalisability across time than the robustness of the study. The claim seemed to show a behaviour of the courts which could be subject to political correction which would make it hard to find the same effect if courts used in future replications behaved differently (perhaps as a result of the original study).</p>
<p>The whole exercise brought to light a good many considerations of what we mean by replicability and which factors we should or should not care about. It was also like a rapid-fire journal club with pizza, which is about the best way to spend an academic afternoon! I really like the ambition of the whole repliCATS project, from the breadth and depth of the disciplines covered to the idea of (responsibly) running machine learning on the studies. It’s a very cool project and I’m really looking forward to seeing how it evolves.</p>
<p><em>Matt Jaquiery <a href="https://verysmalldreams.com/">(verysmalldreams.com)</a> is a final year DPhil student and organises the University of <a href="/journal-clubs/#Oxford">Oxford journal club</a> with Sam Parsons, Paul Thompson, Nils Reimer, Sarah Ashcroft-Jones, and Sam Webb.</em></p>
<h3 id="notes">Notes:</h3>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>According to RepliCATS, the answer to this is that the inferential test is the focus rather than the verbal claim. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>Matt JaquieryReproducibiliTea Oxford RepliCATS session. Photo credit @IlsePittAn undergraduate’s experience of ReproducibiliTea2020-01-20T10:00:00+00:002020-01-20T10:00:00+00:00/jc/2020/01/20/An-undergraduate's-experience-of-ReproducibiliTea<p>The <a href="https://activity.ussu.co.uk/surreyreproducibility">Surrey Reproducibility Society</a> is made up of students and staff who meet once a fortnight to explore issues relating to open research and engage in presentations and training sessions aimed at helping all members of the University implement best practice research methods.
RepreoducibiliTea journal club sessions take place at least once a month and help to facilitate further development of our understanding of open, reproducible and transparent research.</p>
<p>Before my first ReproducibiliTea meeting, I wasn’t entirely sure what open research was.
The replication crisis was mentioned on my course, and I understood that not all research was good research, but that was the extent of my knowledge.
Now, despite joining the society just 3 months ago, I can explain the main open research principles, identify both good and bad practice and I have a much better grasp of where research, especially psychological research, sits in today’s research climate.
I’m even working on a project exploring undergraduate’s understanding of open research as part of my placement as a research assistant.</p>
<p>Attending these meetings has been invaluable, not only by helping me in a practical sense, such as walking me through <a href="https://cos.io/rr/">Registered Reports</a> and <a href="https://cos.io/prereg/">preregistrations</a>, but it has also influenced how I view research more generally.
Many questionable research practices have been discussed in the group.
For some of these, I was completely unaware of these practices before joining the society, for others, what’s worse is that I was aware of the practices and saw no harm in them.
The sessions have really improved my understanding of what good research practice is.</p>
<p>I think that the ReproducibiliTea meetings are a great resource for both established researchers and students, allowing us to question what the norms in our field are, and whether these norms are best practice.
By getting researchers and students involved in these meetings, we can start talking about immediate change in research and we can introduce a new generation of researchers to open research methods.
Not every student will become a researcher, but every future researcher will have been a student, and so the discussion around good practice needs to include students too.
The Surrey Reproducibility Society is an excellent arena for this kind of inclusive discussion where researchers at different levels of their career discuss these ideas together.
As a student, being able to discuss these issues alongside established researchers has been really valuable.</p>
<p>The issue of reproducibility and ‘good science’ is a complex one, but with the growing awareness of practices such as registered reports and preregistration, I think positive changes are coming. ReproducibiliTea increases this awareness, and I hope it inspires future researchers to do better, it has definitely inspired me.</p>
<p><em>Ashley Williams is an undergraduate psychology student at the University of Surrey, currently undertaking a placement within the CoGDeV Lab, and has been a ReproducibiliTea member for 3 months.</em></p>Ashley WilliamsThe Surrey Reproducibility Society is made up of students and staff who meet once a fortnight to explore issues relating to open research and engage in presentations and training sessions aimed at helping all members of the University implement best practice research methods. RepreoducibiliTea journal club sessions take place at least once a month and help to facilitate further development of our understanding of open, reproducible and transparent research.Reflections and UK Reproducibility Network Reporting2019-12-20T00:00:00+00:002019-12-20T00:00:00+00:00/jc/2019/12/20/Reflections-and-UK-Reproducibility-Network-Reporting<p>It has been an absolutely amazing year for ReproducibiliTea.
Starting with only a handful of journal clubs at the beginning of 2019, we now count 53 different constituent journal clubs in 17 diverse countries around the world. This has brought with it new opportunities and also new challenges.
A stellar group of core ReproducibiliTeam members, who volunteer their time to keep ReproducibiliTea running, have taken it in their stride to build a functioning website and workflow that accommodates our rise in numbers, to offer support through our Slack community, to publish blog posts and to continually learn more about the unique challenges journal clubs face and how we can help tackle them.</p>
<p>There are many great initiatives waiting to be launched through the ReproducibiliTea network in 2020, but I will take this short post to reflect on the financial support we have received this year.
Our evident success in the last year has been crucially supported by the UK Reproducibility Network (UKRN) who help publicise our work to communities that we cannot regularly reach (funders, journal editors, university administrators, libraries) and also support our work with generous financial funding. For this, we are truly thankful.</p>
<p>To provide transparency about our organisational setup, I will detail the types of funding we have received and what they are being used for.
The nature and amount of this funding were negotiated with UKRN at the end of October 2019 to match both the goals of the UKRN organisation and ReproducibiliTea.
Firstly to promote our organisation online we received £88 to buy our domain name for 3 years, and to promote ReproducibiliTea internationally we received £283 to buy our beloved Hexstickers that I send out to new journal clubs on a regular basis.
As UKRN is committed to fostering open science in the UK specifically, we also received £1,500 to continue our UK starter pack initiative: where UKRN sends a box of goodies, a teapot and stickers to any new UK based ReproducibiliTea journal club.
We are also hoping to organise an event next year to bring together ReproducibiliTea organisers in one place, for which we were given £800 of funding.
This will only be received if other funding applications are successful and a dedicated group have been fundraising for the event in the last months.
Lastly, we received £590 to further promote and produce the ReproducibiliTea podcast, which highlights ECR issues and allows us to reach those who cannot start a journal club at their institutions.</p>
<p>In 2020 we will focus on how we can support and promote the very diverse range of journal clubs around the world, and ensure that the organisers are supported as much as possible.
To do so we are hoping to obtain more funding both in the UK and internationally. Looking back 2019 has been hugely productive, and we are truly excited about what the new year will bring for ReproducibiliTea.</p>
<p><em>Amy Orben is a member of the ReproducibiliTeam and a college research fellow at the University of Cambridge. She is responsible for high-level organisation of ReproducibiliTea journal clubs and liaising with the UKRN</em></p>Amy OrbenIt has been an absolutely amazing year for ReproducibiliTea. Starting with only a handful of journal clubs at the beginning of 2019, we now count 53 different constituent journal clubs in 17 diverse countries around the world. This has brought with it new opportunities and also new challenges. A stellar group of core ReproducibiliTeam members, who volunteer their time to keep ReproducibiliTea running, have taken it in their stride to build a functioning website and workflow that accommodates our rise in numbers, to offer support through our Slack community, to publish blog posts and to continually learn more about the unique challenges journal clubs face and how we can help tackle them.Building the CommuniTea2019-11-01T10:00:00+00:002019-11-01T10:00:00+00:00/jc/2019/11/01/CommuniTea<p>ReproducibiliTea was originally started in 2018 by Amy Orben, Sam Parsons, and Sophia Crüwell at the University of Oxford as a podcast and journal club. It is now about 18 months old and has started to find its feet; ReproducibiliTea has expanded to 42 journal clubs in 15 countries worldwide at the time of writing. It’s expanding so rapidly that by the time you read this, it will likely be out of date.</p>
<p>Due to this expansion, Amy Orben recruited an organising committee to unite, promote and support the ReproducibiliTea community. Introducing the organising ReproducibiliTeam:</p>
<ul>
<li>Amy Orben <a href="https://twitter.com/OrbenAmy">@OrbenAmy</a>: high-level organisation and UKRN liaising</li>
<li>Sam Parsons <a href="https://twitter.com/Sam_D_Parsons">@Sam_D_Parsons</a>: podcast production</li>
<li>Sophia Crüwell <a href="https://twitter.com/cruwelli">@cruwelli</a>: podcast scheduling and webinars</li>
<li>Matt Jaquiery <a href="https://twitter.com/MJaquiery">@MJaquiery</a>: website design and maintenance</li>
<li>Katie Drax <a href="https://twitter.com/katiedrax">@katiedrax</a>: external communications</li>
<li>Jade Pickering <a href="https://twitter.com/Jade_Pickering">@Jade_Pickering</a>: community building and merchandising</li>
</ul>
<p>My first initiative to build a better sense of communiTea (sorry) was to ask current journal club organisers about their experiences. Was it easy to set up the club? What went well? What didn’t? What could the ReproducibiliTea community do to support them? I distributed these questions, and others, in a survey and received responses from 18 different clubs from Austria, UK, Singapore, Germany, Netherlands, USA and Japan.</p>
<p>Although it’s entirely up to each journal club to decide how they want to run things, many respondents appreciated guidance and ideas. So let’s give the people what they want! Here I have summarised what we learned from the survey, and provided an overview of where we can go from here.</p>
<h1 id="spreading-the-word">Spreading the word</h1>
<p>One of the hardest parts of setting up a ReproducibiliTea journal club seems to be in finding people to attend…a crucial aspect of any meeting! Those clubs with better attendance had a lot of success spreading the word through Twitter and word of mouth. They found advertising through existing university communities, such as postgraduate societies, particularly effective.</p>
<p>A few members of the ReproducibiliTea community felt that the <a href="https://twitter.com/ReproducibiliT">@ReproducibiliT</a> twitter account could do more to support the individual clubs by promoting them. In response to this, the committee will update and rebrand the Twitter account and create an official hashtag. This will allow us to promote local clubs as club organisers and members can use the hashtag when they want to be seen and boosted by the main account.</p>
<h1 id="running-a-club-session">Running a club session</h1>
<p>The respondents gave a description of their standard meeting formats. Most journal clubs follow a generic format. Once every 2-4 weeks the organiser(s) will distribute a paper amongst attendees in preparation for an hour’s session to discuss the paper. Typically, the session leader (normally an organiser) will spend 5-10 minutes summarising the topic of the paper, and then the rest of the session is dedicated to free discussion amongst all attendees, often with prompts from the organiser if the discussion is slower. This is usually accompanied by tea served in a ReproducibiliTeapot, coffee, biscuits, and other snacks. The clue is in the name - refreshments are strongly recommended!</p>
<p>So how do you decide on a paper? There’s a handy list on the <a href="https://osf.io/8r2jg/">Open Science Framework (OSF) page</a> to get you started, and a <a href="https://www.zotero.org/groups/2354006/reproducibilitea/items/">Zotero library</a>. Club organisers are free to add their own suggestions to Zotero if they find any good ones. Over time we can build a curated list of papers together which may reduce the burden on organisers to find papers and willing presenters. Speaking of which, finding a willing presenter is a common problem for many clubs. Some get around this by having one of the organisers choose and present papers but, in the future, it would be ideal to have a mix of presenters across different departments.</p>
<p>A couple of clubs do things differently and, notably, these are the clubs with some of the highest regular attendance rates. At University College London, they have a longer presentation of around 30 minutes with some discussion points throughout before moving on to a freer discussion. The University of Surrey’s ReproducibiliTea club meet under the umbrella of the newly established Surrey Reproducibility Society which is officially registered as part of the Students’ Union; they alternate between the generic format and miscellaneous workshops and presentations.</p>
<p>However you choose to structure your own journal club sessions, there is always an online support community comprising other ReproducibiliTea journal club organisers willing to help you out and support you. Read on for more…!</p>
<h1 id="online-community">Online community</h1>
<p>Setting up your own club can be daunting if you’re not sure where to start. Thankfully, we already have a good community presence…if you know where to find it. It was clear from some respondents that we hadn’t done enough to advertise our community spaces. If you’re a journal club organiser you can access our Slack workspace and invite your members. We use this to share our ideas, woes, accomplishments, and cat (or dog) pictures. The Slack is the best place to be for a sense of community, and to get instant responses from other organisers for any questions or suggestions you might have. Each local branch of ReproducibiliTea can also have their own component on the parent <a href="https://osf.io/3qrj6/">ReproducibiliTea OSF project page</a> and their own page on the <a href="https://reproducibilitea.org/">reproducibilitea.org</a> website. If you’re not on the Slack, OSF, or website and you think you should be - do let us know at <a href="reproducibilitea@gmail.com">reproducibilitea@gmail.com</a>!</p>
<h1 id="bringing-clubs-together-in-real-life">Bringing clubs together in real life</h1>
<p>We live in a glorious age of connectivity online, but that isn’t always a substitute for meeting like-minded people in real life. One of the most popular suggestions/requests in the survey was for an event or gathering where we could get together offline.</p>
<p>Watch this space! ReproducibiliTea: The Big Meeting, coming soon.</p>
<p><em>Jade Pickering <a href="https://www.jade-pickering.com/">(jade-pickering.com)</a> is a final year PhD student and co-leads the University of Manchester journal club with George Farmer, Daniel Poole, and Thomas Richardson.</em></p>Jade PickeringReproducibiliTea was originally started in 2018 by Amy Orben, Sam Parsons, and Sophia Crüwell at the University of Oxford as a podcast and journal club. It is now about 18 months old and has started to find its feet; ReproducibiliTea has expanded to 42 journal clubs in 15 countries worldwide at the time of writing. It’s expanding so rapidly that by the time you read this, it will likely be out of date. Due to this expansion, Amy Orben recruited an organising committee to unite, promote and support the ReproducibiliTea community. Introducing the organising ReproducibiliTeam: Amy Orben @OrbenAmy: high-level organisation and UKRN liaising Sam Parsons @Sam_D_Parsons: podcast production Sophia Crüwell @cruwelli: podcast scheduling and webinars Matt Jaquiery @MJaquiery: website design and maintenance Katie Drax @katiedrax: external communications Jade Pickering @Jade_Pickering: community building and merchandising My first initiative to build a better sense of communiTea (sorry) was to ask current journal club organisers about their experiences. Was it easy to set up the club? What went well? What didn’t? What could the ReproducibiliTea community do to support them? I distributed these questions, and others, in a survey and received responses from 18 different clubs from Austria, UK, Singapore, Germany, Netherlands, USA and Japan. Although it’s entirely up to each journal club to decide how they want to run things, many respondents appreciated guidance and ideas. So let’s give the people what they want! Here I have summarised what we learned from the survey, and provided an overview of where we can go from here. Spreading the word One of the hardest parts of setting up a ReproducibiliTea journal club seems to be in finding people to attend…a crucial aspect of any meeting! Those clubs with better attendance had a lot of success spreading the word through Twitter and word of mouth. They found advertising through existing university communities, such as postgraduate societies, particularly effective. A few members of the ReproducibiliTea community felt that the @ReproducibiliT twitter account could do more to support the individual clubs by promoting them. In response to this, the committee will update and rebrand the Twitter account and create an official hashtag. This will allow us to promote local clubs as club organisers and members can use the hashtag when they want to be seen and boosted by the main account. Running a club session The respondents gave a description of their standard meeting formats. Most journal clubs follow a generic format. Once every 2-4 weeks the organiser(s) will distribute a paper amongst attendees in preparation for an hour’s session to discuss the paper. Typically, the session leader (normally an organiser) will spend 5-10 minutes summarising the topic of the paper, and then the rest of the session is dedicated to free discussion amongst all attendees, often with prompts from the organiser if the discussion is slower. This is usually accompanied by tea served in a ReproducibiliTeapot, coffee, biscuits, and other snacks. The clue is in the name - refreshments are strongly recommended! So how do you decide on a paper? There’s a handy list on the Open Science Framework (OSF) page to get you started, and a Zotero library. Club organisers are free to add their own suggestions to Zotero if they find any good ones. Over time we can build a curated list of papers together which may reduce the burden on organisers to find papers and willing presenters. Speaking of which, finding a willing presenter is a common problem for many clubs. Some get around this by having one of the organisers choose and present papers but, in the future, it would be ideal to have a mix of presenters across different departments. A couple of clubs do things differently and, notably, these are the clubs with some of the highest regular attendance rates. At University College London, they have a longer presentation of around 30 minutes with some discussion points throughout before moving on to a freer discussion. The University of Surrey’s ReproducibiliTea club meet under the umbrella of the newly established Surrey Reproducibility Society which is officially registered as part of the Students’ Union; they alternate between the generic format and miscellaneous workshops and presentations. However you choose to structure your own journal club sessions, there is always an online support community comprising other ReproducibiliTea journal club organisers willing to help you out and support you. Read on for more…! Online community Setting up your own club can be daunting if you’re not sure where to start. Thankfully, we already have a good community presence…if you know where to find it. It was clear from some respondents that we hadn’t done enough to advertise our community spaces. If you’re a journal club organiser you can access our Slack workspace and invite your members. We use this to share our ideas, woes, accomplishments, and cat (or dog) pictures. The Slack is the best place to be for a sense of community, and to get instant responses from other organisers for any questions or suggestions you might have. Each local branch of ReproducibiliTea can also have their own component on the parent ReproducibiliTea OSF project page and their own page on the reproducibilitea.org website. If you’re not on the Slack, OSF, or website and you think you should be - do let us know at reproducibilitea@gmail.com! Bringing clubs together in real life We live in a glorious age of connectivity online, but that isn’t always a substitute for meeting like-minded people in real life. One of the most popular suggestions/requests in the survey was for an event or gathering where we could get together offline. Watch this space! ReproducibiliTea: The Big Meeting, coming soon. Jade Pickering (jade-pickering.com) is a final year PhD student and co-leads the University of Manchester journal club with George Farmer, Daniel Poole, and Thomas Richardson.Sheffield ReproducibiliTea: what happened in our first meeting?2019-10-18T15:00:00+00:002019-10-18T15:00:00+00:00/jc/2019/10/18/Sheffield-ReproducibiliTea<p><a href="/journal-clubs/#Sheffield">Sheffield University</a> held its first ReproducibiliTea meeting in October 2019, with about 15 people attending from medicine, biology, engineering, psychology, and other departments.
<a href="https://twitter.com/stavrhina">Stavrina Dimosthenous</a>, a PhD researcher in materials science, lead the discussion and introduced us to a psychology paper by Pashler and Harris, 2012: <a href="https://doi.org/10.1177/1745691612463401">Is the Replicability Crisis Overblown? Three Arguments Examined</a>.
We discussed what <a href="https://en.wikipedia.org/wiki/P-value">p-values</a> are, what <a href="https://en.wikipedia.org/wiki/Data_dredging">p-hacking</a> is, and how p-hacking can arise when researchers exclude data that doesn’t support their hypothesis.</p>
<p>Our discussion of hypothesis testing led us to think about what happens when a researcher finds a <a href="https://en.wikipedia.org/wiki/Null_result">negative result</a>, i.e. when the data does not support the hypothesis.
In our meeting, some people had seen negative outcomes of clinical trials published in “<a href="https://en.wikipedia.org/wiki/Impact_factor">high impact</a>” journals, because these negative outcomes risked harming patients.
But inconclusive results, when they show no clear outcome either way, <a href="https://en.wikipedia.org/wiki/Publication_bias">often don’t get published</a>.
Despite these gaps in the publication record, some journal club attendees with clinical research experience felt that, in general, experiments were well-designed and publication quality was high.
In other areas where research can be less systematic and more exploratory, such as in computational sciences and engineering, some attendees thought publication quality was more variable, and questioned whether journals and <a href="https://en.wikipedia.org/wiki/Peer_review">peer reviewers</a> had sufficient incentive (or time) to ensure high quality.</p>
<p>The next <a href="/journal-clubs/#Sheffield">Sheffield ReproducibiliTea</a> is at 1pm 14th November, Pam Liversidge E06, where we’ll be covering a recent opinion piece by Daniele Fanelli, 2018: <a href="https://doi.org/10.1073/pnas.1708272114">Is science really facing a reproducibility crisis, and do we need it to?</a>
Log into your Sheffield University account, and you can <a href="https://bit.ly/tuosrepro">join our Google group</a> and <a href="https://calendar.google.com/calendar?cid=c2hlZmZpZWxkLmFjLnVrX3BxYnVmMWQzNDk5dXVyaTgzZDJyYjVvYXJzQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20">subscribe to our Google calendar</a>.
See you there!</p>
<p><em><a href="https://www.datumedge.co.uk">James Shaw</a> is the organiser of Sheffield Uni ReproducibiliTea.
If you’re interested in co-organising our meetings then <a href="https://bit.ly/tuosrepro">please get in touch</a>!</em></p>James ShawSheffield University held its first ReproducibiliTea meeting in October 2019, with about 15 people attending from medicine, biology, engineering, psychology, and other departments. Stavrina Dimosthenous, a PhD researcher in materials science, lead the discussion and introduced us to a psychology paper by Pashler and Harris, 2012: Is the Replicability Crisis Overblown? Three Arguments Examined. We discussed what p-values are, what p-hacking is, and how p-hacking can arise when researchers exclude data that doesn’t support their hypothesis.