This is the first in a series of posts about each of the teams that will be attending SCI 2023 and their projects.

Preprints in journalism: Challenges and opportunities for (public) trust

This project brings together a group of scholars and journalists to investigate questions and tensions surrounding the use of preprints. The COVID-19 pandemic has brought attention to how, whether, and under what circumstances research that has not been peer reviewed should be shared with public audiences. Preprints have also renewed debates about the value and equity of peer review.

While the role of the media in the trust relationship between science and society is important, public engagement in and perceptions of science are also influenced by changes in the scholarly communication landscape. Research is increasingly driven by technological advances and exogenous pressure to publish more, and to do so at speed.

Enter the preprint – an increasingly accepted scholarly publication format, shared openly and without having undergone peer review. Promoted as a method to increase the pace and accessibility of scholarly communication, preprints demonstrated their value during the COVID-19 pandemic. However, preprints lack the journal-centric cues that scholars and journalists have traditionally used to evaluate credibility, such as journal reputation and peer-review status. This raises questions about how to decide whether preprints are reliable sources (Soderberg et al., 2020), as well as how the media and attentive publics interpret and (re)communicate the claims published in preprints. It also invites consideration of new approaches to scholarly publishing, including experimentation with peer review (e.g. eLife).

Despite evidence that preprint-based journalism is becoming a “new normal” (Fleerackers et al., 2022b), and the emergence of some guidelines (Van Schalkwyk & Dudek 2022), journalists as intermediaries between science and the public lack evidence-based best practices for reporting from preprints. Moreover, the institutionalization of preprints in the science communication system draws attention to both the benefits and risks of open science more broadly, and raises questions about the impact of more transparent science on public trust in the institution.

We will bring together a team of emerging scholars and established journalists who have been paying attention to the use of preprints. We will come together for the first time to better understand the societal value and risks of preprints beyond the pandemic. Specifically, we will work together to examine the heuristic cues that journalists use for assessments of research credibility and identify strategies for making credibility judgements while acknowledging the news value dimension of preprints. Equally important is how publics receive and interpret media cues regarding the provisional nature of scientific findings and how the communication of “pre-truth” science can be conveyed to preserve public trust in science.

Unpacking and preventing problematic evaluations of preprints in journalism is an essential step for meeting the promise that preprints hold for opening and accelerating scholarly and public communication of science. It could also help to mitigate risks that are commonly associated with media coverage of preprints, such as the potential to spread flawed research and misinformation. More broadly, developing guidance to verify and evaluate preprint knowledge raises larger philosophical questions about how to assess ‘quality’ in science; when findings can truly be considered confirmed; the veracity and fairness of peer review; and what this means for the evolving trust relationship between science and society. In pursuing these questions, the team hopes to inform future research and practice at the intersections of scholarly communication, journalism, and science communication.

Although our goals may evolve as a result of our participation in Triangle SCI, our collaboration will enable us to develop evidence-based, culturally sensitive, globally relevant, and practical resources to guide reporting on preprints. We also hope to address the important but underexplored questions that arise as preprints become cornerstones of public discourse. We will consider such topics as (1) the value and limits of peer review as a mechanism for ensuring the ‘trustworthiness’ of science, (2) the evolving relationships between science and science communication, (3) the societal challenges brought on by open science, and (4) the usefulness of the criteria traditionally relied on to delineate trustworthy, scientific ‘truths’ from false truth-claims. These questions can form the foundation of a rich research agenda, both for our team and for others interested in the intersections of scholarly communication and public engagement with science.

Our expertise on preprints, science communication, and trust

Our topic touches on many dimensions of trust laid out in the call for proposals. It considers trust as part of a larger communication network that includes both academic and public, analogue and (increasingly) digital channels of knowledge dissemination. We will take into account the interconnected nature of scholarly publishing, journalism and “mass self-communication” (Castells, 2009)—systems that are often treated as separate but that, as the public engagement with preprints has revealed, are in fact interwoven. We will also pay close attention to the infrastructures, norms, and power structures within academia, journalism, and social media. In doing so, our team’s work will critically examine key issues related to the integrity of processes of communication.

Our prior research has demonstrated that even experienced science journalists do not feel qualified to vet the trustworthiness of preprints and are concerned about their potential to contribute to misinformation (Fleerackers et al., 2022a; Massarani et al., 2021a). We have also identified how inconsistently journalists cover preprints (Van Schalkwyk & Dudek, 2022; Massarani & Neves, 2022; Massarani et al., 2021; Fleerackers et al., 2022b) and how poorly their audiences seem to understand this term (Ratcliff et al., 2023). Others have investigated the submission requirements, level of transparency, and research integrity considerations associated with different preprint servers (Malički et al., 2020), and proposed steps for overcoming these issues (Tijdink et al., 2020). Triangle SCI would provide an opportunity to bring together insights from our individual lines of inquiry related to preprints to address essential challenges facing journalists and their audiences, as well as scientists and the preprint servers that house their work.

Finally, our team is balanced in terms of gender and geography, bringing together experts from both the Global North and South, from both research and practice. Together, we could contribute diverse and complementary perspectives on issues of diversity, equity, inclusion, and access. An intercultural, interdisciplinary, and intersectoral lens is essential for addressing the challenges and questions laid out above. Specifically, while US-based journalism organizations have recently started to propose guidelines for reporting on preprints responsibly (e.g., Khamsi, 2020; Miller, 2021), these guidelines were formulated with the Global North in mind and may not be appropriate for journalists working in the Global South, whose access to experts, resources, and editorial support often look very different (Nguyen & Tran, 2019). The journalists on our team—based in Brazil and Malaysia—will provide the experiences and reflections needed to ensure that all outputs of our work (but especially the guidelines) adequately consider the lived realities of journalists covering research in diverse reporting contexts.

Preparation and outcomes

Preparation

To prepare for Triangle SCI, our team will meet on Zoom to share current work, lessons learned, and lingering questions related to preprints, peer review, science journalism, and trust in science. We will take notes in a collaborative document throughout these meetings to structure our conversations at SCI. We will also identify and collect existing guidelines for journalists and other stakeholders regarding the use of preprints, building on our prior work (Van Schalkwyk & Dudek, 2022; Malički et al., 2020a). Doing so will help us to establish a sense of shared understanding of the problem space, and identify the challenges that will be most pressing to address during our time at the institute.

Outcomes

1. Guidelines for journalists: This meeting would enable us to develop media reporting guidelines from an “ecosystem” perspective, drawing together the expertise and prior investigations of each team member to create guidelines that are responsive to the considerations and challenges of each stakeholder group. We intend to draw on our advanced reading, prior research and experiences, and discussions and investigations during the Institute, to develop evidence-based guidelines for journalists covering preprints. With the help of the bilingual members of our team, we will translate these guidelines into Portuguese, Mandarin, and French. We will work to ensure these guidelines are useful to journalists working in a diversity of global settings and contexts. We will also integrate recent research on how preprints and preliminary science are received by the public, in order to recommend approaches for publicly discussing preprint science that are likely to enhance public trust in journalists, scientists, and the scientific process.
2. Blog posts: We will share recommendations and reflections in blog posts and news articles written by the science writers on the team in collaboration with the other team members. These will be published on the ScholCommLab blog, the Imidibaniso and SCI websites, and, possibly, SciDev.net, as well as being promoted through the team’s social media networks. We will also reach out to outlets such as the London School of Economics Impact Blog, Simon Fraser University’s Radical Access Blog, and Library Journal’s INFOdocket to amplify the reach of our blog posts.
3. Podcast series: The team may interview other participants of SCI to gather reflections on recent evolutions in scholarly communication, social and news media, and their influence on trust in science. These interviews will be recorded, edited, and published under a Creative Commons license as a special mini-series of Telling Science Stories, a podcast produced and hosted by Fleerackers.
4. Identify areas for future collaborative research: We hope that participating in SCI will spark conversations between team members, most of whom have not worked together before. These conversations could lead to new collaborations on research at the intersection of journalism and scholarly communication. More broadly, the team could work together to explore potential mechanisms for improving or restoring trust between science, media, and citizens in diverse geographic contexts.

Team

• François van Schalkwyk is a postdoctoral research fellow at the Centre for Research on Evaluation, Science and Technology (CREST), Stellenbosch University, South Africa. His research interests span higher education studies, science communication, scholarly communication, critical data studies, and open science—all of which fall under the umbrella of an interest in sociology of knowledge production. He holds a bachelor’s degree in English Literature (University of Cape Town), master’s degrees in publishing (Stirling University, UK) and education (University of the Western Cape, South Africa), and a PhD in science and technology studies (Stellenbosch University, South Africa). François has published on preprints in the media (Van Schalkwyk & Dudek, 2022, and currently has a paper under consideration regarding the role of regional preprint servers in countering established information asymmetries in science. He has also studied the use of open science by the anti-vaccination movement (Van Schalkwyk, 2019; Van Schalkwyk et al., 2020), and explored issues of trust in science during the COVID-19 pandemic (Weingart et al., 2022). This research—in addition to the fact that he is a founder trustee of the open access scholarly publisher African Minds, associate editor of the scholarly journal Learned Publishing, and an active participant in the ScholarLed initiative—places him in a favorable position to contribute to the broader discussions at Triangle SCI.
• Alice Fleerackers is an award-winning researcher and freelance health and science writer. She is a researcher at the Scholarly Communications Lab, a director on the board of the Science Writers and Communicators of Canada, and a member of the Scientific Committee for the Public Communication of Science and Technology Network. She holds a master’s degree in publishing, a bachelor’s degree in psychology and English literature, and is pursuing an interdisciplinary PhD on science and health journalism at Simon Fraser University, Canada. As part of the team, she will contribute insights from more than three years of research into journalists’ use of preprints. More broadly, she will inform conversations at SCI about (public) trust in science by sharing knowledge of the norms, practices, challenges, and impacts of science journalism drawn from both professional and scholarly experience.
• Luisa Massarani is a science journalist with a PhD in science communication and has published several scientific articles and books in this specialty. She has been SciDev.Net’s regional coordinator for Latin America and the Caribbean since 2003. As such, she is responsible for managing the independent collaborators of SciDev.Net and other key people and organizations in the region, as well as writing articles. She also coordinates meetings, workshops and other training events on science and scientific journalism dissemination. As part of the team, she can provide a more international perspective on journalists’ engagement with preprints, drawn from her work at SciDev.Net as well as her recent research into the challenges journalists in different countries and regions faced in covering preprints responsibly and effectively during the COVID-19 pandemic. Massarani is also the coordinator of the Brazilian Institute of Public Communication of Science and Technology.
• Chelsea Ratcliff is an assistant professor of health communication in the Department of Communication Studies, University of Georgia. She is the lead of the Communicating Uncertain Science to the Public (CUSP) Lab. A former journalist, her scholarly interests and research pertain to news media presentation of biomedical research; public understanding of science, especially scientific uncertainty; effects of science communication; and metascience. As part of the team, she can provide evidence-based insights into how members of the public understand preprints and how media coverage of these unreviewed papers can influence their levels of trust in science and journalism. More broadly, her practical and scholarly knowledge of science journalism, health communication, and public responses to scientific uncertainty can inform conversations at Triangle SCI about how to transparently and ethically broker science knowledge to the public.
• Yao-Hua Law is a freelance science journalist based in Malaysia who covers environment and health. He has written for Science, Science News, The Scientist, BBC Earth, Mosaic, and others. He runs Monsoon, a podcast on science stories in Southeast Asia, and co-founded Macaranga, an environmental journalism portal focused on Malaysia. In addition, since 2021, Yao-Hua has been reporting extensively on forest use in his role as a Pulitzer Center Rainforest Investigations Network Fellow. He’s a winner of a Sigma Data Journalism Award (2023) and One World Media Print Award (2020). He brings his extensive expertise as a journalist, editor, and founder of a media outlet to the project to shed light on the constraints, needs, and norms that shape how, when, and why journalists use (or don’t use) preprints in their coverage.
• Mario Malički is a postdoctoral research fellow at METRICS, Stanford University, where he is focusing on the meta-research of preprints. He is also Editor-in-Chief of Research Integrity and Peer Review journal (RIPR). His research interests include authorship, peer review, duplicate publications, and publication bias. He obtained an MA in Literature and Medicine at King’s College, London, UK, and a PhD in Medical Ethics from the University of Split’s School of Medicine. As part of the team, he can provide essential background context about evolutions in peer review and preprint publishing, with a focus on the challenges and opportunities preprints offer from a scholarly communications perspective. More broadly, his expertise in research integrity and peer review can inform important conversations at Triangle SCI about the evolving publishing landscape and the implications for trust in science.

References

• Castells, M. (2009). Communication Power. Blackwell.
• Fleerackers, A., Moorhead, L.L., Maggio, L.A., Fagan, K., & Alperin, J.P. (2022a). Science in motion: A qualitative analysis of journalists’ use and perception of preprints. PLOS ONE, 17(11), e0277769. doi: 10.1371/journal.pone.0277769
• Fleerackers, A., Riedlinger, M., Moorhead, L., Ahmed, R., & Alperin, J.P. (2022b). Communicating scientific uncertainty in an age of COVID-19: An investigation into the use of preprints by digital media outlets. Health Communication, 37(6), 726–738. doi: 10.1080/10410236.2020.1864892
• Khamsi, R. (2020). What best practices are you following in covering preprints during the pandemic? [Blog post]. Health Journalism. https://healthjournalism.org/core-topic.php?id=10andpage=sharedwisdom
• Malički, M., Jeroncic, A., Ter Riet, G., Bouter, L.M., Ioannidis, J.P., Goodman, S.N., & Aalbersberg, I.J. (2020). Preprint servers’ policies, submission requirements, and transparency in reporting and research integrity recommendations. JAMA, 324(18), 1901–1903
• Massarani, L., & Neves, L.F.F. (2022). Reporting COVID-19 preprints: fast science in newspapers in the United States, the United Kingdom and Brazil. Ciência & Saúde Coletiva, 27: 957–968.
• Massarani, L., Entradas, M., Neves, L.F.F., & Bauer, M.W. (2021). Global Science Journalism Report 2021 (1st ed.). CABI.
• Miller, N. (2021). News media outlets vary widely in how they cover preprint studies. The Journalist’s Resource (26 January). https://journalistsresource.org/health/how-media-cover-preprint-studies
• Nguyen, A., & Tran, M. (2019). Science journalism for development in the Global South: A systematic literature review of issues and challenges. Public Understanding of Science, 28(8), 973–990. DOI: 10.1177/0963662519875447
• Ratcliff, C. L., Fleerackers, A., Wicke, R., Harvill, B., King, A. J., & Jensen, J. D. (2023). Framing COVID-19 preprint research as uncertain: A mixed-method study of public reactions. Health Communication, 1–14. https://doi.org/10.1080/10410236.2023.2164954
• Soderberg, C. K., Errington, T. M., & Nosek, B. A. (2020). Credibility of preprints: an interdisciplinary survey of researchers. Royal Society Open Science, 7(10). https://doi.org/10.1098/rsos.201520
• Tijdink, J., Malički, M., & Bouter, L. (2020). Are preprints a problem? 5 ways to improve the quality and credibility of preprints. Impact of Social Sciences Blog (23 September). https://blogs.lse.ac.uk/impactofsocialsciences/2020/09/23/are-preprints-a-problem-5-ways-to-improve-the-q uality-and-credibility-of-preprints/
• van Schalkwyk, F. (2019). The amplification of uncertainty: The use of science in the social media by the anti-vaccination movement. In Science Communication in South Africa: Reflections on Current Issues (pp. 170–212). African Minds. https://doi.org/10.5281/zenodo.3557217
• van Schalkwyk, F., & Dudek, J. (2022). Reporting preprints in the media during the COVID-19 pandemic. Public Understanding of Science, 31(5), 608–616. https://doi.org/10.1177/09636625221077392
• van Schalkwyk, F., Dudek, J. & Costas, R. (2020). Communities of shared interests and cognitive bridges: the case of the anti-vaccination movement on Twitter. Scientometrics, 125, 1499–1516. https://doi.org/10.1007/s11192-020-03551-0
• Weingart, P., van Schalkwyk, F., & Guenther, L. (2022). Democratic and expert legitimacy: Science, politics and the public during the COVID-19 pandemic. Science and Public Policy, 49(3), 499–517. https://doi.org/10.1093/scipol/scac003

Examples of Guidelines on the Use of Preprints by Journalists

• Haelle T (2020) Tips on covering preprints about coronavirus research. [Blog post]. Association of Health Care Journalists. https://healthjournalism.org/resources-tips-details.php?id=1113#.Xu_FTy2ZPxU.
• Hamilton E (2020) How should journalists cover coronavirus preprint studies? University of Wisconsin-Madison [website]. https://news.wisc.edu/how-should-journalists-cover-coronavirus-preprint-studies/.
• Hanage B and Lipsitch M (2020, February 23) How to report on the Covid-19 outbreak responsibly. Scientific American. https://blogs.scientificamerican.com/observations/how-to-report-on-the-covid-19-outbreak-responsibly/.
• Khamsi R (2020) What best practices are you following in covering preprints during the pandemic? [Blog post]. Health Journalism. https://healthjournalism.org/core-topic.php?id=10andpage=sharedwisdom.
• Ordway D-M (2020a, April 2) Covering biomedical research preprints amid the coronavirus: 6 things to know. Journalist’s Resource. https://journalistsresource.org/tip-sheets/research/medical-research-preprints-coronavirus/.
• Ordway D-M (2020b, March 6) Covering Covid-19 and the coronavirus: 5 tips from a Harvard epidemiology professor. Journalist’s Resource. https://journalistsresource.org/studies/society/public-health/covid-19-coronavirus-epidemiology/.

[ Photos by Bank Phrom and Ousa Chea used under Unsplash free license ]