#peer-review — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #peer-review, aggregated by home.social.
-
From this perspective on AI and "peer" review, I'm surprised to actually not see any increase in the rate of growth of articles with LLMs?
https://www.science.org/content/article/can-ai-help-solve-peer-review-crisis-here-are-its-promises-and-pitfalls #AcademicPublishing #PeerReview #AI -
Peer review is "the bedrock of science" – and "it's not standing strong" right now. 🔬🧪
Saima Sidik for Ars Technika.
👉 https://arstechnica.com/science/2026/08/peer-review-is-overwhelmed-can-it-survive-in-the-ai-era/
-
📣 Join our editorial team and support scientific open source!
We have new scientific Python packages waiting on peer review, and need editors with domain expertise to help guide them through.
Looking for backgrounds in:
⚛️ Physics
🧪 Chemistry
😷 Public healthEditors recruit reviewers, facilitate conversation, and help authors cross the finish line. Flexible commitment, full onboarding + support provided.
Apply: https://bit.ly/pyoseditor
-
With systematic institutional de-funding and ongoing stigmatisation comes the erosion of academic community. This is the case for US #anthropology here, but
the same can be observed for other targeted disciplines, particularly in the humanities. #peerreview without peers results in no review.
RE: https://bsky.app/profile/did:plc:45agxc42amzvl3vpnr2tl5xj/post/3mucvlmyvns2x -
New study: At Nature Communications (2016-2022), papers approved by #OpenPeerReview had higher #altmetrics scores than papers approved by conventional #PeerReview.
www.researchsquare.com/article/rs-9...
#Impact #ScholComm
Does Open Peer Review Correlat... -
New study: At Nature Communications (2016-2022), papers approved by #OpenPeerReview had higher #altmetrics scores than papers approved by conventional #PeerReview.
https://www.researchsquare.com/article/rs-9962990/v1 -
added to my #toReadList: "On the defence of responsible human judgement in #peerreview by Soares & Adams, link: https://onlinelibrary.wiley.com/doi/10.1002/cjce.70497
-
If you don’t have access, you can read the article in your browser via this link: https://rdcu.be/fCrll.
And if you’d like a PDF version, head over to RG: https://www.researchgate.net/publication/413676790_Poor_bibliometric_studies_do_not_make_bibliometrics_poor_science_a_response_to_Ellegaard
#Bibliometrics #Scientometrics #ResearchQuality #MetaScience #BibliometricAnalysis #PeerReview
-
From Volunteerism to Duty: Reforming Peer Review with Tokens | Communications of the ACM
https://dl.acm.org/doi/full/10.1145/3770921?download=trueAn interesting idea on how to incentivise peer-review by conditioning the ability to publish on it.
-
🔖 Read the latest issue from our 2026 PREreview Champions interview series!
This time Mabel Omoniwa underscores the impact of moving from theory to practice in Open Science advocacy, and shares her experience turning her learnings from the Champions program into engaging opportunities for the African research community.
👉 https://content.prereview.org/interview-prereview-champion-mabel-omoniwa/
-
Tracking, Valuing and Evaluating the Diversity of HSS Research Outputs
This was originally a script for a video I recorded for a meeting of the Australian Deans and Directors of Creative Arts (DDCA) group. It is addressed to those who are in institutional positions who need to make judgements about the qualities of research outputs in Humanities and Social Sciences with a particular focus on outputs that are “non-traditional” (actually often very “traditional” in their field!) or otherwise “not easily measurable by a STEM audience”.
In my view there are two key roles for those specifically engaged in supporting those doing HSS research. The first is to advocate for financial systems that can support small and numerous research projects and the funders who want to support them. The second key role is to defend the diversity of research practices, outputs and impacts in HSS and to create a meaningful narrative around their qualities for institutional audiences.
I’m going to focus on the second here, but I wanted to link both to underline the point that what is of value in HSS research is precisely its diversity, and its difference from the homogeneity and scale of conventional STEM research and its outputs. There is not infrequently a greater diversity of research outputs from single HSS groups, centres and departments than there is from entire faculties of science, engineering and medicine.
That diversity is at the core of the value that HSS research and scholarship brings to the table.
Every social epistemology of knowledge from Latour to Longino via Merton, Ravetz, Kuhn and Fleck have at their heart the idea that the core is the negotiation of knowledge across boundaries, or between groups. This places diversity, of all kinds, at the heart of this process, and linked to it the institutional systems that keep it alive. Arguably it is the purpose of a university to support and bring into contact diverse approaches, experiences and ways of knowing.
The issue of course is that managing that diversity is a challenge. At the scale of a university, systems are a necessity. Consistency is needed and economies of scale are real. Particularly in HSS but across other disciplines as well, it is impossible for any leader to be across the detail of how good every specific body of work is. If you believe in the argument for diversity then this as it should be – anything else would be a sign of insufficient diversity. That doesn’t mean that leaders can’t be effective at telling the stories about why diverse research matters. Nor does it mean that they can’t make judgement calls on the distribution of resources. But it means those decisions are subjective and contextual. And that the evidence that supports them will be specific to each case.
But that doesn’t solve the problem either. So what are the pragmatic and principled approaches available to addressing this complexity? In my view the key lies in separating out the challenge into two parts, one relatively straightforward and immediately useful, the other more complex but I would argue ultimately tractable.
An important, but oddly overlooked distinction in research evaluation and particularly technology discussions is the distinction between tracking outputs and evaluating them. There is often a tendency to lump these together and to imagine for example, that if scholarly books were better tracked then that would necessarily mean there would be improved citation data for them. The homogeneity of STEM research outputs means that issues with data coverage and completeness are often conflated to the question “is it indexed” using the presence of a specific journal in a specific database both as a reliable marker of quality itself but also of confidence in the quality and relevance of generally accepted performance data.
So, although this is an oversimplification, in STEM the tracking of outputs is standardised and homogenous. You look it up in an index like Web of Science.
In HSS, and particularly for creative works, the situation is far more complex. Institutions generally have very poor data on the scope and volume of creative outputs, or other forms of (so-called) “non-traditional” outputs. Work done by Niamh Quigley, within the COKI team (Quigley, 2022) showed generally used recording systems (systems like Symplectic, Pure and the like) tended to be poor at capturing aspects of research outputs important to creative researchers. Further, there was a tendency for the information that was manually inputted by researchers (that they care enough to type in!) to be stripped away as it made its way through institutional systems to the data warehouses used for high level analysis.
From a systems perspective what is needed is systems that make it easy for creative practice researchers to register outputs, to flexibly record the evidence that they consider important about the qualities of these works, and to also make it easy for institutions in turn to gather that information up. Recent technical developments provide improved paths to making this happen.
The Open Research Contributor and ID (ORCID) system provides a mechanism for researchers to authoritatively identify themselves, and to record outputs they have contributed to. In 2025 ORCID have increased the number of work types (Petro, 2025) to include a range relevant in humanities (including images, video, designs, sound and other creative works) as well as improving the mapping of some other relevant work types including reports. A researcher can choose what works they make visible, and thereby designate as “research”. This is an important distinction for some creative workers. Institutions can then harvest those outputs without requiring further work from researchers.
ORCID doesn’t solve all the problems though. Badly implemented as an institutional system it is yet another place creative researchers have to battle through manually adding their work while colleagues in STEM subjects have it all done for them by journal publishers. Critical to making this work is a commitment from you and your teams that ORCID be the only place researchers have to make sure their work is available and pulling that information from ORCID is an institutional responsibility.
But that still leaves the manual input issue. There isn’t – currently – a complete solution for that (although one – or several – could be built) but there is at least a solution that offers some additional value for creative researchers. Zenodo is a large scale repository, managed by CERN (yes the particle accelerator in Switzerland CERN) that offers a place for anyone to deposit research outputs. It provides a safe, publicly accessible platform from which researchers can showcase their work. It even allows for dark deposit for those creative researchers who need to restrict access to the core of their work for artistic, ethical or commercial reasons.
Most importantly Zenodo provides DataCite DOIs and this means that – while you do have to manually put information into Zenodo – that information can automatically flow through to ORCID. One place to input the information and then it can flow to the other places it needs to be.
This flow of information should be the case for any repository that integrates with DataCite (or Crossref) to provide DOIs. Zenodo is large scale and flexible, but there can be an argument for specialist repositories. I am generally sceptical of the value of creating new local repositories, whether at institutions or nationally, but the key questions to ask of any repository are “does it integrate with DOI providers” and “does it provide the recording functionality creative research practitioners need”? That flexibility is key and Zenodo has some strengths here that can also help us to address, if not yet fully solve, the second problem.
Creative practice researchers know the reasons that their work is important, and can help provide evidence of that. But that evidence is highly diverse.
I know the pressures that institutional leaders face to articulate a comparative quantitative value of research outputs to the rest of the university. And I would argue that we nonetheless need to collectively hold the line that this makes no sense. If you value a diversity of research and researchers, you can’t protect that diversity by homogenising the way that value is measured.
But that doesn’t mean we can’t articulate value. It doesn’t mean we can’t compare how disparate research projects and outputs achieve specific goals. It just means, in the best tradition of HSS scholarship, that the evidence that we use is contextual. That it is diverse.
And here is how repositories like Zenodo can help. Where they support the creation of packages of work, researchers can bundle up the work itself with evidence of its value and impact. This will start unstructured, but over time, and in collaboration with research communities it will be possible to develop some standards. For specific cases. Where appropriate. This might mean giving some background on the gallery that invited an exhibition, the footfall or an audience survey. It might be details of the film festival that selected a documentary for screening, or audience numbers when it was syndicated or broadcast. It might be structured in a particular way to make it easier to process (or even generate). But the key is to build on systems that always allow for the flexibility of a narrative statement that allows for the flexibility that research statements already provide.
This workflow places researchers in control, gives them flexibility they need to provide access and to evidence the value of their work. It provides a single point of entry from which communities, institutions, and others can draw in the data to help track those outputs and collate the evidence of their value. It is not controlled by corporate interests but by academic communities. It is not subject to the funding whims of one national government but an international set of consortia.
Perhaps best of all, this is an area where the humanities and social sciences can lead. It solves a problem by putting researchers in control while supporting institutional needs. It helps and simplifies monitoring while offering a path to gathering more sophisticated evidence to tell better stories. These are not problems that are restricted to HSS, but you could argue that much of STEM evaluation has lost its way, focusing on numbers instead of substance, journal lists instead of journal content, impact factors instead of actual impact. This recontextualisation, and re-situation is where humanities and social sciences scholars, and leadership can show a way to better forms of evaluation.
References
- Petro, J. 2025. ORCID and Humanities: Celebrating the New Year with New Work Types. ORCID Blog. ORCID. https://info.orcid.org/new-work-types/.
- Quigley, N. 2022. Increasing the visibility of grey literature: Project report (Version 1). Zenodo. https://doi.org/10.5281/zenodo.6501506.
-
How to peer review a paper in 2026
https://jilltxt.net/how-to-peer-review-a-paper-in-2026/ -
Can #GenAI help scrutinise the scientific literature, not just create it?
:doi: https://doi.org/10.1007/s10805-026-09761-0
In our new paper, we tested #ChatGPT and #DeepSeek as triage tools for post-publication peer review. The result: useful potential, but human verification remains essential.
#ResearchIntegrity #PeerReview #LLMs #ScholarlyCommunication
-
🤦♂️ " #NIH Wants to Remove Peer Review Scores From #Grant Applications "
-
NIH is proposing a change on how they report back review outcomes on grant applications. Rather than providing a specific impact score, applications would be put into one of three categories, "most competitive," "competitive," and "not discussed." https://grants.nih.gov/grants/guide/notice-files/NOT-OD-26-088.html
cc @petersuber
-
Ars Technica: Peer review is overwhelmed—can it survive in the AI era?. “The number of papers indexed in the databases Scopus and Web of Science, for example, has recently been increasing exponentially, at a rate of 5.6 percent per year. By one estimate, researchers around the globe are devoting a collective 15,000 years of work to peer review every year—work that, if paid, would cost $1.5 […]
https://rbfirehose.com/2026/08/14/ars-technica-peer-review-is-overwhelmed-can-it-survive-in-the-ai-era/ -
An AI system has written scientific papers that passed peer review, out-performing more than half of human submissions. This step raises questions about how AI will reshape scientific research, peer review, and the evaluation of scientific contributions.
Learn more about how AI affects the industry and #PeerReview: https://cacm.acm.org/news/ai-prepped-paper-passed-peer-review-now-what/
-
“Research Gold, a site that advertises services for medical researchers, including drafting peer-review ready manuscripts, systemic reviews and meta-analyses, claims that it’s “100% human-written, never AI,” and lists a number of PhD reviewers and professional methodologists on staff that carry out this meticulous, difficult work.
The problem: The PhD reviewers Research Gold lists on its site are AI-generated and don’t exist. Other methodologists it lists are real, but are not aware their identity is being used by Research Gold. When I tried calling the company, an AI agent that refused to concede it was AI answered and kept trying to sell me Research Gold’s services. Email and chat communication with the company were also AI generated.”
https://www.404media.co/company-offering-100-human-written-never-ai-peer-review-is-entirely-ai/
#AI #GenerativeAI #AcademicPublishing #ResearchGold #PeerReview
-
#PeerReview is overwhelmed—can it survive in the #AI era?
Researchers need peer-reviewed publications to secure jobs and grants, so they continue to slog through the system even though it’s becoming “extremely protracted and painful”
The number of papers in databases Scopus and Web of Science has recently been increasing exponentially, at a rate of 5.6% per year. By one estimate, researchers around globe are devoting a collective 15,000yr of work to peer review every year
https://arstechnica.com/science/2026/08/peer-review-is-overwhelmed-can-it-survive-in-the-ai-era/ -
Company Offering ‘100% Human-Written, Never AI’ Medical Research Is Entirely AI
-
"Peer review is overwhelmed—can it survive in the AI era?
As research and AI-assisted papers surge, volunteer reviewers struggle to keep up."#Science #Engineering #Peer #PeerReview #Publications ...
https://arstechnica.com/science/2026/08/peer-review-is-overwhelmed-can-it-survive-in-the-ai-era/
-
One of the current #TDCC projects we were able to help make possible is #Metacheck. Blanket statement from me: If you are a #researcher, go check it out!
-
#AI #PeerReview #science #blogs
"Peer review is overwhelmed—can it survive in the AI era?
As research and AI-assisted papers surge, volunteer reviewers struggle to keep up.
The number of papers indexed in the databases Scopus and Web of Science, for example, has recently been increasing exponentially, at a rate of 5.6 percent per year. By one estimate, researchers around the globe are devoting a collective 15,000 years of work to peer review every year—work that, if paid, would cost $1.5 billion for the share done in the US alone. Increasingly, researchers are finding it hard to donate that kind of time.
'I struggle like anything to get peer reviewers, said Haseeb Irfanullah, who is on the editorial board of Wiley’s Learned Publishing but whose comments represent his own views as an independent consultant. Steven Mack, an editor for Human Immunology, has recently had to email around thirty researchers to find just one who would review a paper. Five years ago, he estimates it would have taken only five to 10 emails to find three willing reviewers. In this landscape, it’s not surprising that editors are assigning reviews to people who aren’t qualified or don’t have the time to concentrate on them.
In most fields, researchers need peer-reviewed publications to secure jobs and grants, so they continue to slog through the system even though it’s becoming 'extremely protracted and painful,' in the words of microbiologist Sebastian Lourido from the Whitehead Institute.
Increasingly, many are advocating for change. Meanwhile, a few bold researchers have decided that the traditional publishing system just isn’t worth it anymore—AI research in particular has seen a shift toward communicating impactful findings through very informal means, like blogs.
Peer review is 'the bedrock of science,' Semprini said. But at this moment, 'it’s not standing strong.'"
https://arstechnica.com/science/2026/08/peer-review-is-overwhelmed-can-it-survive-in-the-ai-era/
-
When research is ready, long waits for feedback slow progress. At #JPsyExp, we aim for initial screening to take one week and double-blind peer review to take two to four weeks.
-
Interview with yours truly on journal rank, peer review and a journal replacement:
"Beyond Journal Prestige: Rebuilding Peer Review and Scholarly Publishing"
https://www.youtube.com/watch?v=miWuXGbJJl8
#academicchatter #publishing #journals #impactfactor #science #peerreview
-
🚨 #AI BREAKING NEWS! 🚨 In an astonishing turn of events, the digital gatekeepers at #OpenReview have decided that even computers must verify their existence before accessing the sacred halls of Open Peer Review. 🤖🔒 Because, clearly, nothing screams "cutting-edge research" like #CAPTCHA tests for #LLMs. 😂🔍
https://openreview.net/challenge?redirect=%2Fforum%3Fid%3DklU4737opt #News #PeerReview #HackerNews #ngated