#replicability — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #replicability, aggregated by home.social.
-
Too much money? Too much pressure? Only about 12% #replicability:
https://elifesciences.org/articles/67995
and, on top of that, there now appears to be evidence of citation distortion in certain cancer journals:
"Here, we show that molecular cancer articles sharing features with retracted papers from paper mills display citation patterns that suggest systematic inflation. These articles were published in journals in the top decile of journal rankings."
https://www.biorxiv.org/content/10.64898/2026.05.25.727627v1 -
Too much money? Too much pressure? Only about 12% #replicability:
https://elifesciences.org/articles/67995
and, on top of that, there now appears to be evidence of citation distortion in certain cancer journals:
"Here, we show that molecular cancer articles sharing features with retracted papers from paper mills display citation patterns that suggest systematic inflation. These articles were published in journals in the top decile of journal rankings."
https://www.biorxiv.org/content/10.64898/2026.05.25.727627v1 -
Thanks to Gary King for pushing students and colleagues alike to take the #replicability issue seriously from the 1990s
-
Thanks to Gary King for pushing students and colleagues alike to take the #replicability issue seriously from the 1990s
-
From 20-22 July, the 4th #ACM #Conference on Reproducibility and Replicability will take place in #Delft.
The conference welcomes contributions on methods, tools, case studies, and community efforts around #reproducibility and #replicability in #computational research.Deadlines: 10 March (abstracts)/17 March (papers)
https://acm-rep.github.io/2026/cfp/
Foto by ThisIsEngineering on unsplash
-
From 20-22 July, the 4th #ACM #Conference on Reproducibility and Replicability will take place in #Delft.
The conference welcomes contributions on methods, tools, case studies, and community efforts around #reproducibility and #replicability in #computational research.Deadlines: 10 March (abstracts)/17 March (papers)
https://acm-rep.github.io/2026/cfp/
Foto by ThisIsEngineering on unsplash
-
The final paper in our upcoming special issue on #Replicability and #Reproducibility, Joseph Holler and colleagues use open science practices to develop a GIScience study on access t oCOVID-19 healthcare in Illinois, US https://doi.org/10.1080/15230406.2025.2591124 #OpenAccess #GISchat
-
The final paper in our upcoming special issue on #Replicability and #Reproducibility, Joseph Holler and colleagues use open science practices to develop a GIScience study on access t oCOVID-19 healthcare in Illinois, US https://doi.org/10.1080/15230406.2025.2591124 #OpenAccess #GISchat
-
Reading a paper this morning https://onlinelibrary.wiley.com/doi/10.1002/pds.4295 that follows Nosek & Errington (2017) https://elifesciences.org/articles/23383.pdf in muddling the distinction between #replicability as a broader scientific aim vs #reproducibility as captured in the narrow (purely 'computational') idea of #reproducibleresearch discussed e.g. in a #Guix context here https://guix.gnu.org/cookbook/en/html_node/Reproducible-Research.html.
This mistake is not uncommon, and leads (in this present paper, at least) to insufficient stress on computational reproducibility as a sine qua non of any higher-order quality attribute.
-
Reading a paper this morning https://onlinelibrary.wiley.com/doi/10.1002/pds.4295 that follows Nosek & Errington (2017) https://elifesciences.org/articles/23383.pdf in muddling the distinction between #replicability as a broader scientific aim vs #reproducibility as captured in the narrow (purely 'computational') idea of #reproducibleresearch discussed e.g. in a #Guix context here https://guix.gnu.org/cookbook/en/html_node/Reproducible-Research.html.
This mistake is not uncommon, and leads (in this present paper, at least) to insufficient stress on computational reproducibility as a sine qua non of any higher-order quality attribute.
-
An #openaccess toolbox aimed at unifying the field of auditory reverse correlation studies by providing a consistent experimental and analysis framework allowing for better #replicability and #reproducibility
https://fediscience.org/@LeoVarnet/115627592918266083
#revcorr #revcor #psychphysics #psychology #psychoacoustics #OpenScience @psycholinguistics @psychology -
An #openaccess toolbox aimed at unifying the field of auditory reverse correlation studies by providing a consistent experimental and analysis framework allowing for better #replicability and #reproducibility
https://fediscience.org/@LeoVarnet/115627592918266083
#revcorr #revcor #psychphysics #psychology #psychoacoustics #OpenScience @psycholinguistics @psychology -
A new article by Chloe Patton shows how debates about #OpenScience often slip into absurdity – like demanding #replication from the #Humanities. You can’t replicate history, culture, or interpretation the way you replicate a physics experiment. It’s a different kind of knowledge.
:doi: https://doi.org/10.1093/reseval/rvaf052
Forcing STEM-style standards onto the humanities doesn’t improve #science – it just adds bureaucracy and limits academic freedom.
-
A new article by Chloe Patton shows how debates about #OpenScience often slip into absurdity – like demanding #replication from the #Humanities. You can’t replicate history, culture, or interpretation the way you replicate a physics experiment. It’s a different kind of knowledge.
:doi: https://doi.org/10.1093/reseval/rvaf052
Forcing STEM-style standards onto the humanities doesn’t improve #science – it just adds bureaucracy and limits academic freedom.
-
Have you checked out our #OpenScience Bites #podcast yet? Stay tuned!
Very soon, we’ll be talking with Felipe Romero, Assistant Professor at the Faculty of #Philosophy. From a #MetaScience perspective, we’ll dive into the concepts of #reproducibility and #replicability, and why they matter for #research practice.
We’ll let you know as soon as the new episode is online!
In the meantime, feel free to listen to our earlier episodes:
-
Have you checked out our #OpenScience Bites #podcast yet? Stay tuned!
Very soon, we’ll be talking with Felipe Romero, Assistant Professor at the Faculty of #Philosophy. From a #MetaScience perspective, we’ll dive into the concepts of #reproducibility and #replicability, and why they matter for #research practice.
We’ll let you know as soon as the new episode is online!
In the meantime, feel free to listen to our earlier episodes:
-
How Scrutiny and Iteration Made Behavioral Economics Better https://www.chicagobooth.edu/review/how-scrutiny-iteration-made-behavioral-economics-better
"However, direct #replicability should not be confused with #generalizability or universality. While the basic patterns of behavior documented in the original experiments reliably show up in subsequent replications, their magnitudes can vary substantially across contexts. Loss aversion might be stronger in some settings than in others. Understanding this variation has become a central focus of contemporary behavioral economics research. Rather than viewing such #heterogeneity as a challenge to the field’s foundations, researchers increasingly see it as a source of insight into the underlying psychological mechanisms driving behavioral anomalies."
#BehavioralEconomics -
How Scrutiny and Iteration Made Behavioral Economics Better https://www.chicagobooth.edu/review/how-scrutiny-iteration-made-behavioral-economics-better
"However, direct #replicability should not be confused with #generalizability or universality. While the basic patterns of behavior documented in the original experiments reliably show up in subsequent replications, their magnitudes can vary substantially across contexts. Loss aversion might be stronger in some settings than in others. Understanding this variation has become a central focus of contemporary behavioral economics research. Rather than viewing such #heterogeneity as a challenge to the field’s foundations, researchers increasingly see it as a source of insight into the underlying psychological mechanisms driving behavioral anomalies."
#BehavioralEconomics -
Even discounting bad faith and charlatans, in the biological sciences (which I am familiar with) and probably other research, we get results... but even with meticulous lab books, it's hard to reproduce the exact same experiment and replicate the exact same results. Because we really don't know all the variables
#Science #Experiments #ReplicabilityUnreplicable results are an open secret | Dynamic Ecology
https://dynamicecology.wordpress.com/2025/07/17/unreplicable-results-are-an-open-secret/
-
Even discounting bad faith and charlatans, in the biological sciences (which I am familiar with) and probably other research, we get results... but even with meticulous lab books, it's hard to reproduce the exact same experiment and replicate the exact same results. Because we really don't know all the variables
#Science #Experiments #ReplicabilityUnreplicable results are an open secret | Dynamic Ecology
https://dynamicecology.wordpress.com/2025/07/17/unreplicable-results-are-an-open-secret/
-
New I4R discussion paper by Philipp Knöpfle, @drfollowmario, and me: "A Replication Report on "Political polarization of news media and
influencers on Twitter in the 2016 and 2020 US presidential elections" by Flamino et al. 2023
https://www.econstor.eu/handle/10419/319836
#replicability #computationalsocialscience -
New I4R discussion paper by Philipp Knöpfle, @drfollowmario, and me: "A Replication Report on "Political polarization of news media and
influencers on Twitter in the 2016 and 2020 US presidential elections" by Flamino et al. 2023
https://www.econstor.eu/handle/10419/319836
#replicability #computationalsocialscience -
How to make Rscripts more reproducible with Nix package Manager : Rix for R make it possible to go a step further than using only Renv or even Docker ; see documentation here https://docs.ropensci.org/rix/index.html#quick-start-for-returning-users #nix #R #replicability
-
Si vous êtes chercheur.se à l'Université de Rennes / Rennes 2 ou dans des écoles de l'EPE, l'atelier ARDoISE peut vous accompagner lors d'une éventuelle candidature au prix SO du logiciel libre de recherche #FAIR #replicability #SoftwareHeritage #HAL https://scienceouverte.univ-rennes.fr/ardoise-atelier-rennais-de-la-donnee https://www.ouvrirlascience.fr/les-candidatures-pour-le-prix-science-ouverte-du-logiciel-libre-de-recherche-2025-sont-ouvertes/
-
How to describe #steganography methods in a comparable and unified way to aid #replicability?
We combined pre-existing methodology into a single framework. New pre-print + online tool prototype (will get improved soon) on our website: https://patterns.omi.uni-ulm.de/news/
Full version of the paper and the online tool will be presented at the ARES'25 CUING workshop in August.
#replicability #steganography #covertchannels #informationhiding #infosec #cybersecurity #security #research
-
Had a great time at the CSS workshop in Mannheim organized by Ruben Bach & Valerie Hase. You can find the slides from my presentation on "Reproducibility & Replicability in Computational Social Science Research" here: https://doi.org/10.6084/m9.figshare.29086382.v1
#reproducibility #replicability #computationalsocialscience -
Had a great time at the CSS workshop in Mannheim organized by Ruben Bach & Valerie Hase. You can find the slides from my presentation on "Reproducibility & Replicability in Computational Social Science Research" here: https://doi.org/10.6084/m9.figshare.29086382.v1
#reproducibility #replicability #computationalsocialscience -
New publication: Code-sharing policies are associated with increased #reproducibility potential of ecological findings, by @antica_c and others. #codesharing #replicability #reliability
https://doi.org/10.24072/pcjournal.541 -
"Estimating the #replicability of Brazilian #biomedical #science"
... Replication rates for these experiments varied between 15 and 45% according to five predefined criteria ...
-
"Estimating the #replicability of Brazilian #biomedical #science"
... Replication rates for these experiments varied between 15 and 45% according to five predefined criteria ...
-
🎉 Our method Anisotropic Specular Image-Based Lighting Based on BRDF Major Axis Sampling has received the Replicability Stamp!
🔍 Check it out here: https://www.replicabilitystamp.org/index.html#https-github-com-iota97-anisotropyeditor
🧑💻 Code & data: https://github.com/iota97/AnisotropyEditor#ComputerGraphics #Rendering #Replicability #OpenScience #BRDF
-
The University for #HumanisticStudies/ UvH joins the NLRN as an institutional member!
At the NLRN, we want to make a range of epistemic voices heard in the discussion around #reproducibility and #replicability of #science and #scholarship. With their research portofolio focusing on #worldviews and humane societies, the UvH complements the voices of the other, more technical and generalistic institutions TU Delft, TU/e, RUG and escience Center.
Foto: Gaelle Marcel for Unsplash
-
Les interventions de la journée d'étude ARDoISE du 17 décembre sont en ligne et accessibles depuis le site ScienceConf : https://je2024-ardoise.sciencesconf.org/resource/page/id/1 #reproductibilté #replicability #ResearchSoftware Interventions de @zimoun @rougier @NaudetFlorian , Isabelle Blanc administratrice ministérielle des données, code source et logiciels, et d'autres encore...
-
Comparing the predicted #replicability of a study with its actual replicability.
https://www.nature.com/articles/s41562-024-01961-1"Both beginners and more-experienced participants…have some ability to make better-than-chance predictions about the reliability of ‘fast science’ under conditions of high uncertainty."
For more along these lines, esp #PredictionMarkets for #replication, see the Twitter thread I started in 2020, which I plan to continue here in Mastodon. Watch this space.
https://x.com/petersuber/status/1259521012196167681 -
Comparing the predicted #replicability of a study with its actual replicability.
https://www.nature.com/articles/s41562-024-01961-1"Both beginners and more-experienced participants…have some ability to make better-than-chance predictions about the reliability of ‘fast science’ under conditions of high uncertainty."
For more along these lines, esp #PredictionMarkets for #replication, see the Twitter thread I started in 2020, which I plan to continue here in Mastodon. Watch this space.
https://x.com/petersuber/status/1259521012196167681 -
'Estimating the Replication Probability of Significant Classification Benchmark Experiments', by Daniel Berrar.
http://jmlr.org/papers/v25/24-0158.html
#classifiers #replicability #hypothesis -
'Estimating the Replication Probability of Significant Classification Benchmark Experiments', by Daniel Berrar.
http://jmlr.org/papers/v25/24-0158.html
#classifiers #replicability #hypothesis -
Très content d'avoir pu écouter - et rencontrer - @arthurperret parler d'organisation des connaissances, prise de notes et graphs de documents au @CampusDuLibre 2024 aujourd'hui !
#PKM #Zettlr #markdown #dataviz #visualthinking #openscience #replicability #campusdulibre -
Do professors with less PC views self-censor?
Clark et al. reported only the linear relationship: the less PC their view, the more reluctant professors were to share it. Kudos to Clark et al. for publishing their data so Luke could detect a better-fitting non-linear, non-unified explanation: most professors were not self-censoring; they were either uncertain or else unreluctant to share.
https://doi.org/10.31234/osf.io/ab34v
#edu #higherEd #psychMethods #logic #replicability #manyAnalysts #metaScience
-
Do professors with less PC views self-censor?
Clark et al. reported only the linear relationship: the less PC their view, the more reluctant professors were to share it. Kudos to Clark et al. for publishing their data so Luke could detect a better-fitting non-linear, non-unified explanation: most professors were not self-censoring; they were either uncertain or else unreluctant to share.
https://doi.org/10.31234/osf.io/ab34v
#edu #higherEd #psychMethods #logic #replicability #manyAnalysts #metaScience
-
The state of scientific #replicability in AI for large models is quite depressing. It seems to be currently well-accepted that, given that very few actors in the world can afford to replicate model training (which is certainly true), then it is OK to not release training code for your experiments to reviewers and the broader scientific community. I vehemently disagree and, frankly, consider this a mire in how much we can trust associated "scientific" results.
-
The state of scientific #replicability in AI for large models is quite depressing. It seems to be currently well-accepted that, given that very few actors in the world can afford to replicate model training (which is certainly true), then it is OK to not release training code for your experiments to reviewers and the broader scientific community. I vehemently disagree and, frankly, consider this a mire in how much we can trust associated "scientific" results.
-
Research integrity irony:
"A high-profile paper about ways to improve the rigor of research papers has been retracted after critics attacked its own rigor." 🤦
#PreRegistration #ResearchIntegrity #PublicationEthics #Retractions #Reproducibility #Replicability #RIirony
-
Interesting presentation on #Reproducible #DataScience: https://www.youtube.com/watch?v=wK3vPbcSoFE
One point that was made at the beginning is about that "reproducibility" might not be well defined. I think that it will be good to read the 2019 NASEM report: "Reproducibility and Replicability in #Science" https://nap.nationalacademies.org/catalog/25303/reproducibility-and-replicability-in-science -- Reproducibiliy and #Replicability are clearly defined and delimited.
-
Will a religious #mindfulness practice improve #wellbeing?
Among more than 700 self-identifying #Christians, "we found no differences on well-being between a religiously framed #meditation practice (experimental condition), a nonreligious framed meditation practice (active control), or a condition that asked participants to notice their daily life (passive control)."
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11218060/
#religion #cogSci #psychology #psychiatry #replicability #philosophyOfReligion #xPhi
-
I’m part of the #EEGManyLabs project testing the #replicability of influential #EEG studies. We are using #PredictionMarkets as a tool in this effort and you are invited to take part, especially if you have some expertise in EEG research, no matter how little. See below for details.
You may well know about the success of “prediction markets” in forecasting the likelihood of replication (e.g., Dreber et al., PNAS 2015). We are delighted to announce that we have partnered with economists who led these seminal studies to test the wisdom of the EEG community.
From today (as we near the end of recruitment for this project - please see last calls below), we are opening a survey to ask you to vote on the likelihood of some hypotheses studied in the #EEGManyLabs project. Subsequently, you will be invited to bet on the likelihood of success through a stock market platform, where you will earn real money for you or a selected charity.
The success of this effort will become clear when we complete the full project in a few years time. But the results will immediately tell us about the degree of optimism/pessimism amongst our community.
So, please share this widely and place your bets now...
How can I sign up for the prediction markets? Registrations to participate in the prediction markets are administered via the sign-up form linked below. You must have experience of working with EEG (for example, through collecting and/or analysing EEG data, which may be evidenced by having published peer-reviewed articles or preprints with EEG or equivalent experience e.g. designing, collecting and analysing data from EEG experiments).
#neuroscience #psychology #replication #replicationcrisis #reproducibility #metascience
-
Short reminder for our Meta-Rep conference on #metaScience, #replicability, robustness issues, ... in the beh/soc/cog sciences.
Attend/present your work at META-REP 2024 in Munich (Oct 28-31): https://www.conference2024.meta-rep.uni-muenchen.de/index.html
Submission deadline is April 30 (there will be no extension)
Our two keynote speakers will be Fiona Fidler and Daniël Lakens @lakens.
-
*CALL FOR SUBMISSIONS*
Interested in #metaScience, #replicability, robustness issues, ... in the beh/soc/cog sciences? Then attend/present your work at META-REP 2024 in Munich (Oct 28-31): https://www.conference2024.meta-rep.uni-muenchen.de/index.html
Submission deadline April 30.
-
Last week I attended the 6th Perspectives on Scientific Error Conference at @TUEindhoven
I learned so much! About #metascience #preregistration #replicability #qrp questionable research practices, methods to detect data fabrication, #peerreview, #poweranalysis artefacts in #ML machine learning...
I'm impressed by the commitment of participants to improve science through error detection & prevention. Thanks to the organizers Noah van Dongen, @lakens @annescheel Felipe Romero and @annaveer -
A debiasing intervention that works across cultures?
Explaining solutions to base rate problems has helped people in Western, Educated, Industrialized, Rich, and Democratic (WEIRD) cultures solve the problems.
It also helped a less WEIRD sample (N = 84).
The effect was similar between urban and village residents but seemed to benefit from prior education.
https://doi.org/10.1016/j.cognition.2023.105681
#statistics #psychology #BehavioralEconomics #decisionScience #crossCulturalPsychology #replicability #edu #math