#nullresults — Public Fediverse posts

It must be very hard to publish null results https://d.repec.org/n?u=RePEc:zbw:i4rdps:281&r=&r=cmp
"Most of the statistical tests … are severely Underpowered … implies that the majority of null hypothesis significance tests we run should yield #nullResults. Yet, the published literature tells a very different story.
… central mechanism behind this gap is selection on #significance, a filter that makes statistically significant estimates more likely to appear in print than null results.
… published estimates are afflicted by a “winner’s curse” that biases them away from zero, researchers commit more Type I error, metaanalyses become unreliable, and published studies are less likely to be replicable. When null results remain hidden in the file drawer, researchers may waste time and money studying the same ineffective interventions over and over
… share of pure null results is extremely small: fewer than 2% of abstracts report only null findings. By contrast, over 90% of articles that rely on statistical methods prominently claim to reject at least one null hypothesis.
… estimates that are statistically distinguishable from zero must be at least one order of magnitude more likely to appear in print than null results. Arguably more realistic assumptions can easily produce magnitudes of selection on significance around 100x"
#science #statistics #bias

Great job by Lucia Coll, a student working in my group. Lucia's #scicomm video about why #nullresults are interesting made it to the jury mentions of the national science communication competition #fastforwardscience in Germany. Check out the video here! https://www.instagram.com/p/DIgx4cNtWo0/?hl=en

more about the competition here: https://fastforwardscience.de/en/2025/08/young-scientist-award-short-shortlist/

PS: I'm sympathetic to those short on time or caught up in job changes. But those pointing to #nullresults didn't understand the point of #preregistration. Those whose submissions were rejected by journals could still share the results as #preprints.

PS: I'm sympathetic to those short on time or caught up in job changes. But those pointing to #nullresults didn't understand the point of #preregistration.

Those whose submissions were rejected by journals could still share the results as #preprints.

#agu23 attendees! As you submit your abstracts, remember: Results offer valuable lessons, even if they are not what you were hoping to get.

Embrace your null-results too! Check out our null-results reports in our type of publications for more information:

https://seismica.library.mcgill.ca/author-guidelines/#publication-types

#seismology #earthquakescience #openscience #openaccess #diamondopenaccess #peerreview #AGU2023 #failedexperiment #nullresults #Scientificjournals