#effectsize — Public Fediverse posts

Another #PeerReview done.

Manuscript c3,000 words
Review c2,300 words
3.25hrs

I do love Null results.*

Nevertheless, a good theoretical background is important (and ideally written down before the results are known).

It should be clear what an effect could look like.
#EffectSize #SampleSize

Superiority is different to non-inferiority.
#RCT

#PreRegistration #RegisteredReport

* ad libbing on Julia Rohrer's post here:
https://www.the100.ci/2017/06/01/why-we-should-love-null-results/

When designing a scientific experiment, a key factor is the sample size to be used for the results of the experiment to be meaningful.

How many cells do I need to measure? How many people do I interview? How many patients do I try my new drug on?

This is of great importance especially for quantitative studies, where we use statistics to determine whether a treatment or condition has an effect. Indeed, when we test a drug on a (small) number of patients, we do so in the hope our results can generalise to any patient because it would be impossible to test it on everyone.

The solution is to perform a "power analysis", a calculation that tells us whether given our experimental design, the statistical test we are using is able to see an effect of a certain magnitude, if that effect is really there. In other words, this is something that tells us whether the experiment we're planning to do could give us meaningful results.

But, as I said, in order to do a power analysis we need to decide what size of effect we would like to see. So... do scientists actually do that?

We explored this question in the context of the chronic variable stress literature.

We found that only a few studies give a clear justification for the sample size used, and in those that do, only a very small fraction used a biologically meaningful effect size as part of the sample size calculation. We discuss challenges around identifying a biologically meaningful effect size and ways to overcome them.

Read more here!
https://physoc.onlinelibrary.wiley.com/doi/10.1113/EP092884

#experiments #ExperimentalDesign #effectsize #statistics #stress #research #article #power #biology

Application of JNDs to meta-science. Very sensible!

"For example, in clinical settings researchers may specify this smallest effect size of interest as the smallest difference in a health condition that patients themselves notice...this practice has been used at least since the advent of psychophysics in the second half of the 19th century"

#metascience #psychology #neuroscience #effectsize #psychophysics #samplesize

https://www.nature.com/articles/s41562-024-01858-z?utm_source=nathumbehav_etoc&utm_medium=email&utm_campaign=toc_41562_8_5&utm_content=20240529

`This review holds two main aims. The first aim is to explain the importance of sample size and its relationship to effect size (ES) and statistical significance. The second aim is to assist researchers planning to perform sample size estimations by suggesting and elucidating available alternative software, guidelines and references that will serve different scientific purposes.`

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7745163/

#sampleSize #effectSize

A #SysReview from the "#ResponseShift – in Sync Working Group" analysed 150 studies
https://link.springer.com/article/10.1007/s11136-023-03495-x

Apart from the interest in the psychological phenomenon, the relative size of such effects compared to intervention effects (e.g., in #RCT https://link.springer.com/article/10.1186/s41687-022-00510-6) is very important for #StudyDesign in #HRQL research (see also https://doi.org/10.1007/s11136-023-03347-8).

Therefore an interesting descriptive finding: it was possible only for 105 of these studies to calculate #EffectSize-s.

#Psychometrics