#effectsize — Public Fediverse posts on home.social

This is of great importance especially for quantitative studies, where we use statistics to determine whether a treatment or condition has an effect. Indeed, when we test a drug on a (small) number of patients, we do so in the hope our results can generalise to any patient because it would be impossible to test it on everyone.

The solution is to perform a "power analysis", a calculation that tells us whether given our experimental design, the statistical test we are using is able to see an effect of a certain magnitude, if that effect is really there. In other words, this is something that tells us whether the experiment we're planning to do could give us meaningful results.

But, as I said, in order to do a power analysis we need to decide what size of effect we would like to see. So... do scientists actually do that?

We explored this question in the context of the chronic variable stress literature.

We found that only a few studies give a clear justification for the sample size used, and in those that do, only a very small fraction used a biologically meaningful effect size as part of the sample size calculation. We discuss challenges around identifying a biologically meaningful effect size and ways to overcome them.

#experiments #ExperimentalDesign #effectsize #statistics #stress #research #article #power #biology

#experiments #experimentaldesign #effectsize #statistics #stress #research

Nicola Romanò @[email protected] · 2025-07-09 · 20:05 UTC

When designing a scientific experiment, a key factor is the sample size to be used for the results of the experiment to be meaningful.

How many cells do I need to measure? How many people do I interview? How many patients do I try my new drug on?

This is of great importance especially for quantitative studies, where we use statistics to determine whether a treatment or condition has an effect. Indeed, when we test a drug on a (small) number of patients, we do so in the hope our results can generalise to any patient because it would be impossible to test it on everyone.

The solution is to perform a "power analysis", a calculation that tells us whether given our experimental design, the statistical test we are using is able to see an effect of a certain magnitude, if that effect is really there. In other words, this is something that tells us whether the experiment we're planning to do could give us meaningful results.

But, as I said, in order to do a power analysis we need to decide what size of effect we would like to see. So... do scientists actually do that?

We explored this question in the context of the chronic variable stress literature.

We found that only a few studies give a clear justification for the sample size used, and in those that do, only a very small fraction used a biologically meaningful effect size as part of the sample size calculation. We discuss challenges around identifying a biologically meaningful effect size and ways to overcome them.

#experiments #ExperimentalDesign #effectsize #statistics #stress #research #article #power #biology

#experiments #experimentaldesign #effectsize #statistics #stress #research

Nicola Romanò @[email protected] · 2025-07-09 · 20:05 UTC

When designing a scientific experiment, a key factor is the sample size to be used for the results of the experiment to be meaningful.

How many cells do I need to measure? How many people do I interview? How many patients do I try my new drug on?

This is of great importance especially for quantitative studies, where we use statistics to determine whether a treatment or condition has an effect. Indeed, when we test a drug on a (small) number of patients, we do so in the hope our results can generalise to any patient because it would be impossible to test it on everyone.

The solution is to perform a "power analysis", a calculation that tells us whether given our experimental design, the statistical test we are using is able to see an effect of a certain magnitude, if that effect is really there. In other words, this is something that tells us whether the experiment we're planning to do could give us meaningful results.

But, as I said, in order to do a power analysis we need to decide what size of effect we would like to see. So... do scientists actually do that?

We explored this question in the context of the chronic variable stress literature.

We found that only a few studies give a clear justification for the sample size used, and in those that do, only a very small fraction used a biologically meaningful effect size as part of the sample size calculation. We discuss challenges around identifying a biologically meaningful effect size and ways to overcome them.

#experiments #ExperimentalDesign #effectsize #statistics #stress #research #article #power #biology

#biology #power #article #research #stress #statistics

Nicola Romanò @[email protected] · 2025-07-09 · 20:05 UTC

When designing a scientific experiment, a key factor is the sample size to be used for the results of the experiment to be meaningful.

How many cells do I need to measure? How many people do I interview? How many patients do I try my new drug on?

This is of great importance especially for quantitative studies, where we use statistics to determine whether a treatment or condition has an effect. Indeed, when we test a drug on a (small) number of patients, we do so in the hope our results can generalise to any patient because it would be impossible to test it on everyone.

The solution is to perform a "power analysis", a calculation that tells us whether given our experimental design, the statistical test we are using is able to see an effect of a certain magnitude, if that effect is really there. In other words, this is something that tells us whether the experiment we're planning to do could give us meaningful results.

But, as I said, in order to do a power analysis we need to decide what size of effect we would like to see. So... do scientists actually do that?

We explored this question in the context of the chronic variable stress literature.

We found that only a few studies give a clear justification for the sample size used, and in those that do, only a very small fraction used a biologically meaningful effect size as part of the sample size calculation. We discuss challenges around identifying a biologically meaningful effect size and ways to overcome them.

#experiments #ExperimentalDesign #effectsize #statistics #stress #research #article #power #biology

#experiments #experimentaldesign #effectsize #statistics #stress #research

Bjørn Sætrevik @[email protected] · 2025-01-30 · 11:52 UTC

An even better solution would be a table where you could select which type of effect #effectSize measure to show (calculated using e.g. these calculations https://www.escal.site/). If anyone has the skills to implement that in #wikipedia #markup, please do so!

#effectsize #wikipedia #markup

Bjørn Sætrevik @[email protected] · 2025-01-29 · 14:50 UTC

It always takes me some minutes to look up the interpretation guidelines for various effect size measures (yes, I know the rules of thumb are somewhat arbitrary). Today I edited Wikipedia to show three different guidelines for four different measures in the same table. Hopefully this can save some time for other researchers.

#methodology #psychometrics #EffectSize #OpenScience #wikipedia

#methodology #psychometrics #effectsize #openscience #wikipedia

guyjantic has moved! @[email protected] · 2024-10-06 · 16:59 UTC

#malcolmGladwell has another book, I guess trying to rescue his much-nitpicked #TippingPoint.

IDK if he's a net positive force in the world or not. As a #psychologist I've occasionally looked up the original #research he cites. He tends to portray findings in black-and-white terms, like "People do X in Y situation!" when, most often, I've found the research best supports something like "In some studies 12% of people did X in Y situation despite previous #models predicting it should only be 7%" or "The mean of the P group was 0.3 standard deviations higher than the mean of the Q group".

I see many of his grand arguments as built more or less on a house of cards. Or rather, built on a house of semi-firm jell-o that he treats as if it were solid bricks.

I'm not knocking (most of) the #behavioralScience he cites; Hell, I'm a behavioral scientist and I think this meta-field has a ton to offer. I just think it's important to keep #EffectSize and #PracticalSignificance built into any more complex theories or models that rely on the relevant research instead of assuming that #StatisticalSignificance means "Everything at 100%". I'm sure there's some concise way to say this.

Overall, I think he plays fast and loose with a lot of scientific facts, stacking them up as if they were all Absolutely Yes when they're actually Kinda Maybe or Probably Sort Of and I don't think the weight of the stack can be borne by the accumulated uncertainty and partial applicability indicated by the component research.

So I take everything he says with huge grains of salt and sometimes grimaces, even though I think sometimes he identifies really interesting perspectives or trends.

But is it overall good to have someone presenting behavioral research, heavily oversimplified to fit the author's pet theory? It gets behavioral science in the public eye. It helps many people with no connection to behavioral science understand the potential usefulness and perhaps scale of the fields. It also sets everyone--especially behavioral scientists--up for a fall. It's only a matter of time after each of his books before people who understand the research far better than he does show up to try to set the record straight, and then what has happened to public confidence in behavioral science?

Meh.

#statistics #data #competence

#malcolmgladwell #tippingpoint #psychologist #research #models #behavioralscience

Joel Snyder @[email protected] · 2024-05-29 · 14:56 UTC

Application of JNDs to meta-science. Very sensible!

"For example, in clinical settings researchers may specify this smallest effect size of interest as the smallest difference in a health condition that patients themselves notice...this practice has been used at least since the advent of psychophysics in the second half of the 19th century"

#metascience #psychology #neuroscience #effectsize #psychophysics #samplesize

https://www.nature.com/articles/s41562-024-01858-z?utm_source=nathumbehav_etoc&utm_medium=email&utm_campaign=toc_41562_8_5&utm_content=20240529

#metascience #psychology #neuroscience #effectsize #psychophysics #samplesize

katch wreck @[email protected] · 2023-11-30 · 07:52 UTC

`This review holds two main aims. The first aim is to explain the importance of sample size and its relationship to effect size (ES) and statistical significance. The second aim is to assist researchers planning to perform sample size estimations by suggesting and elucidating available alternative software, guidelines and references that will serve different scientific purposes.`

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7745163/

#sampleSize #effectSize

#samplesize #effectsize

Jan R. Boehnke @[email protected] · 2023-09-15 · 08:43 UTC

A #SysReview from the "#ResponseShift – in Sync Working Group" analysed 150 studies
https://link.springer.com/article/10.1007/s11136-023-03495-x

Apart from the interest in the psychological phenomenon, the relative size of such effects compared to intervention effects (e.g., in #RCT https://link.springer.com/article/10.1186/s41687-022-00510-6) is very important for #StudyDesign in #HRQL research (see also https://doi.org/10.1007/s11136-023-03347-8).

Therefore an interesting descriptive finding: it was possible only for 105 of these studies to calculate #EffectSize-s.

#Psychometrics

#sysreview #responseshift #rct #studydesign #hrql #effectsize