#phacking — Public Fediverse posts

“Promiscuous dichotomisation is yet another way of torturing the data to produce false confessions of which we must be eternally wary.” #statistics #P-hacking #stats

RE: https://bsky.app/profile/did:plc:3zyo4iakhqs47bttjalqgbk7/post/3m5jwkr7swc2j

#phacking #LLM performance:

"The researchers found that big players like Meta, Google, OpenAI, and Amazon are given special privileges to privately test multiple versions of their models and only publish the best results. This hidden practice allows them to inflate their rankings by cherry picking data, making their models appear stronger than they actually are."

https://arxiv.org/abs/2504.20879?utm_source=beehiiv&utm_medium=newsletter&utm_campaign=mediamobilize&_bhlid=045dede5ad6eedcb96ec953f2fbac11a166a2243

via Sabine Hossenfelder's newsletter

#phacking #LLM performance:

"The researchers found that big players like Meta, Google, OpenAI, and Amazon are given special privileges to privately test multiple versions of their models and only publish the best results. This hidden practice allows them to inflate their rankings by cherry picking data, making their models appear stronger than they actually are."

https://arxiv.org/abs/2504.20879?utm_source=beehiiv&utm_medium=newsletter&utm_campaign=mediamobilize&_bhlid=045dede5ad6eedcb96ec953f2fbac11a166a2243

via Sabine Hossenfelder's newsletter

Fun class class survey of other undergrads, current N=55. I'm doing #irresponsible #DataAnalysis bc not actual #research.

Still #WTF?

ghost_recv_log = How many times have you been ghosted? (log-transformed)

mosi = Misperception of others' sexual interest

bjw = Belief in a just world

swls = Satisfaction with life

csei = College self-efficacy

High self-esteem assholery? IDK.

OH WAIT. Gender!

Shit. We didn't ask.

#NotResearch #Datafishing #phacking but it's #OK I'm a #professional #oops

Fun class class survey of other undergrads, current N=55. I'm doing #irresponsible #DataAnalysis bc not actual #research.

Still #WTF?

ghost_recv_log = How many times have you been ghosted? (log-transformed)

mosi = Misperception of others' sexual interest

bjw = Belief in a just world

swls = Satisfaction with life

csei = College self-efficacy

High self-esteem assholery? IDK.

OH WAIT. Gender!

Shit. We didn't ask.

#NotResearch #Datafishing #phacking but it's #OK I'm a #professional #oops

Table 1 of the "False-positive psychology" paper (Simmons, Nelson & Simonsohn 2011, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2916240) estimate the false-positive rate of some questionable #ResearchPractices, both alone and in combination (see attached figure).

I remember reading somewhere that the authors later stated that the numbers were inaccurate and should have been somewhat higher. Does anyone have a reference for this? #FalsePositivePsychology #QRP #phacking #HARKing #ReproducibiliTea

Table 1 of the "False-positive psychology" paper (Simmons, Nelson & Simonsohn 2011, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2916240) estimate the false-positive rate of some questionable #ResearchPractices, both alone and in combination (see attached figure).

I remember reading somewhere that the authors later stated that the numbers were inaccurate and should have been somewhat higher. Does anyone have a reference for this? #FalsePositivePsychology #QRP #phacking #HARKing #ReproducibiliTea

[en] Cheating in Science: Harvard "Honesty Scholar" May Have Been Caught in Dishonesty

"... dishonesty can lead to creativity" - an interesting and somewhat amusing read.

The New York Times: "Questions about a widely cited paper are the latest to be raised about methods used in #behavioral research."

https://datacolada.org/111

#ResearchHighlights #honesty #dishonesty #phacking #harking #dredging #gino #harvard #fraud #cheating #academic #datacolada

“It’s a bit like seeing a rabbit shape in the clouds and then testing whether all clouds look like rabbits… using the same cloud. I hope you appreciate that you’re going to need some new clouds to test your theory.

Any datapoint you use to inspire a theory or question can’t be used to test that same theory.”

#phacking #statistics #rstats

https://towardsdatascience.com/the-most-powerful-idea-in-data-science-78b9cd451e72

@sociology @communicationscholars @rstats

“It’s a bit like seeing a rabbit shape in the clouds and then testing whether all clouds look like rabbits… using the same cloud. I hope you appreciate that you’re going to need some new clouds to test your theory.

Any datapoint you use to inspire a theory or question can’t be used to test that same theory.”

#phacking #statistics #rstats

https://towardsdatascience.com/the-most-powerful-idea-in-data-science-78b9cd451e72

@sociology @communicationscholars @rstats

"Do not chase better metrics. Chase better models."

Excellent lighting talk on how spatial sampling affects the quality of data driven models by Luísa Lucchese

https://youtu.be/q9dy2dQuTak

Chasing better #metrics is like #phacking

#GeoAI #gischat #datascience #machinelearning

"Do not chase better metrics. Chase better models."

Excellent lighting talk on how spatial sampling affects the quality of data driven models by Luísa Lucchese

https://youtu.be/q9dy2dQuTak

Chasing better #metrics is like #phacking

#GeoAI #gischat #datascience #machinelearning

If you’re interested in #statistics, #Metascience, #openScience, #pHacking, etc., this is quite a good read.

https://joebakcoleman.com/blog/2023/pcurve/

If you’re interested in #statistics, #Metascience, #openScience, #pHacking, etc., this is quite a good read.

https://joebakcoleman.com/blog/2023/pcurve/