#scientivism — Public Fediverse posts

#LLMs "have the potential of playing an important role in [...]opinion formation in online social media"🫡🤖
Not surprising. But also not encouraging

Is it a potential we even want to research? under what terms?
Thoughts?
https://arxiv.org/abs/2312.15523

#NLProc #scientivism #ethics #ml #machinelearning #chatgpt

Solar mixes two base-model copies
to create a larger one
Then train it a bit more and beat other open models out there.
How? and my thoughts 🧵

(no author with a handle?!)
https://arxiv.org/abs/2312.15166
#scientivism #LLM #LLMS #pretraining #MIXTRAL #SOLAR #machinelearning #ml #NLP

So many warn that evaluating with GPT favors GPT

(or any LLM evaluating itself).

Now it is also shown

Science, not just educated guesses

(Fig: T5, GPT, Bart each prefer their own) https://arxiv.org/abs/2311.09766

#enough2skim #scientivism #NLP #nlproc #GPT #LLM #eval #data

You know what?
I will stop sharing any LLM "news" if they don't share with me first (models or code)

#thereOrIDontCare
#scientivism
#uShareFirst

Thanks delip rao for inspiration
And the new vision and language that did open unlike XXXX-e
https://twitter.com/DrJimFan/status/1633179734803890177?t=VaQS4E56eEq55NHKN3y5hg&s=19
#machinelearning #cv #nlproc #nlp

So often we are reminded that good work goes unnoticed
I share others' papers to change that
What else could we do?
What mechanisms better allow propagation by value rather than by fame?
Is there something we can do to make science better?
https://blog.samaltman.com/you-and-your-research

#scientivism #ScienceMastodon #PR #NLProc #machinelearning #CV

A surprising take on why we should open LLMs:
otherwise empirical research would suffocate and
rule-based (nativist) would return

Not sure I am buying it or even that it is dreadful, but more the reason to share and hear opinions
https://arxiv.org/abs/2301.05272
Patrick Perrine
#LLM #NLP #nlproc #machinelearning #ML #scientivism

We want to pretrain🤞
Instead we finetune🚮😔
Could we collaborate?🤗

ColD Fusion:
🔄Recycle finetuning to multitask
➡️evolve pretrained models forever

On 35 datasets
+2% improvement over RoBERTa
+7% in few shot settings
🧵

#NLProc #MachinLearning #NLP #ML #modelRecyclying #collaborativeAI #scientivism #pretrain

We want to pretrain🤞
Instead we finetune🚮😔
Could we collaborate?🤗

ColD Fusion:
🔄Recycle finetuning to multitask
➡️evolve pretrained models forever

On 35 datasets
+2% improvement over RoBERTa
+7% in few shot settings
🧵

#NLProc #MachinLearning #NLP #ML #modelRecyclying #collaborativeAI #scientivism #pretrain

We want to pretrain🤞
Instead we finetune🚮😔
Could we collaborate?🤗

ColD Fusion:
🔄Recycle finetuning to multitask
➡️evolve pretrained models forever

On 35 datasets
+2% improvement over RoBERTa
+7% in few shot settings
🧵

#NLProc #MachinLearning #NLP #ML #modelRecyclying #collaborativeAI #scientivism #pretrain

We want to pretrain🤞
Instead we finetune🚮😔
Could we collaborate?🤗

ColD Fusion:
🔄Recycle finetuning to multitask
➡️evolve pretrained models forever

On 35 datasets
+2% improvement over RoBERTa
+7% in few shot settings
🧵

#NLProc #MachinLearning #NLP #ML #modelRecyclying #collaborativeAI #scientivism #pretrain

We want to pretrain🤞
Instead we finetune🚮😔
Could we collaborate?🤗

ColD Fusion:
🔄Recycle finetuning to multitask
➡️evolve pretrained models forever

On 35 datasets
+2% improvement over RoBERTa
+7% in few shot settings
🧵

#NLProc #MachinLearning #NLP #ML #modelRecyclying #collaborativeAI #scientivism #pretrain

🔖Reviewing has so many faults📖
Finally, there is a dataset of reviews, edits and everything else!

5 venues 5K papers 11K reviews
Enjoy!

https://arxiv.org/abs/2211.06651
Nils Dycke, Ilia Kuznetsov, Iryna Gurevych

#NLProc #review #CV #machinelearning #scientivism

Are findings as good as ACL?

years since the first findings papers were introduced
since chris manning & ani nenkova called for a yearly analysis
since they were first done

Who's game for the yearly analysis?

https://twitter.com/chrmanning/status/1451261089644089394

For earlier analysis and code (old, not on :mastodondance: , next year links from here?)

https://twitter.com/gneubig/status/1451317437392113665?s=20&t=B0RYiSShHJQPpITBWbKF8g
https://twitter.com/ryandcotterell/status/1451551344012181514?s=20&t=bURi5XYeaZS93-GtCWW23Q
https://twitter.com/wilkeraziz/status/1451896682321485824?s=20&t=bURi5XYeaZS93-GtCWW23Q
https://twitter.com/sanxing_chen/status/1451325907918934019?s=20&t=cQ8RTuwN1plau9mrLtExXw
#NLProc #findings #scientivism #ACL

What do we know about using a fine-tuned model rather than the pretrained
They are sometimes much better, but what else?

A story of great #scientivism hypotheses and their rejections

The story of a field
Survey 🧵