#eleutherai — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #eleutherai, aggregated by home.social.
-
TechCrunch: EleutherAI releases massive AI training dataset of licensed and open domain text. “The dataset, called the Common Pile v0.1, took around two years to complete in collaboration with AI startups Poolside, Hugging Face, and others, along with several academic institutions. Weighing in at 8 terabytes in size, the Common Pile v0.1 was used to train two new AI models from EleutherAI, […]
-
EleutherAI is a grassroots non-profit AI research group, formed in July 2020 by Connor Leahy, Sid Black, and Leo Gao. Known for creating open-source models like GPT-Neo, GPT-J, and GPT-NeoX, their Pile dataset is widely used for training large language models. In early 2023, they incorporated as the EleutherAI Institute. #AI #OpenSource #EleutherAI #MachineLearning #GPT
https://eleuther.ai -
YouTube creators surprised to find Apple and others trained AI on their videos - Enlarge / YouTuber Marques Brownlee discusses iOS 18 in a new video. Th... - https://arstechnica.com/?p=2037316 #largelanguagemodels #eleutherai #anthropic #thepile #youtube #google #apple #tech #ai
-
Did you remember #EleutherAi exists?
I honestly completely forgot about them.
#ArtificialIntelligence #AI -
How the Foundation Model Transparency Index Distorts Transparency | EleutherAI Blog https://blog.eleuther.ai/fmti-critique/
I saw the Foundation Model Transparency Index paper come out recently and was surprised that OpenAI scored as high as they did. This Eleuther AI post breaks down how the Foundation Model Transparency index gets it all wrong, and is not really measuring transparency at all.
#fmti
#foundationmodeltransparencyindex
#opensource
#LLM
#eleutherai -
CW: Long thread/40
Some "open AI" is much more open than the industry dominating offerings. There's #EleutherAI, a donor-supported nonprofit whose model comes with documentation and code, licensed #Apache2. There are also some smaller academic offerings: #Vicuna (UCSD/CMU/Berkeley); #Koala (Berkeley) and #Alpaca (Stanford).
These are indeed more open (though Alpaca - which ran on a laptop - had to be withdrawn because it "hallucinated" so profusely).
40/
-
🚀 New episode of The Changelog!
This week we’re taking you to the hallway track of The #Linux Foundation’s #OSSummit North America 2023 in Vancouver, Canada 🇨🇦
This episode features three conversations about #opensource #AI:
1️⃣ Beyang Liu (Co-founder and CTO at #Sourcegraph)
2️⃣ @dennyglee (Developer Advocate at #Databricks)
3️⃣ Stella Biderman (Head of Research at #EleutherAI) -
Releasing 3B and 7B #RedPajama-#INCITE family of models including base, instruction-tuned & chat models — #TOGETHER
"The biggest takeaway is the demonstration that performant #LLMs can be built quickly by the open-source community. This work builds on top of our 1.2 trillion token RedPajama dataset, EleutherAI’s #Pythia training code, #FlashAttention from #Stanford and #Together, the #HELM benchmarks from Stanford #CRFM and generous support from #MILA, #EleutherAI & #LAION for compute time on the #Summit #supercomputer within the INCITE program award 'Scalable Foundation Models for Transferable Generalist AI'. We believe these kind of open collaborations, at larger scales, will be behind the best #AI systems of the future. "
-
“A really big deal”—Dolly is a free, open source, ChatGPT-style AI model - Enlarge (credit: Databricks)
On Wednesday, Databricks released... - https://arstechnica.com/?p=1931693 #largelanguagemodels #machinelearning #textsynthesis #apachespark #databricks #eleutherai #finetuning #biz #pythia #dolly #llama #meta #ai
-
There are some really good papers that have sought to make the best of the current situation, but #EleutherAI had the compute to do it the right way and so we did.
https://arxiv.org/abs/2211.08411
https://arxiv.org/abs/2202.07646
https://arxiv.org/abs/2202.07206
https://arxiv.org/abs/2207.14251We hope that this work will empower more people to work on questions in interpretability, especially the causal impact of training data on model behavior!
-
What do LLMs learn over the course of training? How do these patterns change as you scale? To help answer these questions, we are releasing a Pythia, suite of LLMs + checkpoints designed for research on interpretability and training dynamics!
The models have sizes ranging from 19M to 13B parameters, contain 143 intermediate checkpoints, and were trained on the same exact data in the same exact order.