#grokking — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #grokking, aggregated by home.social.
-
Sobre grokking: “probablemente el fenómeno más importante de la IA del que casi nadie habla”
-
Как я поймал Трансформер на читерстве: гроккинг, математика и Mechanistic Interpretability
Феномен Grokking и Mechanistic Interpretability — главные тренды в исследованиях лабораторий уровня OpenAI и Anthropic. Я решил потрогать эти концепции своими руками на уровне тензоров. Цель казалась тривиальной: заставить кастомный микро-Трансформер (всего 1М параметров) выучить базовую арифметику с нуля. Однако вместо математического гения я получил ленивого мошенника. Эта статья — инженерный детектив о том, как нейросети пытаются нас обмануть (Specification Gaming), и как вскрытие Attention-матриц помогает поймать их за руку. Вскрыть Трансформер
https://habr.com/ru/articles/1008656/
#machine_learning #transformers #grokking #mechanistic_interpretability #pytorch #specification_gaming #ai_alignment
-
Как я поймал Трансформер на читерстве: гроккинг, математика и Mechanistic Interpretability
Феномен Grokking и Mechanistic Interpretability — главные тренды в исследованиях лабораторий уровня OpenAI и Anthropic. Я решил потрогать эти концепции своими руками на уровне тензоров. Цель казалась тривиальной: заставить кастомный микро-Трансформер (всего 1М параметров) выучить базовую арифметику с нуля. Однако вместо математического гения я получил ленивого мошенника. Эта статья — инженерный детектив о том, как нейросети пытаются нас обмануть (Specification Gaming), и как вскрытие Attention-матриц помогает поймать их за руку. Вскрыть Трансформер
https://habr.com/ru/articles/1008656/
#machine_learning #transformers #grokking #mechanistic_interpretability #pytorch #specification_gaming #ai_alignment
-
Как я поймал Трансформер на читерстве: гроккинг, математика и Mechanistic Interpretability
Феномен Grokking и Mechanistic Interpretability — главные тренды в исследованиях лабораторий уровня OpenAI и Anthropic. Я решил потрогать эти концепции своими руками на уровне тензоров. Цель казалась тривиальной: заставить кастомный микро-Трансформер (всего 1М параметров) выучить базовую арифметику с нуля. Однако вместо математического гения я получил ленивого мошенника. Эта статья — инженерный детектив о том, как нейросети пытаются нас обмануть (Specification Gaming), и как вскрытие Attention-матриц помогает поймать их за руку. Вскрыть Трансформер
https://habr.com/ru/articles/1008656/
#machine_learning #transformers #grokking #mechanistic_interpretability #pytorch #specification_gaming #ai_alignment
-
Как я поймал Трансформер на читерстве: гроккинг, математика и Mechanistic Interpretability
Феномен Grokking и Mechanistic Interpretability — главные тренды в исследованиях лабораторий уровня OpenAI и Anthropic. Я решил потрогать эти концепции своими руками на уровне тензоров. Цель казалась тривиальной: заставить кастомный микро-Трансформер (всего 1М параметров) выучить базовую арифметику с нуля. Однако вместо математического гения я получил ленивого мошенника. Эта статья — инженерный детектив о том, как нейросети пытаются нас обмануть (Specification Gaming), и как вскрытие Attention-матриц помогает поймать их за руку. Вскрыть Трансформер
https://habr.com/ru/articles/1008656/
#machine_learning #transformers #grokking #mechanistic_interpretability #pytorch #specification_gaming #ai_alignment
-
Provable Scaling Laws of Feature Emergence from Learning Dynamics of Grokking
https://arxiv.org/abs/2509.21519
#HackerNews #ProvableScalingLaws #FeatureEmergence #LearningDynamics #Grokking #AIResearch
-
Scammers Exploit Grok AI With Video Ad Scam to Push Malware on X – Source:hackread.com https://ciso2ciso.com/scammers-exploit-grok-ai-with-video-ad-scam-to-push-malware-on-x-sourcehackread-com/ #1CyberSecurityNewsPost #artificialintelligence #CyberSecurityNews #cybersecurity #Vulnerability #Infostealer #SocialMedia #ElonMusk #Grokking #Hackread #security #malware #twitter #Fraud #Grok #Scam #xAI #X
-
Scammers Exploit Grok AI With Video Ad Scam to Push Malware on X https://hackread.com/scammers-exploit-grok-ai-video-ad-scam-x-malware/ #ArtificialIntelligence #Cybersecurity #Vulnerability #Infostealer #SocialMedia #Security #ElonMusk #Grokking #Malware #twitter #Fraud #Grok #Scam #xAI #X
-
"Behold, the ultimate 120-page miracle cure for your terminal-phobia, offering salvation to those too enlightened to read the actual manual. 📚💸 Pay what you want, because apparently, 'Grokking' the command line shouldn't bankrupt you, unless you count the cost of overused #buzzwords. 🙄✨"
https://commandline.stribny.name/ #miraclecure #terminalphobia #paywhatyouwant #commandline #grokking #HackerNews #ngated -
"Behold, the ultimate 120-page miracle cure for your terminal-phobia, offering salvation to those too enlightened to read the actual manual. 📚💸 Pay what you want, because apparently, 'Grokking' the command line shouldn't bankrupt you, unless you count the cost of overused #buzzwords. 🙄✨"
https://commandline.stribny.name/ #miraclecure #terminalphobia #paywhatyouwant #commandline #grokking #HackerNews #ngated -
"Behold, the ultimate 120-page miracle cure for your terminal-phobia, offering salvation to those too enlightened to read the actual manual. 📚💸 Pay what you want, because apparently, 'Grokking' the command line shouldn't bankrupt you, unless you count the cost of overused #buzzwords. 🙄✨"
https://commandline.stribny.name/ #miraclecure #terminalphobia #paywhatyouwant #commandline #grokking #HackerNews #ngated -
"Behold, the ultimate 120-page miracle cure for your terminal-phobia, offering salvation to those too enlightened to read the actual manual. 📚💸 Pay what you want, because apparently, 'Grokking' the command line shouldn't bankrupt you, unless you count the cost of overused #buzzwords. 🙄✨"
https://commandline.stribny.name/ #miraclecure #terminalphobia #paywhatyouwant #commandline #grokking #HackerNews #ngated -
e509 — Maverick and Marbles
e509 with Michael and Michael - stories and discussion all around #AI, #LLMs, #llamas, generated #Quake, #grokking, #generalization and much more.
https://gamesatwork.biz/2025/04/14/e509-maverick-and-marbles/
-
e509 — Maverick and Marbles
e509 with Michael and Michael - stories and discussion all around #AI, #LLMs, #llamas, generated #Quake, #grokking, #generalization and much more.
https://gamesatwork.biz/2025/04/14/e509-maverick-and-marbles/
-
Le #grokking : Les #chercheurs ont identifié un phénomène étrange : après une longue période d' #apprentissage #infructueux , l' #intelligence #artificielle #IA #AI donne soudain des résultats.
-
'Grokking phase transitions in learning local rules with gradient descent', by Bojan Žunkovič, Enej Ilievski.
http://jmlr.org/papers/v25/22-1228.html
#grokking #tensor #models -
Two years ago, Yuri Burda and Harri Edwards, researchers at the San Francisco–based firm OpenAI, were trying to find out what it would take to get a language model to do basic arithmetic.
They wanted to know how many examples of adding up two numbers the model needed to see before it was able to add up any two numbers they gave it.
At first, things didn’t go too well. The models memorized the sums they saw but failed to solve new ones.
By accident, Burda and Edwards left some of their experiments running far longer than they meant to
—days rather than hours.
The models were shown the example sums over and over again, way past the point when the researchers would otherwise have called it quits.
But when the pair at last came back, they were surprised to find that the experiments had worked.
They’d trained a language model to add two numbers
—it had just taken a lot more time than anybody thought it should.Curious about what was going on, Burda and Edwards teamed up with colleagues to study the phenomenon.
They found that in certain cases, models could seemingly fail to learn a task
and then all of a sudden just get it, as if a lightbulb had switched on.
This wasn’t how deep learning was supposed to work.
They called the behavior #grokking -
People think #LLM #chatbots are just memorizing facts, and this belief reflects to benchmarks (multiple choice questions about trivia), and to prompt design (zero shot examples instead of explanation).
That's not what they do though. Because of #grokking and the training regime where the network always has to predict the next word where it has never seen the sentence or the document in whole before, it's task is not memorization and has never been.
What it does is understanding the world and everything in it, to be able to predict new sentences in new contexts it has never seen before which are about that world.
The misconception that LLMs are about memorizing facts is also visible in the current branch of research where people try to make LLMs forget specific facts. This misconception is really holding the whole field down.
-
At long last, the blog post I've been working on for what seems like forever is finished!
https://cprimozic.net/blog/growing-sparse-computational-graphs-with-rnns/
It's packed with lots of really cool stuff: ML #interpretability, #grokking, #tinygrad, #graphviz, and more