No. Nooooo. Noooooooooooooooooooo.
Search
40 results for “gabrielesvelto”
-
@gabrielesvelto @mkb If you look at the late addition of Q3 figures of #Anthropic's #S3, I don't know what else you need to see to conclude this is a huge clusterfuck of shady accounting.
-
that link won't open for me, but I found this
"NVIDIA Allegedly Engineered Scarcity By Funding AI Startups To Buy Its GPUs"
https://www.techpowerup.com/354814/nvidia-allegedly-engineered-scarcity-by-funding-ai-startups-to-buy-its-gpus -
@gabrielesvelto
#LLM will be the disappearware Adobe Flash of the 2020's. -
RE: https://mas.to/@gabrielesvelto/115916333817755928
So it was artificial scarcity and price manipulation all along? Shocking, who would have thought!
-
@abartlet @gabrielesvelto I wouldn't be too sure of that. If you look into #Anthropic's S3, there is a whopping 100 Billion figure about mark-adjusted losses from the fact that their valuation went up. > 90% of that "value" is held by the magnificent seven.
This is just the tip of the iceberg. If any two neoclouds fail at the same time, #Oracle is looking at a margin call that will very likely require >50% of their total operational assets. You can't tell me that won't move markets.
-
RE: https://mas.to/@gabrielesvelto/117615583548422327
I don't think we talk enough about how massive wealth consolidation has led major investors to constantly look around for new investment vehicles and opportunities. And since they're looking for large returns, they're vulnerable to wishful thinking.
Arguably a less skewed distribution of wealth in an appropriately regulated investment environment would decrease this specific kind of systemic risk.
-
RE: https://mas.to/@gabrielesvelto/117615583548422327
It feels like I should tell people to take this with a grain of salt, but that's just because I wholeheartedly agree with all that is being said.
But I can't, using this thinking of mine, see it any other way. Well put!
-
RE: https://mas.to/@gabrielesvelto/117615583548422327
This is a dense but understandable thread on the #LLM, #GPU, memory hypecycle. Worth a read.
-
RE: https://mas.to/@gabrielesvelto/117615608547700100
And read this whole thread...
-
RE: https://mas.to/@gabrielesvelto/117615583548422327
a thread about consequences
-
Good detail on why textual watermarking was actually introduced https://mas.to/@gabrielesvelto/117615602429791179
-
Hier die Erklärung, wieso dieser ganze KI Müll implodieren wird. Die jüngeren unter uns erinnern sich ja noch an die subprime krise der 00er Jahre (looking at you, leman!). Das gleiche, nur mit Rechenzentren und Hardware. Der Knall wird kommen.
PS: wir brauchen nur noch nen guten name. Wie wär's mit Token-Crash?
@gabrielesvelto https://mas.to/@gabrielesvelto/117615583548422327
-
This is a very interesting article if you want to know how signals work within the kernel. It also reminds me that at its core a Unix signal is a very simple communication mechanism between the kernel and a process or between processes. That's it: you send a signal and some code is executed on the other side with the signal number as a parameter (and some other limited information). Their nature however makes supporting them quite convoluted and hard to handle properly. It's the kind of thing that makes me want to step back and rethink this mechanism, because there is surely a better way that's not bound by the design constraints of programming in the '70s.
https://vulpinecitrus.info/blog/what-if-you-could-handle-sigkill/
-
This morning someone sent a few messages to a classroom parent group chat I'm in. I started reading the message and boy it was long. Then there was another one from the same parent, equally long. And another one. With a bulleted list. And text formatting. By the fourth very long message a realization dawned on me.
This person is gonna get a very polite but stern private message from me and I hope that'll be enough because the alternative is a combined-arms offensive.
-
RE: https://mstdn.social/@animefeminist/117610442855091564
I'm going to quote the bit about The Seven Knights of the Marronier Kingdom because it's a delightful show:
"Don’t sleep on this series, folks! It’s a well-paced, well-animated blend of wonder tale and medieval politics featuring strong character development and a whole lot of grappling with gender norms. [...]
Put briefly: This is my favorite show of the fall so far. I love the cast and am 100% engaged in their stories. If more people could watch this show so it could get more seasons, I would consider it a personal favor, please and thank you."
-
Enjoy your Sunday with one of the funniest arias in opera: Non più andrai from The Marriage of Figaro. It's funnier if you know Italian but this one has good English subs.
You could say this is almost an anti-war song with how Figaro mocks Cherubino by riffing on military rethoric, as Mozart's music becomes pompous and martial, only to tell him he'll be marching in mud, owning lots of glory and very little money.
Given that Cherubino's proclivites with the ladies got him the boot, I feel like this is an appropriate send off for Jensen Huang.
-
I'll wrap this up with an appeal to the people who have been using LLMs extensively in their work, often becoming dependent on them: this is the time to come off this train before it's too late, though it might already be. Nobody is making any money with these things, but they made plenty of debts, so you'll be left holding the bag for the entire tech sector since you're the only ones paying for them. Over and out. 19/
-
This will cause a huge amount of capital that was poured into the tech sector to shift elsewhere and this is a good thing. Climate change is weighing down on us and on our future as a species, all the resources we have should be concentrated in fighting it, not chasing a technological fad inflated by a bunch of scammers and mega-rich buffoons having mid-life crises on commercial social media. 18/
-
In other words the only way for a modern high-performance silicon process to be profitable is to produce a large amount of chips that will be sold at high price. See how we're coming back to the artificial scarcity where this all started? To keep the semiconductor market going we need more demand and AI represented the largest demand of silicon ever, but none of that demand was real. So when the prices crash the demand evaporates and the foundries will be left with a lot of idling capacity. 17/
-
There's no magic hardware improvement around the corner, quite the contrary. Every new silicon node comes with smaller and smaller increases in density and performance over the previous one, but a significant increase in costs. A fabrication facility to produce state-of-the-art hardware takes months to go from a silicon wafer to a working chip, using machines that cost billions. 16/
-
Which leads us to the hardware itself: this is the point where people will tell you that right now things are looking bad, but the prices will come down because the hardware will improve, and so you'll be able to make powerful local models and whatnot. But here's the thing: hardware improvements are over, in fact they've been for a while. Moore's Law, Dennard scaling, every single factor which lead to faster and cheaper hardware has ceased, years ago. 15/
-
As the internet gets sloppified the chances of keeping slop out diminish, and the training costs increase. And don't forget that all this training has been done without consent, attribution or compensation of the people making the training materials. This space will be regulated eventually, and when it is that will also make the training costs increase further. 14/
-
And this is exactly what happens when you train an LLM on its outputs, because an LLM is at its core a distribution of human language. It's also the reason why LLM vendors started putting invisible watermarks in LLM outputs. They need to filter out the slop they make from their training set. But even then some slop will make its way in, and even the slop that gets filtered out is still an additional cost: you scraped it, you processed it to check for your watermark, and then you threw it away. 13/
-
Now let's say you use your distribution to generate a new bunch of data: it will look a lot like the inputs you used to make it, but it won't be the same. And if you try to make a new distribution out of that data it will be distorted. You'll lose more information and for every iteration the outputs will look different, they will be more uniform, they'll contain less details, like when you recompress a JPEG file over and over and over. 12/
-
Let's talk statistic for a minute to understand model collapse. Say you have a bunch of measurements of something and you want to build a mathematical function that will tell you what values appear in the measurements and how often they appear, this is what is called a distribution. A distribution can be used to reproduce the material it was created from, but the output will not be the same because something was lost in the process of making it. The less frequent samples, the stuff that lives at the fringes of your data set, the details. 11/
-
Training LLMs requires a gigantic amount of hardware, brute force calculations, storage, memory and power. Enormous amounts of data whose scraping is not being paid by the companies involved, it's a cost they've spread over the entire internet. As models get larger training gets more expensive. But even as companies sink more money on training LLMs have already reached a plateau in term of functionality and in some cases are degrading in a process called model collaps. 10/
-
And this is when we start talking about the tech itself and why it is unsustainable. Every single company in this space is losing money, every AI startup, every AI division in an existing company, everybody. They're losing money because the costs of running an LLM - inference - are large, and it's unclear if anybody has been profitable there. But even if they are nobody has been able to cover the escalating costs of training LLMs, which is of course a prerequisite to running them. 9/
-
This is where we have to thank Ed Zitron for its excellent reporting on the finances of these companies and his remarkable work in connecting the dots whenever some CEO talked bullshit to investors while the rest of the press was looking the other way. This is also why the Gamers Nexus investigations about GPU smuggling where so important: the demand could not slow down, by all means necessary. 8/