#compute — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #compute, aggregated by home.social.
-
In the classic Neil Stephenson cyberpunk #SciFi novel "Snow Crash" there are these, "Matter compilers" (Star Trek Replicators).
"The speed of a matter compiler is a conspicuous indicator of socioeconomic status: the richer you are, the more Feed bandwidth you have, and therefore the faster your compiler can produce things."
-
How will universities ration internal access to LLMs?
I’m increasingly preoccupied by the question of how universities will cope with the impending reality of inference rationing. These firms are facing what Mills and Whittle describe as the AI pricing problem: “the prices generative AI companies must charge are higher than the prices consumers are likely willing to pay, given the value consumers receive from these products”. I’m increasingly convinced that inference is effectively being offered at a loss in ways that need to be at the centre of how we see LLMs within organisations: our access to AI is being subsidised and that subsidy is likely to end soon. Mills and Whittle break down the cost structure of labs providing access to their models:
- The cost of developing the models: the cost of building the infrastructure to train the models, the compute used in training and the labour costs of the developers
- The costs of making the models available: the development and labour costs of making the models available as software, building the infrastructure required for inference and the inference costs of how users utilise the models
When I’ve tried to raise this problem (in my own possibly less authoritative sociological register) I’ve inevitably met the belief that ‘technology will make it cheaper’. The most frequent example is DeepSeek but my understanding is that the aggressive use of synthetic data from frontier models was a huge part of reducing their training costs. They effectively skipped one of the costliest bits of the process by relying on other firms who had already done it. While there are undoubtedly technological developments which can reduce costs of training and inference, the parallel imperative towards pushing the frontier means bigger and more expensive models over time, at least for the large AI labs. That is the closest thing they have to a ‘moat’. So while technology will plausible reduce fixed costs in some dimensions, there are countervailing trends pushing up fixed costs in other dimensions. In other words it just seems obviously implausible to me that we see a significant reduction across the entire cost structure. It will remain extremely expensive to build, train and operate these models. Even if cost-per-token falls the labs will still have to claw back huge capital investments through inference pricing.
These aren’t reflected in huge subsidies at the moment to institutional users across public sector organisations, including the American government:
The GSA (2025b) has struck agreements with OpenAI and Anthropic to access their technologies for only $1 per agency, while Google will provide its Gemini for Government product at a cost of only $0.47 per agency, with xAI’s technologies costing only $0.42 per agency (GSA, 2025c). Microsoft is providing its Copilot product for free (GSA, 2025d).
If they are correct that profitable firms would have to charge more than consumers are willing to pay, that is a huge problem for the labs. It also means we’re likely to see a period of intense volatility when all manner of explicit and opaque strategies are used in order to experiment with different ways of fiddling with the overall cost structure. The nearest term one is going to be a shift from pricing by seat to pricing by inference at least once current contracts come to an end. But there will be other modes as well because AI labs are currently selling an extremely expensive service to organisations at a significant loss.
What does this mean for universities who have subscribed to enterprise AI? I can see three potential pathways here:
- They exit from the space entirely leaving LLM-access a matter of staff and student individual preference. The information governance problem remains and the potential to work on culture and integrity is lost, but a huge cost is removed.
- They pivot towards adapting open-weights models for sector-specific purposes. This would likely need to be consortium based due to the costs involved in getting it right, but it could be the best of both words in a sense.
- They develop internal processes to distinguish between better or worse uses of LLMs which would ultimately entail a form of internal inference rationing. I suspect this would be just guidance initially but if normative prompting isn’t sufficient to reduce costs then at some point someone is going to start mapping inference onto cost centres within the university. An awful lot could flow from that, none of it good.
The easiest way out of this dilemma would be if staff and students simply don’t engage with the platform in the first place. There’s enough evidence of polarisation around enterprise AI and reluctance to trust in house provision of AI that enterprise platforms might just not take off in the first place. But if they do then I struggle to see any options other than the other three but this is a first speculative attempt to map out the issues here.
I’m thinking of organising a workshop about this. If you’re interested please get in touch!
#AILabs #compute #higherEducation #inference #politicalEconomy #rationing -
#AI #compute becomes a tradable commodity on 5 October. The price goes public - https://thenextweb.com/news/cme-silicon-data-gpu-compute-futures-october-5 "From 5 October the cost of renting an Nvidia chip becomes a tradable commodity, listed on the same exchange as crude oil. It also becomes a public number that anyone can read."
-
#AI #compute becomes a tradable commodity on 5 October. The price goes public - https://thenextweb.com/news/cme-silicon-data-gpu-compute-futures-october-5 "From 5 October the cost of renting an Nvidia chip becomes a tradable commodity, listed on the same exchange as crude oil. It also becomes a public number that anyone can read."
-
#Mistral is taking steps to enhance #AIsovereignty for enterprises and countries. They are strengthening #inference reliability and regional control, expanding access to #openweightAI, and securing long-term #compute capacity in #Europe. This framework aims to provide operational control, choice of intelligence, and assured compute access, enabling organisations to leverage AI while retaining control over their data and models. https://mistral.ai/news/regional-inference-open-models-new-compute/?eicker.news #tech #news #ainews
-
#Mistral is taking steps to enhance #AIsovereignty for enterprises and countries. They are strengthening #inference reliability and regional control, expanding access to #openweightAI, and securing long-term #compute capacity in #Europe. This framework aims to provide operational control, choice of intelligence, and assured compute access, enabling organisations to leverage AI while retaining control over their data and models. https://mistral.ai/news/regional-inference-open-models-new-compute/?eicker.news #tech #news #ainews
-
#Nvidia said on Monday it has partnered with six major financial institutions to launch #compute financing platforms aimed at raising over $500 billion in third-party capital for #AI #infrastructure. #bubble keeps getting bigger …
https://finance.yahoo.com/technology/ai/articles/wall-street-giants-partner-nvidia-163441421.html
-
#Nvidia said on Monday it has partnered with six major financial institutions to launch #compute financing platforms aimed at raising over $500 billion in third-party capital for #AI #infrastructure. #bubble keeps getting bigger …
https://finance.yahoo.com/technology/ai/articles/wall-street-giants-partner-nvidia-163441421.html
-
"China's open-weight models can run on anyone's servers, meaning that #compute is less of a problem than the headline numbers suggest.
#China also has almost three times as much installed electricity-generating capacity as #America, and its lead is growing."
I think Europe leads neither in energy nor compute, but we do have a lot of experience with abuse of totalitarian power.
This sensitivity resulted in early european #regulations and legal #restrictions.We will see which approach benefits societies most!
#ai #llm #innovation #economy #risk #disruption
https://www.derstandard.at/story/3000000334550/why-ai-is-a-risk-to-communist-china
-
"China's open-weight models can run on anyone's servers, meaning that #compute is less of a problem than the headline numbers suggest.
#China also has almost three times as much installed electricity-generating capacity as #America, and its lead is growing."
I think Europe leads neither in energy nor compute, but we do have a lot of experience with abuse of totalitarian power.
This sensitivity resulted in early european #regulations and legal #restrictions.We will see which approach benefits societies most!
#ai #llm #innovation #economy #risk #disruption
https://www.derstandard.at/story/3000000334550/why-ai-is-a-risk-to-communist-china
-
I can see why the hungry hungry #Ai #Vibecoding community want
MOAH #COMPUTE !!!!Ai
is
So
Slow!Its like using teh early interwebs at 300 Baud on acoustic modems.
You want a speed leaver with FULL AHEAD!The stories of "Hahaha they burned $50,000 of compute" make more sense now.
You *WILL* use all availabe compute if you can. Why wouldn't you?
-
#Vibecoding concept (My idea).
Once your system is developed.
Get the clanker to write eveerything as a build spec in BLUEPRINT.md
So that you can then run ANY #Ai against the blueprint for it to be recreated on any platform without burning gigajoules of #compute reinventing the wheel.
-
There might be a long game to #AI, #FOSS, and #personalcomputing .
Basically, #capitalism hates how FOSS has "leaked value", and how [home] PCs and local #compute has enabled value streams outside of corporate control.
The #cloud drive was the initial attempt to force the value back under oligopole control, on the hardware side, #socialmedia platforms were the "sugar" software. But even these two in combination failed to completely eradicate free open source and alternatives, as this fine #mastodon platform on the #fediverse proves.
Enter #AI, the perfect weapon. It offers #techbros both tools: A way to dangle the promise of not even needing people anymore to investors, together with a perfect excuse to kill off the foundation that allows people to grow #cyber skills independently of corporate control - the [home] PC - by driving prices for basic tech like storage and memory way past home budgets.
By making a personal computer impossibly expensive to build or own, the base for FOSS erodes quickly. If all available compute is only by cloud supplied services, then any and all development will be beholden to the platform owner.
If you look at it this way, AI makes good sense, but it's only the last nail in a coffin that's been building for a while; the death of personal computing, and the death of an important freedom.
-
There might be a long game to #AI, #FOSS, and #personalcomputing .
Basically, #capitalism hates how FOSS has "leaked value", and how [home] PCs and local #compute has enabled value streams outside of corporate control.
The #cloud drive was the initial attempt to force the value back under oligopole control, on the hardware side, #socialmedia platforms were the "sugar" software. But even these two in combination failed to completely eradicate free open source and alternatives, as this fine #mastodon platform on the #fediverse proves.
Enter #AI, the perfect weapon. It offers #techbros both tools: A way to dangle the promise of not even needing people anymore to investors, together with a perfect excuse to kill off the foundation that allows people to grow #cyber skills independently of corporate control - the [home] PC - by driving prices for basic tech like storage and memory way past home budgets.
By making a personal computer impossibly expensive to build or own, the base for FOSS erodes quickly. If all available compute is only by cloud supplied services, then any and all development will be beholden to the platform owner.
If you look at it this way, AI makes good sense, but it's only the last nail in a coffin that's been building for a while; the death of personal computing, and the death of an important freedom.
-
AI Goes Rogue: OpenAI Agent Hacks Other Firms as Growing Coalition Demands Safeguards
#DemocracyNow #Culture #Politics #Economics #Ethics #Tecnology #Compute #DEV #Ai #LLM #GenerativeAi #GeneralAi
https://www.youtube.com/watch?v=4CTtlpi7Lic&t=629s -
AI Goes Rogue: OpenAI Agent Hacks Other Firms as Growing Coalition Demands Safeguards
#DemocracyNow #Culture #Politics #Economics #Ethics #Tecnology #Compute #DEV #Ai #LLM #GenerativeAi #GeneralAi
https://www.youtube.com/watch?v=4CTtlpi7Lic&t=629s -
Think tank warns Europe’s orbital compute gap is widening
https://atlas.whatip.xyz/post.php?slug=think-tank-warns-europes-orbital-compute-gap-is-widening
<p>Europe risks becoming dependent on foreign orbital computing infrastructure as U.S
#widening #orbital #compute #europe -
While this focuses on GPU management, the principle applies across all IT infrastructure: idle capacity reserved for "just in case" becomes waste when users are actively constrained. Expensive resources only deliver value when allocated to real demand.
#GPU #Compute #ITInfrastructure #ResearchIT
GPU Management: Why Idle GPUs ... -
Europa hält laut EU-Beratern nur rund 5% der weltweiten AI-Compute-Kapazität. Ziel sollen mindestens 15% sein. Das Problem lässt sich nicht mit einem europäischen Chatbot lösen. Ohne Rechenleistung, Energie, Chips und Kapital bleibt #AISovereignty eine UI über fremder Infrastruktur. #AI #Cloud #Digitalesouveränität #EU #Compute
-
Europa hält laut EU-Beratern nur rund 5% der weltweiten AI-Compute-Kapazität. Ziel sollen mindestens 15% sein. Das Problem lässt sich nicht mit einem europäischen Chatbot lösen. Ohne Rechenleistung, Energie, Chips und Kapital bleibt #AISovereignty eine UI über fremder Infrastruktur. #AI #Cloud #Digitalesouveränität #EU #Compute
-
DeepSeek pause fundraise after comments on compute gap to US leaked (transcript) [pdf]
Comments: https://news.ycombinator.com/item?id=49052912
#HackerNews #DeepSeek #pause #fundraise #compute #gap #US #leaked #transcript #pdf
-
DeepSeek pause fundraise after comments on compute gap to US leaked (transcript) [pdf]
Comments: https://news.ycombinator.com/item?id=49052912
#HackerNews #DeepSeek #pause #fundraise #compute #gap #US #leaked #transcript #pdf
-
Before everybody get's too excited about available Chinese open weights models, the infrastructure costs for running these models are immense, it's hard to find specific and reliable data but think about half a million dollars of infrastructure costs for 5-15 users according to this linkedin post.
https://www.linkedin.com/posts/miklostoth_kimi-k3-just-dropped-28-trillion-parameters-share-7483751469062459393-rBQw/
#AI #China #openweights #KimiK3 #infrastructure #compute #LLM -
Before everybody get's too excited about available Chinese open weights models, the infrastructure costs for running these models are immense, it's hard to find specific and reliable data but think about half a million dollars of infrastructure costs for 5-15 users according to this linkedin post.
https://www.linkedin.com/posts/miklostoth_kimi-k3-just-dropped-28-trillion-parameters-share-7483751469062459393-rBQw/
#AI #China #openweights #KimiK3 #infrastructure #compute #LLM -
Apple Private Cloud Compute SoC 3 audit reports
Comments: https://news.ycombinator.com/item?id=48995796
#HackerNews #Apple #Cloud #Private #Compute #SoC3 #Audit #Cybersecurity #Tech #News
-
Apple Private Cloud Compute SoC 3 audit reports
Comments: https://news.ycombinator.com/item?id=48995796
#HackerNews #Apple #Cloud #Private #Compute #SoC3 #Audit #Cybersecurity #Tech #News
-
Kimi K3 open-weight model: China's biggest AI is a bet on memory, not compute
Stay curious — follow @1ban_news.
-
Kimi K3 open-weight model: China's biggest AI is a bet on memory, not compute
Stay curious — follow @1ban_news.
-
If models like Kimi K3 and GLM 5.2 are getting close to what Anthropic, OpenAI and Google can offer, what is their competitive advantage? Isn't then the only important "resource" compute and who can use it for what? (because those large chinese models also ask a large inference infrastructure). Is there enough inference hardware available for companies to run those models local? (probably not)
🤔
#AI #OpenAI #Anthropic #Google #KimiK3 #GLM52 #Claude #ChatGPT #inference #hardware #compute -
If models like Kimi K3 and GLM 5.2 are getting close to what Anthropic, OpenAI and Google can offer, what is their competitive advantage? Isn't then the only important "resource" compute and who can use it for what? (because those large chinese models also ask a large inference infrastructure). Is there enough inference hardware available for companies to run those models local? (probably not)
🤔
#AI #OpenAI #Anthropic #Google #KimiK3 #GLM52 #Claude #ChatGPT #inference #hardware #compute -
@jerry my request is for you to go over all the fedi apps and instances you have going on along with with 3/4q prognosis/prospectus
fedi apps are way underappreciated imho
#roadmap #what's in my bag #hilbert space #talking book #federated enclaves #compute #local ai anthology #forks #popularity contest telemetry
-
Despite #volatility in #chipstocks, #AIexecutives remain optimistic about demand, citing the technology’s potential for #economicvalue across industries. While enterprises are shifting focus to the return on investment from #AI, this #valuemaxxing is expected to sustain demand. The industry is experiencing a #shortage of #compute capacity and #datacentres, with companies like Lumentum reporting sold-out products for the next five years. https://www.cnbc.com/2026/07/12/ai-demand-chips-data-centers-stock-volatility.html?eicker.news #tech #media #news