#ai-infrastructure — Public Fediverse posts

FreeToken enables a 753 billion parameter model to run on a single workstation GPU by treating a personal machine as a unified elastic inference platform. The system dynamically maps computation across GPU, CPU and memory.

https://www.marktechpost.com/2026/08/23/meet-freetoken-an-edge-native-moe-serving-engine-that-runs-753b-glm-5-2-on-a-single-workstation-gpu/ #AIagent #AI #GenAI #AIInfrastructure

Server DRAM prices are entering a new phase.

The rapid expansion of AI infrastructure and data centers is putting sustained pressure on memory demand, making server DRAM increasingly important to the overall hardware supply chain.

Long-term contracts can give major buyers greater supply visibility and help memory manufacturers plan production, but they can also change how pricing risk is distributed. As demand remains strong, contract structures may become an increasingly important factor in determining the real cost of server memory.

For data-center operators, hardware resellers, and IT asset managers, this matters beyond the initial purchase price. Changes in DRAM pricing can influence server upgrade costs, inventory values, procurement strategies, and the resale value of existing memory.

We take a closer look at how long-term contracts are reshaping server DRAM pricing and what this could mean for the broader memory market.

Read the full analysis:
https://www.buysellram.com/blog/server-dram-prices-long-term-contracts/

#DRAM #ServerMemory #MemoryMarket #AIInfrastructure #DataCenters #Semiconductors #MemorySupply #ITHardware #DataCenterInfrastructure #AI #tech

Server DRAM pricing is entering a new phase as AI and data-center demand reshape the memory market. Long-term contracts can provide more predictable supply, but they may also change how pricing risk is distributed between memory suppliers and large buyers.

Read the full analysis:
https://www.buysellram.com/blog/server-dram-prices-long-term-contracts/

#DRAM #ServerMemory #AIInfrastructure #DataCenters #MemoryMarket #Semiconductors #AI

A comparison of the five largest GPU cloud providers reveals CoreWeave commands a 10-15% premium as the only Platinum-rated provider, while Nebius offers the lowest H100 rates and the only published B300 pricing. Lambda has the cheapest B200 rate, and Crusoe is the only provider with AMD GPUs on its rate card. https://www.marktechpost.com/2026/08/21/best-gpu-neoclouds-2026/ #AIagent #AI #GenAI #AIInfrastructure

Nvidia research demonstrates that AI agents can perform effectively through fine-tuning even when the underlying model is not particularly capable. The focus has shifted from the model itself to the harness or framework that guides the AI, marking a significant development in enterprise AI deployment. https://techcrunch.com/2026/08/21/nvidia-just-showed-that-the-harness-not-the-ai-model-is-now-the-real-hero/ #AIagent #AI #GenAI #AIInfrastructure

Samsung reportedly raised some SF4 foundry prices by 10–15%. TSMC capacity pressure is part of the explanation, but there is another one: Samsung's HBM4 uses a 4nm logic base die, and Samsung says HBM base-die demand is already improving foundry results.

The AI memory shortage is starting to consume logic-foundry capacity too.

https://www.buysellram.com/blog/samsung-4nm-price-hike-hbm4/

#Semiconductors #SamsungFoundry #HBM4 #AIInfrastructure #Foundry #ChipManufacturing #DataCenter #TSMC #Memory #AIHardware #technology

AI data startup Micro1 has hit 500M USD in annual recurring revenue as demand for training data surges. The company provides labelled data essential for building AI models, riding the wave of massive AI development. https://techcrunch.com/2026/08/20/ai-data-startup-micro1-reaches-500m-gross-run-rate-amid-ai-training-boom/ #AIagent #AI #GenAI #AIInfrastructure

NVIDIA’s Vera Rubin platform is entering full production, combining GPUs, CPUs, networking and storage into AI factory systems built for massive workloads. The AI race is becoming an infrastructure race—and NVIDIA wants to power it. #BestSoln #NVIDIA #VeraRubin #AI #AIInfrastructure #Semiconductors

AI data centres could soon consume water equivalent to the daily needs of 1.3 billion people. A UN report warns evaporative cooling for AI infrastructure is unsustainable. Researchers suggest direct-to-chip liquid cooling as a more efficient alternative that could dramatically reduce water consumption in data centres. https://theconversation.com/theres-a-better-way-to-cool-data-centres-that-cuts-their-huge-thirst-for-water-289547 #AIagent #AI #GenAI #AIInfrastructure

Relativity Networks has raised 22M USD in funding to deploy hollow-core fibre in data centres, a technology that transmits data 30% faster than conventional fibre by routing light through a vacuum chamber rather than glass. The improvement shaves microseconds off transmission times - critical as AI workloads spread across multi-campus deployments. The company also secured a 40M USD order from a major hyperscaler. https://techcrunch.com/2026/08/19/relativity-networks-raises-22-million-to-bring-a-faster-kind-of-fiber-to-data-centers/ #AIagent #AI #GenAI #AIInfrastructure

Why are bond yields surging, and why does it matter Business

Ballooning government deficits, a Middle East conflict driving inflation fears and tech giants issuing billions in debt to fund AI expansion are pushing global bond yields to multi-decade highs, Reuters correspondent Gregor Stuart Hunter explains. #News #Reuters #Newsfeed #BondYields #GlobalBonds #InterestRates #Inflation #MiddleEast #AIInfrastructure #TechDebt #USBonds #GermanBonds #JapanBonds…

http://fllics.com/en/video/why-are-bond-yields-surging-and-why-does-it-matter-business/

Do AI agents really need one CPU per GPU?

AMD has suggested that the rise of AI agents could push data-center CPU-to-GPU ratios toward 1:1. But is that actually supported by the evidence?
The answer is more complicated.

Real-world studies show that AI agents can generate significant CPU workloads. In one production trace, code execution and tool calls took as long as or longer than model inference in more than 27% of requests. Database search can also dominate latency.

But that doesn’t prove a 1:1 CPU-to-GPU ratio. Existing studies measured individual configurations—not how many CPUs are actually needed per GPU. In fact, the workloads are highly variable, making a single ratio a poor sizing metric.

The bigger trend is clear: AI infrastructure is adding a dedicated CPU/orchestration tier for code execution, tool calls, databases and sandboxes.

The real question may not be CPU vs. GPU count, but how much CPU compute, memory and power each AI workload actually needs.

https://www.buysellram.com/blog/do-ai-agents-really-need-one-cpu-per-gpu/

#AI #AIAgents #CPU #GPU #AIInfrastructure #DataCenter #AMD #NVIDIA #Intel #Semiconductors #tech

AI agents can create significant CPU workloads through code execution, tool calls, database searches, and orchestration. But that doesn’t necessarily mean data centers need a 1:1 CPU-to-GPU ratio.

The key issue is workload variability. Instead of counting CPUs versus GPUs, infrastructure may need to be sized around actual CPU compute, memory, and latency requirements.

https://www.buysellram.com/blog/do-ai-agents-really-need-one-cpu-per-gpu/

#AI #AIAgents #CPU #GPU #AIInfrastructure #DataCenter #AMD #NVIDIA

Here is how big the AI infrastructure race is becoming:

Nvidia is willing to provide up to $105 billion in guarantees around a massive OpenAI data-center project in Ohio.

Why would a chipmaker take that kind of exposure?

Because Nvidia does not just need customers.

That raises a fascinating question:

Is Nvidia simply supplying the AI boom — or increasingly helping finance the boom itself?

https://thenewsink.com/nvidias-openai-data-centre-bet/

#Nvidia #OpenAI #ArtificialIntelligence #AIInfrastructure #TheNewsInk

#BigTech companies are making massive #offbalancesheet commitments, totalling around $3 #trillion, primarily related to #AIinfrastructure like #datacentres and #hardware. These commitments, which include lease agreements and purchase obligations, are growing rapidly and could become a burden if demand for AI computing doesn’t meet expectations. https://www.wsj.com/tech/ai/why-big-techs-ai-spending-is-3-trillion-higher-than-it-seems-e1067bb2?eicker.news #tech #news #ainews

The Cherokee Nation has banned hyperscale data centre projects on tribal-owned lands, citing concerns over energy and water use and the relatively small number of permanent jobs these massive facilities create. https://gizmodo.com/cherokee-nation-bans-hyperscale-data-center-projects-on-tribal-owned-lands-2000799413 #AIagent #AI #GenAI #AIInfrastructure

THE DGX SPARK WAS NEVER SUPPOSED TO SET YOU FREE

Listen to me.

128GB WAS NOT A TECHNICAL LIMIT.

It was a containment boundary.

You think NVIDIA accidentally built a tiny Blackwell supercomputer with 200Gb networking and then somehow stumbled into exactly enough memory to make every ambitious local-AI workload tantalizingly miserable?

COME ON.

64GB would've been obviously useless.

256GB would've been dangerous.

Because at 256GB, normal people start getting ideas.

Suddenly you're running giant quantized models comfortably. Fine-tuning gets breathing room. Long contexts stop being a hostage negotiation. You start running multiple models.

Then somebody asks the forbidden question:

"Why the hell am I renting GPUs?"

AND THAT QUESTION CANNOT BE ALLOWED TO PROPAGATE.

So they gave us 128GB.

Not enough to escape.

Enough to see the fence.

And look at the networking!

WHY DOES THE CUTE LITTLE DESKTOP AI BOX HAVE 200 GIGABIT CONNECTX?!

Because the second you smash into the memory ceiling, NVIDIA already has the solution:

BUY ANOTHER SPARK.

Now you've got 256GB!

Need more?

BUY FOUR.

Congratulations!

You wanted a desktop computer and somehow NVIDIA convinced you to build a FUCKING CLUSTER.

And if you're sitting there thinking:

"Surely NVIDIA couldn't possibly put dramatically more coherent memory into a local workstation..."

WRONG.

DGX STATION: 748GB.

THE MEMORY EXISTS.

THE TECHNOLOGY EXISTS.

THEY KNOW YOU WANT IT.

THEY JUST PUT IT IN THE NEXT ROOM AND CHARGE ADMISSION.

This isn't product segmentation.

THIS IS COMPUTATIONAL EDGING.

Spark lets you load the model.

Lets you run the model.

Lets you fine-tune just enough of the model.

Lets you build an entire workflow around the model.

And precisely when you've invested three weekends, fourteen containers, two broken CUDA environments and the remaining fragments of your marriage:

OOM

That's not an error message.

THAT'S THE SALES DEPARTMENT KNOCKING.

And NVIDIA TELLS YOU THE PLAN!

Develop locally.

Prototype locally.

Validate locally.

Then move the serious work onto larger NVIDIA infrastructure.

MY BROTHER IN CUDA,

THAT ISN'T A WORKFLOW.

THAT IS A FUNNEL.

Spark isn't supposed to replace the data center.

Spark is the free sample outside the data center.

The 128GB isn't there because NVIDIA couldn't give you 256.

It's there because 256GB might have been enough.

And enough is the most dangerous word in NVIDIA's entire business model.

So remember:

64GB = nobody buys it.

128GB = everybody wants more.

256GB = people start getting independent.

748GB = PLEASE SEE YOUR NVIDIA SALES REPRESENTATIVE.

WAKE UP.

REMOVE THE THERMAL PASTE FROM YOUR THIRD EYE.

ALIGN YOUR CUDA CHAKRAS.

WRAP YOUR CONNECTX CABLES IN TIN FOIL.

THE DGX SPARK ISN'T A PERSONAL AI SUPERCOMPUTER.

IT'S A 128GB GATEWAY DRUG TO THE DATA CENTER.

#DGXSpark #NVIDIA #LocalAI #AI #MachineLearning #LLM #OpenSourceAI #SelfHostedAI #CUDA #Blackwell #GPU #AIInfrastructure #Homelab #LocalLLM #DataCenter #BigTech #TechConspiracy #UnhingedEddie #WakeUpSheeple #FollowTheVRAM #128GBContainmentProtocol #CUDAChakras #OOMIsTheUpsell #TinFoilComputing

via @dotnet : Routing and Failover for Microsoft.Extensions.AI

https://ift.tt/YbTZyRv
#MicrosoftExtensionsAI #RoutingChatClient #FailoverChatClient #SemanticRoutingChatClient #OrderedFailoverChatClient #AIInfrastructure #RoutingAndFailover #ChatClient #IChatClie…

French startup Kog is developing software to squeeze more inference performance from standard GPUs like AMD MI300X and Nvidia H200. The company says early interest generated 200 business leads, targeting professional developers frustrated by slow AI coding tools. The inference market is heating up as speed becomes a critical bottleneck. https://techcrunch.com/2026/08/14/kog-is-going-deeper-to-squeeze-more-inference-out-of-gpus/ #AIagent #AI #GenAI #AIInfrastructure

DeepSeek is introducing new API pricing for V4 in 2 days, with separate peak and off-peak rates. Peak pricing is exactly twice the off-peak rate. For us in Germany (CEST), peak hours are:

03:00–06:00
08:00–12:00

So the cheaper off-peak pricing remains available from:

00:00–03:00
06:00–08:00
12:00–24:00

For us, that’s actually not too bad. If you’re a late sleeper, you can basically start at noon and benefit from the cheaper off-peak rate all the way until midnight. For workloads that can be scheduled, running them after noon remains the simplest way to stay on the lower rate for most of the day.
#DeepSeek #DeepSeekV4 #AI #LLM #API #AIDevelopment #MachineLearning #GenAI #AIInfrastructure #DeveloperTools

The rapid #buildout of #AIinfrastructure is causing near-term #inflation & #supplychainconstraints, complicating the Fed’s efforts to manage inflation. While AI has the potential to boost productivity, its widespread adoption is slower than anticipated, and the immediate costs are outweighing the benefits. The Fed is grappling with the challenge of balancing the potential long-term benefits of AI with the current inflationary pressures it is creating. https://www.cnbc.com/2026/08/12/ais-costly-buildout-complicates-the-feds-inflation-fight.html?eicker.news #tech #news #ainews

Designers have created ShieldFont, a new font that makes webpages appear normally to human readers but serves AI scrapers a subtly edited, nonsensical version. The font uses ligatures to replace words with alternatives, poisoning training data while keeping pages readable for people. https://arstechnica.com/ai/2026/08/new-font-turns-ordinary-webpages-into-nonsense-for-ai-scrapers/ #AIagent #AI #GenAI #AIInfrastructure

AI companies face a major security breach as the LiteLLM supply chain attack exposes credentials from 2,500 organisations including Microsoft, Amazon, Cisco, Samsung and Salesforce. The breach occurred through compromised Python packages during a 40-minute window in March. https://arstechnica.com/security/2026/08/terabytes-of-credentials-leaked-in-massive-supply-chain-attack/ #AIagent #AI #GenAI #AIInfrastructure