#ai-inference — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #ai-inference, aggregated by home.social.
-
#Starcloud, a #startup developing #satellites for #AIinference in orbit, raised an additional $250 million, valuing the company at $2.3 billion. The funding will support the development of their largest orbital #datacentre #spacecraft, #Starcloud3, and secure launch capacity as the market tightens. Starcloud is collaborating with #Nvidia to develop a space-ready #GPU, aiming to launch it in late 2028. https://techcrunch.com/2026/08/21/starcloud-raises-200-million-for-orbital-data-centers-as-launch-options-dry-up/?eicker.news #tech #news #ainews
-
Cerebras is taking a rack-scale approach to AI inference with the CS-4, combining three wafer-scale engines and faster I/O instead of filling racks with separate GPUs.
-
Sandisk and SK hynix's HBF spec points to a new near-memory tier for AI inference, with much larger close-to-compute capacity as the big goal.
-
https://winbuzzer.com/2026/07/29/kimi-k3-reportedly-gets-alibaba-huawei-day-zero-support-xcxwbn/
Moonshot AI has released Kimi K3 weights, with reported Alibaba Cloud and Huawei Ascend support plus documented SGLang and Baseten deployment paths.
#AI #KimiK3 #MoonshotAI #Alibaba #AlibabaCloud #Huawei #AIModels #AICompute #AIInference #ChinaAI
-
Qualcomm HBC is taking aim at the AI inference memory wall by stacking LPDDR closer to compute instead of leaning on traditional HBM.
#tech #technology #qualcomm #aiinference #semiconductors #memorybandwidth
-
https://winbuzzer.com/2026/07/21/openrouter-sale-talks-put-ai-model-routing-in-focus-xcxwbn/
OpenRouter reportedly considers a multibillion-dollar sale; control of its AI model gateway could shape provider choice, costs, and reliability.
#AI #OpenRouter #AIModels #AIServices #AIIntegration #AIInference
-
via #Microsoft : Microsoft expands Azure AI and HPC infrastructure with AMD
https://ift.tt/AX8IuJx
#Microsoft #Azure #AI #HPC #AMD #EPYC #HXv2 #HDv2 #ND_MI455X #AIinfrastructure #silicondesign #EDA #AIinference #CPU #GPUs #3DVCache #AzureHDv2 #AzureHXv2 #Synopsys #EDA #datace… -
Sakana AI Plans to Add Nvidia Nemotron Models to Fugu
#Nemotron #SakanaAI #NVIDIA #ArtificialIntelligenceAI #AIModels #LargeLanguageModelsLLMs #AIIntegration #AIAgents #OpenSourceAI #AgenticAI #AICoding #AIBenchmarks #AIInference #AIPartnerships #GenerativeAI
-
Enterprise AI training vs. #AIInference: two fundamentally different workloads.
#AITraining: intensive GPU compute over days/weeks
Inference: fast, continuous production responsesConflate them and you overpay for infrastructure, choose the wrong hardware, and you miss compliance. Both introduce distinct security risks.
The solution?
Private, sovereign AI infrastructure = full data control + compliance.https://amazee.ai/blog/ai-training-vs-ai-running-security-guide
-
https://winbuzzer.com/2026/07/15/rambus-ddr5-chipset-pushes-server-memory-to-9600-mts-xcxwbn/
Rambus has introduced a DDR5 9600 server memory chipset reaching 9,600 MT/s, targeting AI and high-performance computing bandwidth constraints in data centers.
#AI #DDR59600 #Semiconductors #AIInfrastructure #AIInference #CloudComputing #DataCenters
-
https://winbuzzer.com/2026/07/14/openai-eases-gpt-56-timing-limits-keeps-weekly-caps-xcxwbn/
OpenAI gives paid Codex and ChatGPT Work users more scheduling flexibility, adds banked resets, and targets about 10% more effective usage.
#AI #GPT56 #GPT56Sol #OpenAI #ChatGPT #ChatGPTPlus #AIModels #AIReasoningModels #AICompute #AIInference #AIAgents #AICoding
-
https://winbuzzer.com/2026/07/14/german-consortium-launches-soofi-s-for-sparse-industrial-ai-xcxwbn/
Germany’s Soofi S AI model pairs sparse architecture with strong project-run benchmarks, but licensing gaps and long-context limits temper its promise.
#AI #SoofiS30BA3B #AIModels #MixtureOfExperts #AITraining #AICompute #AIBenchmarks #AIInference #AIModelDevelopment
-
Dentro la gerarchia della memoria delle GPU: come i server AI spostano i dati dagli SSD alla HBM
Ti sei mai chiesto come i modelli di intelligenza artificiale trasferiscono i dati alla GPU? Questo articolo spiega in modo semplice i diversi livelli di memoria all'interno di un server AI, perché la memoria della GPU è diventata un collo di bottiglia e quali nuove tecnologie potrebbero migliorare le prestazioni dell'intelligenza artificiale.
buysellram.com/blog/inside-the…
#HBM #HBM4 #GPUMemory #AIInfrastructure #DataCenter #CXL #HighBandwidthFlash #AIHardware #MemoryHierarchy #AIInference #NVMe #ITAD
-
Why can't a GPU just carry more HBM? Interposers max out in size, stacking 12 to 16 DRAM dies compounds yield losses, and every stack sits beside a kilowatt-class package that hates sharing heat. So capacity climbs in careful steps — 80 GB on the H100, 141 on the H200, 192 on the B200, 288 on Blackwell Ultra — while KV caches for long-context inference balloon past 40 GB per request.
That gap between what models demand and what packaging permits is reshaping server design. NVIDIA's Rubin platform treats CPU memory and HBM as one coherent pool. SanDisk and SK hynix are standardizing High Bandwidth Flash as a capacity tier under HBM, with first samples due this half. CXL 4.0 pooling hardware is landing in racks now.
This article walks the whole memory hierarchy, from on-chip SRAM to NVMe, and explains what each emerging technology actually solves — and what it doesn't.
#HBM #HBM4 #GPUMemory #AIInfrastructure #DataCenter #CXL #HighBandwidthFlash #AIHardware #MemoryHierarchy #AIInference #NVMe #ITAD #technology
-
A modern AI server runs five layers of memory, from nanosecond on-chip SRAM to petabyte-scale SSDs, and keeping the GPU fed is the whole engineering game. This piece walks the full hierarchy: why HBM became the bottleneck, why adding more isn't simple, and where HBF, CXL, and PIM fit into the next generation.
#HBM #HBM4 #GPUMemory #AIInfrastructure #DataCenter #CXL #HighBandwidthFlash #AIHardware #MemoryHierarchy #AIInference #NVMe #tech
-
https://winbuzzer.com/2026/07/04/nvidia-ties-ai-cloud-financing-to-future-cloud-revenue-xcxwbn/
Nvidia's new AI cloud financing strategy lowers upfront GPU buildout costs while tying supported capacity to future cloud revenue, with key payment terms still unclear.
#AI #NVIDIA #AIInfrastructure #AICompute #AIInference #AIChips
-
https://winbuzzer.com/2026/07/03/deepseek-v4-may-add-peak-hour-pricing-to-its-api-xcxwbn/
DeepSeek V4 is moving toward peak/off-peak API pricing, leaving developers to budget for busier windows while official rate details stay incomplete.
#AI #DeepSeekV4 #DeepSeek #APIPricing #AIInference #AIModels #ChinaAI
-
https://winbuzzer.com/2026/07/02/openai-engineers-say-ai-inference-costs-could-be-halved-xcxwbn/
OpenAI engineers have outlined possible AI inference optimizations that could halve model-serving costs.
#AI #AIInference #OpenAI #ChatGPT #AIModels #AIInfrastructure #AICompute
-
https://winbuzzer.com/2026/06/30/sambanova-may-seek-10b-valuation-in-ai-chip-funding-push-xcxwbn/
SambaNova is weighing a $10B AI chip funding round at about a $10 billion valuation.
#AI #SambaNova #AIInference #AIChips #AICompute #AIInfrastructure
-
https://winbuzzer.com/2026/06/29/qualcomm-dragonfly-roadmap-expands-ai-data-center-fight-xcxwbn/
Qualcomm has unveiled Dragonfly as an AI data center platform spanning CPUs, accelerators, custom silicon, connectivity, and software.
#AI #QualcommDragonfly #Qualcomm #AIInfrastructure #AIChips #AIInference #AICompute
-
Running AI in production? A lot of the bill is memory. Every token a model generates is served from fast memory next to the GPU — and that memory, HBM, is scarce and expensive.
HBF is built to attack that cost: much of HBM's bandwidth at a fraction of the price per GB. A new explainer covers what HBF is, how it could lower inference costs, and how close it actually is.https://www.buysellram.com/blog/high-bandwidth-flash-ai-inference-cost/
#HBF #HighBandwidthFlash #AIinference #HBM #AImemory #GPU #DataCenter #NAND #SanDisk #MemoryWall
-
https://winbuzzer.com/2026/06/25/openai-and-broadcom-unveil-jalapeo-ai-inference-chip-xcxwbn/
OpenAI and Broadcom have unveiled Jalapeño, a custom AI inference chip for OpenAI workloads, with late-2026 deployment planned and benchmarks still pending.
#AI #Jalapeno #Broadcom #OpenAI #AIInference #AIChips #AIInfrastructure #AICompute
-
#Baseten, a San Francisco-based company, is raising $1.5bn in a dual-tiered #funding round valuing it at up to $13bn. The company provides software and computing capacity for businesses to run #AIinference, primarily using cheaper #opensource models. This funding round comes amid a surge in demand for AI inference infrastructure and a price war in the open-source model market. https://thenextweb.com/news/baseten-1-5bn-round-13bn-valuation-ai-inference?eicker.news #tech #media #news
-
https://winbuzzer.com/2026/06/19/microsoft-expands-y-combinator-ai-startup-access-xcxwbn/
Microsoft and Y Combinator have expanded Azure, Foundry, GPU, credit, and sales-channel access for AI founders facing production infrastructure demands.
#AI #MicrosoftFoundry #YCombinator #AIStartups #Microsoft #MicrosoftAzure #AzureAI #AICompute #AIInference
-
DeepSeek has topped Ramp's June AI vendor list as US firms are increasingly betting on cheaper models.
#AI #DeepSeek #EnterpriseAI #AIModels #AIServices #AIInference #OpenSourceAI #OpenAI #Anthropic #ChinaAI