#llmd — Public Fediverse posts

Red Hat and Tesla engineers tackled a real production problem together.

3x output tokens/sec, 2x faster TTFT on Llama 3.1 70B with KServe + llm-d + vLLM. Fixes pushed upstream to KServe along the way.

This is what open source looks like. 🤝 🚀

https://llm-d.ai/blog/production-grade-llm-inference-at-scale-kserve-llm-d-vllm

#RedHat #Tesla #RedHatAI #vLLM #Pytorch #Kubernetes #OpenShift #KServe #llmd #Llama #OpenSource

Red Hat and Tesla engineers tackled a real production problem together.

3x output tokens/sec, 2x faster TTFT on Llama 3.1 70B with KServe + llm-d + vLLM. Fixes pushed upstream to KServe along the way.

This is what open source looks like. 🤝 🚀

https://llm-d.ai/blog/production-grade-llm-inference-at-scale-kserve-llm-d-vllm

#RedHat #Tesla #RedHatAI #vLLM #Pytorch #Kubernetes #OpenShift #KServe #llmd #Llama #OpenSource

Red Hat and Tesla engineers tackled a real production problem together.

3x output tokens/sec, 2x faster TTFT on Llama 3.1 70B with KServe + llm-d + vLLM. Fixes pushed upstream to KServe along the way.

This is what open source looks like. 🤝 🚀

https://llm-d.ai/blog/production-grade-llm-inference-at-scale-kserve-llm-d-vllm

#RedHat #Tesla #RedHatAI #vLLM #Pytorch #Kubernetes #OpenShift #KServe #llmd #Llama #OpenSource

Red Hat and Tesla engineers tackled a real production problem together.

3x output tokens/sec, 2x faster TTFT on Llama 3.1 70B with KServe + llm-d + vLLM. Fixes pushed upstream to KServe along the way.

This is what open source looks like. 🤝 🚀

https://llm-d.ai/blog/production-grade-llm-inference-at-scale-kserve-llm-d-vllm

#RedHat #Tesla #RedHatAI #vLLM #Pytorch #Kubernetes #OpenShift #KServe #llmd #Llama #OpenSource

Red Hat and Tesla engineers tackled a real production problem together.

3x output tokens/sec, 2x faster TTFT on Llama 3.1 70B with KServe + llm-d + vLLM. Fixes pushed upstream to KServe along the way.

This is what open source looks like. 🤝 🚀

https://llm-d.ai/blog/production-grade-llm-inference-at-scale-kserve-llm-d-vllm

#RedHat #Tesla #RedHatAI #vLLM #Pytorch #Kubernetes #OpenShift #KServe #llmd #Llama #OpenSource

Red Hat AI 3 is GA!

https://docs.redhat.com/en/documentation/red_hat_ai/3

#RedHat #RedHatAI #RHAIIS #OpenShift #OpenShiftAI #vLLM #KServe #Kubeflow #llmd #AI #GenAI #AIPlatform #OpenSource #OpenSourceAI

3 things to know about Red Hat AI 3

https://www.youtube.com/watch?v=eztORiJWYMs

#RedHat #AI #RedHatAI #llmd #Agentic #MCP #ModelContextProtocol #LlamaStack #OpenSource #OpenShift #OpenShiftAI

Updating stickers on laptops... let's see how many I can tag

@matrix @ansible @InstructLab @thinkpadmuseum @trustyai @github @fedora

#ramalama #docling #vllm #llmd #pytorch #NERC #redhat #ospo #upstream #ansible #thinkpad #womeninfedora #expo2025 #kubeflow #trustyAI #cushingcenter #operationstickybusiness

Big thanks to everyone contributing code, reviews, and ideas — this integration is shaping up to be a game-changer for 𝗞𝘂𝗯𝗲𝗿𝗻𝗲𝘁𝗲𝘀-𝗻𝗮𝘁𝗶𝘃𝗲 𝗟𝗟𝗠 𝘀𝗲𝗿𝘃𝗶𝗻𝗴. Stay tuned for next release!

#KServe #llmd #GenerativeAI #MLOps #Kubernetes #ModelServing #AIInfrastructure

Red Hat Launches the llm-d Community, Powering Distributed Gen AI Inference at Scale

https://www.redhat.com/en/about/press-releases/red-hat-launches-llm-d-community-powering-distributed-gen-ai-inference-scale

#RedHat #CoreWeave #Google #IBM #NVIDIA #AMD #Cisco #HuggingFace #Intel #Lambda #Mistral #OpenSource #AI #llmd #vllm #Linux #Kubernetes

🎉 Behold! The #llmd #community emerges from the depths of the #tech abyss, promising the holy grail of Kubernetes-native distributed #LLM #inference. 🤖 Because who doesn't want their #AI #deployments served with extra buzzwords and a side of "competitive performance per dollar"? 🍽️
https://llm-d.ai/blog/llm-d-announce #Kubernetes #news #HackerNews #ngated