#litellm — Public Fediverse posts

LiteLLM Proxy Connector for WordPress AI

While I’m rather sceptical about all stuff AI I still play around with it. Especially with local setup since I have a machine powerful enough to run sophisticated models locally.

My home setup is behind an LiteLLM proxy which fronts all my models and endpoints and controls access via it’s virtual API Keys.

This way I can monitor the consumptions and have a unified endpoint for various model providers (majority runs with Ollama on my laptop, some smaller models run with Ollama on my Linux servers) and central MCP.

WordPress has added new AI capabilities in their 7.x release with the AI Plugin as the basis. Additional Plugins provide Connectors to various AI providers.

Of course I wanted to use my self-hosted setup with LiteLLM and Ollama. Couldn’t find an existing plugin for this setup and decided to vibe code it with Claude Code. The result can be found on my Forgejo Git Repo:

https://repos.mxhdr.net/maxheadroom/ai-provider-for-litellm

My setup looks like this:

The WordPress plugin adds a new Connector to WordPress’s AI capabilities. Once configured via the settings:

it can be enabled in the AI plugins as the default and various models can be chosen for various AI tasks.

Most important setting is of course the ability to increase the timeout for AI responses. Self-hosted LLMs are usually significantly slower than commercial once. That’s why the plugin allows to set a higher timeout than WordPress’ default of 30s. This might or might not work, depending on your local WordPress settings.

I tested generating Image ALT Texts, suggesting tags, summaries, editorial notes, text expansion and the plugin update so far and that worked with my setups. You mileage my vary.

#weeklyreview 34/2026

Last week of summer school vacation. Kiddo and me used the time to have lunch at IKEA and also the new Korean Restaurant at Oranienburger Strasse.

The other kiddo managed to catch a summer flu and passed it on to me. So the tail end of the week I was slightly miserable and stayed home. Didn’t drove to the countryside to avoid spreading the virus.

Tried to boost the immune system with some ginger shots and lots of garlic 😀

On Friday kiddo and me took a trip to the Alexa shopping centre. She was mostly interested in books and vinyl. Eventually bought “Purple Rain” from Prince on Vinyl. Couldn’t be prouder as a dad 😉

Apparently the trend to print cheap plastic toys with a 3D printer that we’ve seen in China has arrived in Germany. Thalia was selling 3D-printed stuff from a German brand “Spiegelburg”.

Also fiddled a lot with N8N, Paperless and LiteLLM this week to optimise my document workflow. Wrote about it here

Pulled out DeltaChat again as I was curious about their Channel and Group features. Apparently one can set up a Channel to share with other that works as a one distribution from the Initiator to the Subscribers. They can react, but not post or response in the Channel.

Groups work as bidirectional as one would expect. One thing to keep in mind is, that new members will only be added to the channel or group once the initiator comes online. Apparently this can be solved with an always on bot that run’s somewhere.

I currently don’t have any real plans to use these features, was just curious.

Read an interesting post about the Foo Fighters as anti-vaxxer supporters.

↬mastodon.social/@monoxyd/117132016181523455

Свой инференс для 25 разработчиков: 452:1, KV‑пул и почему это не экономит денег

Для тех, кто держит или собирается держать LLM внутри контура: тимлидов, DevOps, архитекторов. Здесь конфиги, цифры и грабли, а не введение в трансформеры. Что вы унесёте: историю пяти последовательных конфигураций с тем, что каждая дала и чего стоила; рабочий набор флагов vLLM под одну карту Blackwell; три неочевидных бага и обходы; разбор реального счёта с отношением вход/выход 452:1. Главный вывод, если дальше читать некогда: на агентской нагрузке кэш префикса решает больше, чем выбор модели, размер карты и всё остальное вместе взятое. Мы шли к этому через четыре промежуточных стенда и потратили лишние месяцы, потому что не понимали, что именно меряем.

https://habr.com/ru/articles/1073300/

#vLLM #LiteLLM #prefix_caching #KVcache #локальные_LLM #инференс_LLM #Qwen #агентская_разработка #RTX_PRO_6000 #claude

Как собрать AI-контур на VPS: подписки ChatGPT и Claude, OmniRoute, LiteLLM и разработка без VPN

Пошагово собираем собственный AI-контур на VPS c зарубежным IP: подключаем ChatGPT и Claude через OmniRoute, объединяем модели в LiteLLM и настраиваем Codex, Qwen Code и VS Code для работы без VPN.

https://habr.com/ru/companies/hostkey/articles/1068632/

#LLM #LiteLLM #OmniRoute #VPS #Docker #OpenAI_API #Claude_Code #AIагенты #Proxy #hostkey

Ars Technica: Terabytes of credentials leaked in massive supply-chain attack. “Terabytes worth of credentials, many belonging to the world’s biggest and most sensitive organizations, have been exposed in a supply-chain attack on LiteLLM, an open source tool that streamlines AI-driven software development. Microsoft, Amazon, Cisco, Samsung, and Salesforce are only a handful of the entities […]

#LiteLLM-Angriff: Mehr als 2500 Unternehmen betroffen | Security https://www.heise.de/news/LiteLLM-Angriff-Mehr-als-2500-Unternehmen-betroffen-11412654.html #SupplyChain #Datenleck #DataLeak #Datenschutz #privacy

This Week in Security: BugTraq, AI Hacks, and Being Dumb On Planes

https://fed.brid.gy/r/https://hackaday.com/2026/08/14/this-week-in-security-bugtraq-ai-hacks-and-being-dumb-on-planes/

Apparently, it takes less than an hour to get a massive amount of credentials.

Terabytes of credentials leaked in massive supply-chain attack

https://arstechnica.com/security/2026/08/terabytes-of-credentials-leaked-in-massive-supply-chain-attack/

#Credentials #Leak #Vulnerability #AI #LiteLLM #SupplyChain #Security #Tech

🧵 …ergänzend dazu noch diesen Toot Reihe von @GossiTheDog auf Englisch: :mastodon: https://cyberplace.social/@GossiTheDog/117082931383028334

#ki #hack #hacking #llm #sublconing #datenleck #hacker #litellm #LLMs

Hey everyone! Terabytes worth of #credentials, for more than 2500 massive Fortune 500 type orgs have been exposed in a supply-chain attack on #LiteLLM.

The breach was reported Tuesday and Wednesday by security firms #CloudSEK and #HudsonRock. (Links below) CloudSEK said it found cloud keys, repository tokens, #SSH keys, #Kubernetes secrets, package publishing credentials, environment variables, and AI provider #keys.

This, in the world of #infosec, is called A Bad Thing. The fact that some of the credentials are still live, is mind boggling.

#ai #security #SupplyChainAttack #hack

https://www.cloudsek.com/blog/ai-supply-chain-breach-2500-companies-434000-cicd-pipelines

Companies exposed: https://exposure.cloudsek.com/ai-supply-chain-incident

https://www.hudsonrock.com/blog/largest-ai-supply-chain-breach-of-2026-litellm-hack-impacts-thousands-of-global-enterprises-claim-your-ethical-disclosure

ArsTecnica overview: https://arstechnica.com/security/2026/08/terabytes-of-credentials-leaked-in-massive-supply-chain-attack/

https://arstechnica.com/security/2026/08/terabytes-of-credentials-leaked-in-massive-supply-chain-attack/

#AI #SecurityBreach #SupplyChain #litellm

“A window of roughly 40 minutes in which the LiteLLM dependency was hacked led to over 430,000 instances in which millions of secrets were harvested. This magnitude pushes us into a completely new world regarding the type of response required from the cybersecurity industry.”

A partial listing of the affected companies follows; the arstechnica article has a link to the full list.

📢⚠️ #TeamPCP used a poisoned Trivy scanner to hijack LiteLLM releases. Researchers linked the breach to 2,500+ companies and 434K CI/CD pipelines. The malware stole cloud, developer, and AI credentials.

Listen/Read: https://hackread.com/litellm-breach-2500-companies-434k-ci-cd-pipelines

#CyberSecurity #LiteLLM #DataBreach #SupplyChain

In a supply chain attack on the popular LiteLLM library in March this year, around 434,000 build pipelines – which were in use at more than 2,500 companies – are believed to have been compromised: https://www.cloudsek.com/blog/ai-supply-chain-breach-2500-companies-434000-cicd-pipelines
How you can protect yourselves against such and similar attacks is explained in our @Python4DataScience tutorial: https://python4data.science/en/latest/productive/security/index.html
#Python #ITSecurity #SupplyChain #LiteLLM

«Nach Supply-Chain-Angriff — Riesiges Datenleck betrifft unzählige große Konzerne:
Hacker sind über kompromittierte LiteLLM-Pakete an Anmeldeinformationen von fast 2.500 teils sehr bekannten Organisationen gelangt. Das Ausmaß ist enorm.»

Wie jetzt, KI wird beklaut und das sind Daten die die KI selbst ungefragt angereichert hatte?

🤖 https://www.golem.de/news/nach-supply-chain-angriff-riesiges-datenleck-betrifft-unzaehlige-grosse-konzerne-2608-211886.html

#ki #hacking #llm #suplychain #datenleck #hacker #litellm

A #supplychainattack on the #opensource tool #LiteLLM exposed #terabytes of #credentials belonging to over 2,500 organisations, including Microsoft, Amazon, and Cisco. The attack, attributed to the group TeamPCP, exploited a #vulnerability in the #vulnerabilityscanner #Trivy and compromised versions of LiteLLM, KICS, and the Telnyx Python SDK. Security firms CloudSEK and Hudson Rock urge affected organisations to rotate credentials. https://arstechnica.com/security/2026/08/terabytes-of-credentials-leaked-in-massive-supply-chain-attack/?eicker.news #tech #news #ainews

The LiteLLM attack's true reach: terabytes of credentials tied to 434,000 pipelines

Follow us and never miss a story.

https://1ban.news/litellm-credential-leak-434000-pipelines-2026/

#1ban #litellm #credential #leak #434000 #tech

Что такое LiteLLM и для чего его едят

Если вы или ваша команда уже используете LLM или планируете интегрировать их в проект, то наверняка у вас возникал вопрос, как лучше и красивее написать такой интерфейс, который мог бы общаться с разными моделями. И на такой вопрос уже есть ответ. Теперь можно сэкономить время разработки и управлять нейросетями в одном месте. Поможет в этом простой инструмент LiteLLM, про который далее и пойдет речь.

https://habr.com/ru/companies/selectel/articles/1064050/

#litellm #selectel #ml #llm

Trying to avoid writing something new, but I'm failing to find a router that does what #litellm does, but supports hosted/paid endpoints like #anthropic and #openai as well as my own #vllm . I don't want to optimize based on token usage, I want to not have to think about picking the most appropriate model for the job. Let the llm do that for me, and route to it automatically.

Ideas?

Watching #Zulip build, it includes #LiteLLM. Interesting... 🍿

#weeklyreview 24/2026

This was a busy week. Two social events and long work days. Of course also more fiddling with tech. But it all started with the 4th

Digital Independence Day Templin/Uckermark

For the fourth time we met in Templin to show and tell people about alternative to big commercial technology. This time a bunch of new faces showed up and we were a total of 8 people. Its getting traction despite our lousy advertisement efforts.

One quote from my partner in crime resonated with me especially:

You don’t need to switch directly from Windows to Linux. Rather start replacing application by application with their open source alternative. Once you’ve replace InternetExplorer or Chrome with Firefox, MS Office with LibreOffice and Outlook with Thunderbird, you’ve likely already covered the majority of your computer use. When you then switch from Windows to Linux you’ll be met with familiar applications already.

The 5th event will take place on July 5th in Templin again.

Movies & Paintings

Back on Sunday evening Berlin K2 started to watch a movie it seem to like. The story is about a woman at the end of the 18th century. She’s supposed to get a portrait of her painted by a another woman.

The scenery looked more and more familiar the longer I watched. Eventually it struck me. One scene of that movie we’ve got as an painting from my dear friend Cristina Carrillo.

The movie is the french “Portrait of a lady on fire” by director Céline Sciamma and it’s really good.

Greybeard gathering

As it was the 2nd Tuesday of the month our group of old hackers gathered again at the Pratergarten. This time it was warm enough to start sitting outside in the beer garden and in the sun. One guy joked that direct sunlight is not the right environment for hackers.

Why greybeards you ask? Because a lot of the original male software developers of Unix (think Kernighan & Ritchie ) were sporting full beards and eventually those became grey. Funny enough in our group I’m the only one with a beard … and I’m not a hacker.

Always quite an interesting meeting as those guys know a lot beside mere programming and computers. Learned quite some things about Taurus weapon systems this time.

Linux to the rescue

A friend of mine water damaged their HP Laptop. Wouldn’t boot up anymore despite the fans turning when powered on. I took out the harddrive to see if that’s still working. And it is. Got Windows 11 on it. Now while we have access to the mere files, some valuable data is still hidden inside programs and their proprietary formats.

Unfortunately the Disk didn’t fit in own HP laptop as it’s got a different SSD connector. What I had though is a SSD-to-USB adapter. I thought it should be possible to create a virtual machine though that simulates the original Laptop to some extend and use the USB-device as it’s hard drive. The normal VM wizard didn’t lead to success so I asked Perplexity to help me create the correct QEMU/KVM call.

And it delivered. I needed to create a software TPM device first as the original Laptop uses UEFI. After some fiddling I actually got the original OS booting up on my trusty Linux Laptop despite some 10 years hardware difference. The Documentation and explanation of the steps can be found here.

My friend but has now bought a similar laptop as replacement. We’ll put the disc into the new Laptop and will hope that the system will just come up. Let’s see. But good to know that my approach would have worked using virtualization.

Beer & Burgers

On Thursday it was time for our beer & burger gathering again. This time in West Berlin … Upper Burger Grill in Charlottenburg.

The Burger was quite OK. Not the best I ever had … but also not the worst 😉

After the burgers we felt fancy and decided to pay the Monkey Bar a visit. That’s a quite popular bar at the moment at the end of the Bikini House Berlin. It oversees the “Zoologischer Garten” and I guess you could see the monkeys during the day.

The view is indeed stunning. You can see over to the Gedaechtniskirche, the Bikini House and of course the Zoo. What I found most interesting was the audience. Every age group from Teenagers (I hope 16+) to people probably approaching their 80s were there. So they’re doing something right attracting such a diverse age group. Then again the Ku’damm area feels a little weird to me. Everyone seems to be pretending their rich and famous. Some surely are. Most just pretend and want to part of the hype. Also… it seems to be mostly russian speaking people nowadays in that Area.

Elk

This week the Version 1.0 of the alternative web client for Fediverse services named “Elk” was release. I’m hosting my own instance and are using it currently as my main Mastodon Client on the web.

One nice notable feature (among many others) is it’s ability to compose longer threads. The compose window let’s you split your longer writings into multiple properly linked posts. I know that some people struggle with this. Many Fediverse instances have a character limit for their posts – usually 500. I’ve patched my Mastodon instance to allow 5000 characters in a single post. So if your instance has a short limit and you want to post longer texts then consider using Elk for this purpose.

Local LLM stuff

I’ve continued playing with my LiteLLM setup. Also finally got OpenCode configured and used Qwen3.6 with Ollama on my laptop to write me some Ansible playbooks for installing and configuring OpenCode. That worked quite well.

I’ve also tried to calculate the cost of my Ollama setup on my Laptop at the energy price of 30ct/kWh. The M2 Max CPU/GPU seem to be able to churn out ~ 50 Tokens/s while consuming 65W. For 1M tokens it would roughly need 5.5 hours at 65W. That’s about 0,3575 kWh and thus ~ 10,7ct/1M tokens.

Of course I build myself a benchmark tool that produces the numbers for all available models on my machine and spits out a respective config file for LiteLLM

ollama-bench Results

Date: 2026–06–15 18:28 Models: 11 Prompts: 3 Runs/prompt: 3 Max tokens: 200
Power draw: 65W Electricity: 30 cent/kWh

Per-Model Details

deepseek-r1:1.5b

Per-run breakdown:

embeddinggemma:latest

No successful runs.

gemma4:12b-mlx

Per-run breakdown:

glm–4.7-flash:latest

Per-run breakdown:

gpt-oss:latest

Per-run breakdown:

lfm2.5-thinking:latest

Per-run breakdown:

llama3.2:1b

Per-run breakdown:

llama3.2:latest

Per-run breakdown:

mistral:latest

Per-run breakdown:

qwen3.6:35b-mlx

Per-run breakdown:

qwen3.6:latest

Per-run breakdown:

Model Comparison

Ranked by generation speed:
1. lfm2.5-thinking:latest — 220.08 tok/s
2. llama3.2:1b — 169.33 tok/s
3. deepseek-r1:1.5b — 164.58 tok/s
4. llama3.2:latest — 81.62 tok/s
5. qwen3.6:35b-mlx — 66.81 tok/s
6. qwen3.6:latest — 57.33 tok/s
7. gpt-oss:latest — 51.89 tok/s
8. glm–4.7-flash:latest — 49.06 tok/s
9. mistral:latest — 36.54 tok/s
10. gemma4:12b-mlx — 25.33 tok/s