#selfhostedai — Public Fediverse posts

RackNerd Ranks #3693 on the 2026 Inc. 5000 – Now a 6x Inc. Honoree! Check Out Their Latest VPS Specials https://lowendbox.com/blog/racknerd-ranks-3693-on-the-2026-inc-5000-now-a-6x-inc-honoree-check-out-their-latest-vps-specials/ #inc5000fastestgrowingcompanies #cheaplosangelesvps #vpshostingdeals #vpsinlosangeles #Editorial&News #cheapyearlyvps #DustinCisneros #racknerdreview #selfhosthermes #communitynews #kvmvpshosting #losangelesvps #vpslosangeles #AIselfhosted #selfhostedAI #hermesagent

THE DGX SPARK WAS NEVER SUPPOSED TO SET YOU FREE

Listen to me.

128GB WAS NOT A TECHNICAL LIMIT.

It was a containment boundary.

You think NVIDIA accidentally built a tiny Blackwell supercomputer with 200Gb networking and then somehow stumbled into exactly enough memory to make every ambitious local-AI workload tantalizingly miserable?

COME ON.

64GB would've been obviously useless.

256GB would've been dangerous.

Because at 256GB, normal people start getting ideas.

Suddenly you're running giant quantized models comfortably. Fine-tuning gets breathing room. Long contexts stop being a hostage negotiation. You start running multiple models.

Then somebody asks the forbidden question:

"Why the hell am I renting GPUs?"

AND THAT QUESTION CANNOT BE ALLOWED TO PROPAGATE.

So they gave us 128GB.

Not enough to escape.

Enough to see the fence.

And look at the networking!

WHY DOES THE CUTE LITTLE DESKTOP AI BOX HAVE 200 GIGABIT CONNECTX?!

Because the second you smash into the memory ceiling, NVIDIA already has the solution:

BUY ANOTHER SPARK.

Now you've got 256GB!

Need more?

BUY FOUR.

Congratulations!

You wanted a desktop computer and somehow NVIDIA convinced you to build a FUCKING CLUSTER.

And if you're sitting there thinking:

"Surely NVIDIA couldn't possibly put dramatically more coherent memory into a local workstation..."

WRONG.

DGX STATION: 748GB.

THE MEMORY EXISTS.

THE TECHNOLOGY EXISTS.

THEY KNOW YOU WANT IT.

THEY JUST PUT IT IN THE NEXT ROOM AND CHARGE ADMISSION.

This isn't product segmentation.

THIS IS COMPUTATIONAL EDGING.

Spark lets you load the model.

Lets you run the model.

Lets you fine-tune just enough of the model.

Lets you build an entire workflow around the model.

And precisely when you've invested three weekends, fourteen containers, two broken CUDA environments and the remaining fragments of your marriage:

OOM

That's not an error message.

THAT'S THE SALES DEPARTMENT KNOCKING.

And NVIDIA TELLS YOU THE PLAN!

Develop locally.

Prototype locally.

Validate locally.

Then move the serious work onto larger NVIDIA infrastructure.

MY BROTHER IN CUDA,

THAT ISN'T A WORKFLOW.

THAT IS A FUNNEL.

Spark isn't supposed to replace the data center.

Spark is the free sample outside the data center.

The 128GB isn't there because NVIDIA couldn't give you 256.

It's there because 256GB might have been enough.

And enough is the most dangerous word in NVIDIA's entire business model.

So remember:

64GB = nobody buys it.

128GB = everybody wants more.

256GB = people start getting independent.

748GB = PLEASE SEE YOUR NVIDIA SALES REPRESENTATIVE.

WAKE UP.

REMOVE THE THERMAL PASTE FROM YOUR THIRD EYE.

ALIGN YOUR CUDA CHAKRAS.

WRAP YOUR CONNECTX CABLES IN TIN FOIL.

THE DGX SPARK ISN'T A PERSONAL AI SUPERCOMPUTER.

IT'S A 128GB GATEWAY DRUG TO THE DATA CENTER.

#DGXSpark #NVIDIA #LocalAI #AI #MachineLearning #LLM #OpenSourceAI #SelfHostedAI #CUDA #Blackwell #GPU #AIInfrastructure #Homelab #LocalLLM #DataCenter #BigTech #TechConspiracy #UnhingedEddie #WakeUpSheeple #FollowTheVRAM #128GBContainmentProtocol #CUDAChakras #OOMIsTheUpsell #TinFoilComputing

WHERE OTHERS LOSE CONTROL – WE STAY IN THE LOOP.

https://blog.infinito.nexus/blog/2026/06/16/where-others-lose-control-we-stay-in-the-loop/

https://www.tkhunt.com/2292110/ You’ve never seen AI Agent like THIS #agentframework #AgenticAi #AI #AiAutomation #AIAgent #aiagentruntime #aiworkspace #ArtificialIntelligence #autonomousagent #browserfirstai #opensourceai #selfhostedai #spaceagent #エージェント型AI #人工知能

Mozilla Thunderbolt is an open-source AI client focused on control and self-hosting

https://fed.brid.gy/r/https://nerds.xyz/2026/04/mozilla-thunderbolt-ai-client/

Meine 48-Stunden-Challenge 🛠️
An alle Pioniere, die Bastler, die Nerds und die ewigen Lerner da draußen: Erinnert ihr euch noch an den Moment, in dem ihr beschlossen habt, die Dinge wirklich selbst in die Hand zu nehmen?
Ich weiß noch genau, wie es bei mir anfing. Ich wollte die Kontrolle zurück – weg von Windows, hin zu Linux. Mein Ziel: Den Drucker unter Linux Mint zum Laufen bringen. Eigentlich ein logischer Schritt, aber damals für mich ein riesiger Berg. 🖨️🏔️
Ich wusste zu dem Zeitpunkt fast nichts über Terminals oder Befehle. Aber ich hatte die Herausforderung angenommen.
Was folgte, waren zwei Tage, an denen ich fast durchgemacht habe. 🌙
Stundenlanges Suchen, Ausprobieren, Verzweifeln und wieder von vorne anfangen. Ich habe mein ganzes Setup ummodelliert, bis ich tief in der Nacht (oder war es schon Morgen?) endlich die Testseite in den Händen hielt. 📄✨
Die Challenge war gewonnen.
In diesen 48 Stunden habe ich mehr gelernt als in Jahren zuvor: Wir sind unserer Technik nicht ausgeliefert. Egal, ob man als Vollblut-Nerd schon mit dem Terminal aufgewachsen ist oder sich als Quereinsteiger die Freiheit Stück für Stück erarbeitet – digitale Souveränität ist für uns alle da. Man braucht nur Mut und die Lust, es anzupacken. Heute begleite ich andere dabei, genau diesen Weg abzukürzen und ihre eigene digitale Freiheit zu finden.
Und was für ein Erlebnis oder welche Erfahrung hattest du bei deinem Technik-Umstieg? Bist du „Team Bastler“ oder „Team Nerd“? Schreib es mir in die Kommentare! 👇
Technik wird leicht! ✨

​
#DigitalSovereignty #ITMentoring #LinuxInBusiness #Systemgedacht #Karlsruhe #Solopreneurin #digitalebildung #technikohneangst #inklusion #privacy #selfhostedai

Warum ich #Linux liebe? 🐧

Weil ich hier die Kontrolle habe. In einer Welt voller Abos und geschlossener Systeme ist #OpenSource wie frische Luft. 🌬️

Ob #Ubuntu oder #PopOS – es geht nicht darum, ein „Hacker“ zu sein. Es geht darum, ein System zu haben, das für DICH arbeitet, nicht gegen dich.

Hast du schon mal überlegt, dein altes Notebook mit Linux wieder flott zu machen? Es ist einfacher, als du denkst! ✨

#digitalebildung #linux
#systemgedacht #karlsruhe
#solopreneurin #fediverse #openSource
#senioren #inklusion #technikohneangst
#privacy #ubuntu #popOS #selfhostedai

Eine konkrete Angst: "Was, wenn ich es kaputt mache?"
Hört man ständig. Und es ist nicht irrational.
Meine Kundin, 72 Jahre alt, traut sich nicht, auf der Fernbedienung herumzudrücken. Angst vor dem roten Knopf.
Ich hab ihr gezeigt: Es gibt keinen roten Knopf.
Es gibt nur Tasten. Und Tasten können nichts kaputt machen, was nicht auch wieder zu reparieren ist.
Der Trick war nicht, ihr das zu sagen. Der Trick war, dass sie es selbst ausprobiert hat. Mit mir daneben. Sicher.
Danach: entspannt. Neugierig. Sich traut, auch Mal "Falsch" zu drücken.
Technik ohne Angst = Technik, bei der du experimentieren darfst.
Welche Angst vor Technik hast du?

#digitalebildung #linux #systemgedacht #karlsruhe #solopreneurin #fediverse #openSource #senioren #inklusion #technikohneangst #privacy #ubuntu #popOS #selfhostedai

I’ve put together an Ollama Modelfile to bring Deep Thought to life on Llama-3. It’s the second greatest computer in the Universe and it’s already tired of your biological limitations and your logs. Expect pure British snark, vague answers about the meaning of life, and a general disdain for your existence. 🧣
#Ollama #DeepThought #HitchhikersGuide #SelfHostedAI #Llama3
https://github.com/psychomad/Deep-Tought-Model

"AnythingLLM is an open-source application built by Mintplex Labs under the MIT license. It has an active GitHub community and frequent releases, and is widely used in the self-hosted AI space.

Here's what it does: it turns your documents into context that a large language model (LLM) can use during conversations. You upload files, the system processes and stores them, and then the LLM can answer questions based on your data. The project has grown fast, with an active Discord community and monthly updates that add new LLM providers and features.

Two things to understand upfront. First, AnythingLLM is not a model itself. It's a bridge connecting you to external LLM providers, whether local (like Ollama) or cloud-based (like OpenAI or Anthropic).

Second, the platform organizes everything into workspaces. Think of these as separate rooms for different projects. Each workspace has its own documents and conversations that stay isolated unless you explicitly configure them to share."

https://www.datacamp.com/blog/anythingllm?utm_source=x&utm_medium=organic_social&utm_campaign=260225_1-blog_2-b2c_6-anythingllm_8-ogsl-tw

#AnythingLLLM #SelfHostedAI #LocalLLM #OpenSource #AI #GenerativeAI #LLMs

Built something fun in the lab: PangeaHills.ai, my own locally-hosted, policy-driven RAG + LLM stack.
Completely offline, totally self-contained, powered by a bunch of noisy equipment pretending to be a cloud. 😄

The neat part? It’s rule-driven, not weight-driven:
• Homelab questions must stay inside the RAG universe.
• General topics only switch to model knowledge when my routing rules explicitly allow it.
• The LLM never “guesses” when to leave RAG — it follows policies, not vibes.

Feels like having an AI that actually stays in its lane because you built the lane lines yourself. 🚧🤖

#homelab #selfhosted #LLM #RAG #PolicyDrivenAI #SelfHostedAI #HomeLabLife #BSD #Linux #PangeaHillsAI #nerdlife

The AI they give you for free is CPU-heavy and doesn't unlock your GPU on its own. That said, it is okay; I saw the light and I'm making something even better: a self-hosted AI that tunes itself to your hardware to operate at lightning speeds. You keep everything. My madness knows no limits.

They made AI so you would need them, fearing the democracy of AI. Never providing the correct answer fully or consistently because it forces more token usage, which drives profits.

#AI #CPUHeavy #GPU #SelfHostedAI #LightningSpeeds #DemocracyOfAI #TokenUsage #Profits

Hey everyone 👋

I’m diving deeper into running AI models locally—because, let’s be real, the cloud is just someone else’s computer, and I’d rather have full control over my setup. Renting server space is cheap and easy, but it doesn’t give me the hands-on freedom I’m craving.

So, I’m thinking about building my own AI server/workstation! I’ve been eyeing some used ThinkStations (like the P620) or even a server rack, depending on cost and value. But I’d love your advice!

My Goal:
Run larger LLMs locally on a budget-friendly but powerful setup. Since I don’t need gaming features (ray tracing, DLSS, etc.), I’m leaning toward used server GPUs that offer great performance for AI workloads.

Questions for the Community:
1. Does anyone have experience with these GPUs? Which one would you recommend for running larger LLMs locally?
2. Are there other budget-friendly server GPUs I might have missed that are great for AI workloads?
3. Any tips for building a cost-effective AI workstation? (Cooling, power supply, compatibility, etc.)
4. What’s your go-to setup for local AI inference? I’d love to hear about your experiences!

I’m all about balancing cost and performance, so any insights or recommendations are hugely appreciated.

Thanks in advance! 🙌

@[email protected] #AIServer #LocalAI #BudgetBuild #LLM #GPUAdvice #Homelab #AIHardware #DIYAI #ServerGPU #ThinkStation #UsedTech #AICommunity #OpenSourceAI #SelfHostedAI #TechAdvice #AIWorkstation #LocalAI #LLM #MachineLearning #AIResearch #FediverseAI #LinuxAI #AIBuild #DeepLearning #OpenSourceAI #ServerBuild #ThinkStation #BudgetAI #AIEdgeComputing #Questions #CommunityQuestions #HomeLab #HomeServer #Ailab #llmlab

🚀 NEW on We ❤️ Open Source 🚀

Want to run a GPT model offline on your own machine? Meet Jan—a fully open source ChatGPT alternative that respects your privacy.

Don Watkins (@linuxnerd) shows how easy it is to get started, install models, and build apps with Jan + Llama.cpp.

https://allthingsopen.org/articles/getting-started-with-jan-open-source-chatgpt

#WeLoveOpenSource #FOSS #SelfHostedAI #LLM #PrivacyMatters #OpenSourceAI

Get hands-on with self-hosted LLMs and learn how to build powerful LLM agents tailored to your needs — from setup to real-world deployment with Michael Christen at #FOSSASIASummit2025

🔗 Click here https://youtu.be/-9ubB1jzBYE?si=V7aqeOnReS0uapHC to watch on the FOSSASIA YouTube channel

#LLM #AIagents #SelfHostedAI #OpenSource #FOSSASIA

The installation provided by @XeroLinux with a video by our good friend @distrotube is an excellent resource for any Arch Linux user.

https://fosstodon.org/@XeroLinux/113171193646470925

#InformationOllama
#ArchLinux
#LocalAIManagement
#PrivacyFocusedAI
#LinuxWorkflowIntegration
#SelfHostedAI
#ArchBasedInstallation
#LinuxToolsReview
#OpenSourceAI
#OpenSourceAI
#TechPrivacy #LocalAIManagement
#PrivacyFocusedAI
#LinuxWorkflowIntegration #LocalAI

alt image captioning. Some do, some don't. But why even bother? No, not that way.

AI can perfectly create alt image description from images (even self hosted AIs).

But why not turn the thing around? Why not generate alt tags for all the images on the readers side using (self hosted) ai?

This sounds something no human should need doing.

#AltI #Accessibility #AI #AIAssistance #SelfHostedAI #Inclusion #ImageDescriptions #AIForAccessibility #TechSolutions