#localllms — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #localllms, aggregated by home.social.
-
Learn how KV cache spilling hurts LLM inference, how to detect the cliff, and how to estimate it using real VRAM and throughput signals. https://hackernoon.com/how-to-find-the-context-cliff-in-local-llms #localllms
-
Learn how KV cache spilling hurts LLM inference, how to detect the cliff, and how to estimate it using real VRAM and throughput signals. https://hackernoon.com/how-to-find-the-context-cliff-in-local-llms #localllms
-
Learn how KV cache spilling hurts LLM inference, how to detect the cliff, and how to estimate it using real VRAM and throughput signals. https://hackernoon.com/how-to-find-the-context-cliff-in-local-llms #localllms
-
Learn how KV cache spilling hurts LLM inference, how to detect the cliff, and how to estimate it using real VRAM and throughput signals. https://hackernoon.com/how-to-find-the-context-cliff-in-local-llms #localllms
-
Learn how KV cache spilling hurts LLM inference, how to detect the cliff, and how to estimate it using real VRAM and throughput signals. https://hackernoon.com/how-to-find-the-context-cliff-in-local-llms #localllms
-
Part two of the article I mentioned yesterday (https://mastodon.social/@arsalan_zaidi/116883996000783250)
#AI #LocalLLMs #LLM #coding #softwaredevelopment #vibecoding
https://martinfowler.com/articles/exploring-gen-ai/local-models-for-coding-experiences.html
-
Part two of the article I mentioned yesterday (https://mastodon.social/@arsalan_zaidi/116883996000783250)
#AI #LocalLLMs #LLM #coding #softwaredevelopment #vibecoding
https://martinfowler.com/articles/exploring-gen-ai/local-models-for-coding-experiences.html
-
Part two of the article I mentioned yesterday (https://mastodon.social/@arsalan_zaidi/116883996000783250)
#AI #LocalLLMs #LLM #coding #softwaredevelopment #vibecoding
https://martinfowler.com/articles/exploring-gen-ai/local-models-for-coding-experiences.html
-
Part two of the article I mentioned yesterday (https://mastodon.social/@arsalan_zaidi/116883996000783250)
#AI #LocalLLMs #LLM #coding #softwaredevelopment #vibecoding
https://martinfowler.com/articles/exploring-gen-ai/local-models-for-coding-experiences.html
-
Another little data point on the viability of local models for coding on modest hardware.
#AI #LocalLLMs #LLM #coding #softwaredevelopment #vibecoding
https://martinfowler.com/articles/exploring-gen-ai/local-models-for-coding-factors.html
-
Another little data point on the viability of local models for coding on modest hardware.
#AI #LocalLLMs #LLM #coding #softwaredevelopment #vibecoding
https://martinfowler.com/articles/exploring-gen-ai/local-models-for-coding-factors.html
-
Another little data point on the viability of local models for coding on modest hardware.
#AI #LocalLLMs #LLM #coding #softwaredevelopment #vibecoding
https://martinfowler.com/articles/exploring-gen-ai/local-models-for-coding-factors.html
-
Another little data point on the viability of local models for coding on modest hardware.
#AI #LocalLLMs #LLM #coding #softwaredevelopment #vibecoding
https://martinfowler.com/articles/exploring-gen-ai/local-models-for-coding-factors.html
-
GLM-4.7-Flash is a 30B model with 3B active params, so why did it crawl on my mini PC? Not the weights, not the quant. The attention design, and the runtime. https://hackernoon.com/a-30b-model-crawled-on-my-mini-pc-but-the-weights-were-innocent #localllms
-
GLM-4.7-Flash is a 30B model with 3B active params, so why did it crawl on my mini PC? Not the weights, not the quant. The attention design, and the runtime. https://hackernoon.com/a-30b-model-crawled-on-my-mini-pc-but-the-weights-were-innocent #localllms
-
GLM-4.7-Flash is a 30B model with 3B active params, so why did it crawl on my mini PC? Not the weights, not the quant. The attention design, and the runtime. https://hackernoon.com/a-30b-model-crawled-on-my-mini-pc-but-the-weights-were-innocent #localllms
-
GLM-4.7-Flash is a 30B model with 3B active params, so why did it crawl on my mini PC? Not the weights, not the quant. The attention design, and the runtime. https://hackernoon.com/a-30b-model-crawled-on-my-mini-pc-but-the-weights-were-innocent #localllms
-
GLM-4.7-Flash is a 30B model with 3B active params, so why did it crawl on my mini PC? Not the weights, not the quant. The attention design, and the runtime. https://hackernoon.com/a-30b-model-crawled-on-my-mini-pc-but-the-weights-were-innocent #localllms
-
Your local LLM runs fine until it doesn't. A look at KV cache spilling from VRAM into shared memory, and why it happens silently on Windows. https://hackernoon.com/why-local-llms-suddenly-slow-down-at-long-context #localllms
-
Your local LLM runs fine until it doesn't. A look at KV cache spilling from VRAM into shared memory, and why it happens silently on Windows. https://hackernoon.com/why-local-llms-suddenly-slow-down-at-long-context #localllms
-
Your local LLM runs fine until it doesn't. A look at KV cache spilling from VRAM into shared memory, and why it happens silently on Windows. https://hackernoon.com/why-local-llms-suddenly-slow-down-at-long-context #localllms
-
Your local LLM runs fine until it doesn't. A look at KV cache spilling from VRAM into shared memory, and why it happens silently on Windows. https://hackernoon.com/why-local-llms-suddenly-slow-down-at-long-context #localllms
-
Your local LLM runs fine until it doesn't. A look at KV cache spilling from VRAM into shared memory, and why it happens silently on Windows. https://hackernoon.com/why-local-llms-suddenly-slow-down-at-long-context #localllms
-
cleaning out my computer to make space for #localLLMs. the biggest file on here is a 40GB Twitch VOD of me streaming my first time playing Overwatch 2 for my friends in chat. 4K & 60FPS but 1) i was so bad at the game 2) the OBS sound mixing was horrendous -- you can hardly hear how sad i was about Blizzard killing OW1. there is no way i am deleting this lol. the memories! #rip #overwatch #activision_blizzard
-
cleaning out my computer to make space for #localLLMs. the biggest file on here is a 40GB Twitch VOD of me streaming my first time playing Overwatch 2 for my friends in chat. 4K & 60FPS but 1) i was so bad at the game 2) the OBS sound mixing was horrendous -- you can hardly hear how sad i was about Blizzard killing OW1. there is no way i am deleting this lol. the memories! #rip #overwatch #activision_blizzard
-
-
-
-
-
-
J'ai pris un moment pour jouer avec les données de #LLM de @betagouv
👉 https://comparia.beta.gouv.fr/ranking
Un modèle local capable consomme 40 à 120x moins d'énergie par token qu'un frontier!!!
Pour mon cas, utiliser les modèles locaux pour le "quotidien" (rédaction, brainsto, recherches) et garder les API frontier seulement pour le dev, c'est ❗une emprunte carbone divisée par 80❗
+ la confidentialité des données :))
Note : la conso comparIA est un ordre de grandeur
-
J'ai pris un moment pour jouer avec les données de #LLM de @betagouv
👉 https://comparia.beta.gouv.fr/ranking
Un modèle local capable consomme 40 à 120x moins d'énergie par token qu'un frontier!!!
Pour mon cas, utiliser les modèles locaux pour le "quotidien" (rédaction, brainsto, recherches) et garder les API frontier seulement pour le dev, c'est ❗une emprunte carbone divisée par 80❗
+ la confidentialité des données :))
Note : la conso comparIA est un ordre de grandeur
-
J'ai pris un moment pour jouer avec les données de #LLM de @betagouv
👉 https://comparia.beta.gouv.fr/ranking
Un modèle local capable consomme 40 à 120x moins d'énergie par token qu'un frontier!!!
Pour mon cas, utiliser les modèles locaux pour le "quotidien" (rédaction, brainsto, recherches) et garder les API frontier seulement pour le dev, c'est ❗une emprunte carbone divisée par 80❗
+ la confidentialité des données :))
Note : la conso comparIA est un ordre de grandeur
-
J'ai pris un moment pour jouer avec les données de #LLM de @betagouv
👉 https://comparia.beta.gouv.fr/ranking
Un modèle local capable consomme 40 à 120x moins d'énergie par token qu'un frontier!!!
Pour mon cas, utiliser les modèles locaux pour le "quotidien" (rédaction, brainsto, recherches) et garder les API frontier seulement pour le dev, c'est ❗une emprunte carbone divisée par 80❗
+ la confidentialité des données :))
Note : la conso comparIA est un ordre de grandeur
-
J'ai pris un moment pour jouer avec les données de #LLM de @betagouv
👉 https://comparia.beta.gouv.fr/ranking
Un modèle local capable consomme 40 à 120x moins d'énergie par token qu'un frontier!!!
Pour mon cas, utiliser les modèles locaux pour le "quotidien" (rédaction, brainsto, recherches) et garder les API frontier seulement pour le dev, c'est ❗une emprunte carbone divisée par 80❗
+ la confidentialité des données :))
Note : la conso comparIA est un ordre de grandeur
-
NVIDIA's RTX Spark brings CUDA and 128GB unified memory to mainstream Windows PCs this fall. The move reshapes local-AI hardware decisions: the question shifts from 'buy the biggest card you can afford' to 'which constraint fails first—memory, bandwidth, or model quality.' https://www.implicator.ai/nvidias-rtx-spark-splits-the-local-ai-hardware-decision-in-two/ #AI #Hardware #LocalLLMs
-
NVIDIA's RTX Spark brings CUDA and 128GB unified memory to mainstream Windows PCs this fall. The move reshapes local-AI hardware decisions: the question shifts from 'buy the biggest card you can afford' to 'which constraint fails first—memory, bandwidth, or model quality.' https://www.implicator.ai/nvidias-rtx-spark-splits-the-local-ai-hardware-decision-in-two/ #AI #Hardware #LocalLLMs
-
NVIDIA's RTX Spark brings CUDA and 128GB unified memory to mainstream Windows PCs this fall. The move reshapes local-AI hardware decisions: the question shifts from 'buy the biggest card you can afford' to 'which constraint fails first—memory, bandwidth, or model quality.' https://www.implicator.ai/nvidias-rtx-spark-splits-the-local-ai-hardware-decision-in-two/ #AI #Hardware #LocalLLMs
-
NVIDIA's RTX Spark brings CUDA and 128GB unified memory to mainstream Windows PCs this fall. The move reshapes local-AI hardware decisions: the question shifts from 'buy the biggest card you can afford' to 'which constraint fails first—memory, bandwidth, or model quality.' https://www.implicator.ai/nvidias-rtx-spark-splits-the-local-ai-hardware-decision-in-two/ #AI #Hardware #LocalLLMs
-
NVIDIA's RTX Spark brings CUDA and 128GB unified memory to mainstream Windows PCs this fall. The move reshapes local-AI hardware decisions: the question shifts from 'buy the biggest card you can afford' to 'which constraint fails first—memory, bandwidth, or model quality.' https://www.implicator.ai/nvidias-rtx-spark-splits-the-local-ai-hardware-decision-in-two/ #AI #Hardware #LocalLLMs
-
Mein #arbeitgeber labert grade in so nem #MicrosoftTeams Call für alle Mitarbeiter was von #digitalesouveranitat und dann soll ich MEHR mit #microsoft #github #copilot machen. Und selbstverständlich wird ALLES #ai. Sogar unsere TLD wechselt von .net auf .ai.
Wir sollen ganz explizit doch bitte #ki in die tägliche #Arbeit einbinden, der Vertrieb soll sich beim. Aber bloß "unsere" nutzen, wegen den Daten. Muss mich gleich mal informieren, ob #localLLMs erlaubt sind.
https://hessen.social/@Moonstone2487/116082676681677166 -
Mein #arbeitgeber labert grade in so nem #MicrosoftTeams Call für alle Mitarbeiter was von #digitalesouveranitat und dann soll ich MEHR mit #microsoft #github #copilot machen. Und selbstverständlich wird ALLES #ai. Sogar unsere TLD wechselt von .net auf .ai.
Wir sollen ganz explizit doch bitte #ki in die tägliche #Arbeit einbinden, der Vertrieb soll sich beim. Aber bloß "unsere" nutzen, wegen den Daten. Muss mich gleich mal informieren, ob #localLLMs erlaubt sind.
https://hessen.social/@Moonstone2487/116082676681677166 -
Mein #arbeitgeber labert grade in so nem #MicrosoftTeams Call für alle Mitarbeiter was von #digitalesouveranitat und dann soll ich MEHR mit #microsoft #github #copilot machen. Und selbstverständlich wird ALLES #ai. Sogar unsere TLD wechselt von .net auf .ai.
Wir sollen ganz explizit doch bitte #ki in die tägliche #Arbeit einbinden, der Vertrieb soll sich beim. Aber bloß "unsere" nutzen, wegen den Daten. Muss mich gleich mal informieren, ob #localLLMs erlaubt sind.
https://hessen.social/@Moonstone2487/116082676681677166 -
Mein #arbeitgeber labert grade in so nem #MicrosoftTeams Call für alle Mitarbeiter was von #digitalesouveranitat und dann soll ich MEHR mit #microsoft #github #copilot machen. Und selbstverständlich wird ALLES #ai. Sogar unsere TLD wechselt von .net auf .ai.
Wir sollen ganz explizit doch bitte #ki in die tägliche #Arbeit einbinden, der Vertrieb soll sich beim. Aber bloß "unsere" nutzen, wegen den Daten. Muss mich gleich mal informieren, ob #localLLMs erlaubt sind.
https://hessen.social/@Moonstone2487/116082676681677166 -
Mein #arbeitgeber labert grade in so nem #MicrosoftTeams Call für alle Mitarbeiter was von #digitalesouveranitat und dann soll ich MEHR mit #microsoft #github #copilot machen. Und selbstverständlich wird ALLES #ai. Sogar unsere TLD wechselt von .net auf .ai.
Wir sollen ganz explizit doch bitte #ki in die tägliche #Arbeit einbinden, der Vertrieb soll sich beim. Aber bloß "unsere" nutzen, wegen den Daten. Muss mich gleich mal informieren, ob #localLLMs erlaubt sind.
https://hessen.social/@Moonstone2487/116082676681677166 -
Mini PC for local LLMs in 2026
-
Mini PC for local LLMs in 2026
-
Mini PC for local LLMs in 2026
-
Mini PC for local LLMs in 2026
-
Mini PC for local LLMs in 2026
-
Running Local LLMs Offline on a Ten-Hour Flight
https://deploy.live/blog/running-local-llms-offline-on-a-ten-hour-flight/
#HackerNews #LocalLLMs #Offline #Flight #Technology #AIApplications #TravelTech
-
Running Local LLMs Offline on a Ten-Hour Flight
https://deploy.live/blog/running-local-llms-offline-on-a-ten-hour-flight/
#HackerNews #LocalLLMs #Offline #Flight #Technology #AIApplications #TravelTech
-
Running Local LLMs Offline on a Ten-Hour Flight
https://deploy.live/blog/running-local-llms-offline-on-a-ten-hour-flight/
#HackerNews #LocalLLMs #Offline #Flight #Technology #AIApplications #TravelTech
-
Running Local LLMs Offline on a Ten-Hour Flight
https://deploy.live/blog/running-local-llms-offline-on-a-ten-hour-flight/
#HackerNews #LocalLLMs #Offline #Flight #Technology #AIApplications #TravelTech
-
Running Local LLMs Offline on a Ten-Hour Flight
https://deploy.live/blog/running-local-llms-offline-on-a-ten-hour-flight/
#HackerNews #LocalLLMs #Offline #Flight #Technology #AIApplications #TravelTech
