home.social

#wllama — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #wllama, aggregated by home.social.

fetched live
  1. Thomas @[email protected] ·

    New week, new slides: Run LLMs Locally

    Added OpenAI's Privacy-Filter model using privacy-filter.cpp.
    New slides about economics and hardware used by the Hugging Face community.

    codeberg.org/thbley/talks/raw/

    #ai #llm #llamacpp #wllama #stablediffusion #qwen #localai #gemma4 #webgpu #opencode #huggingface #privacy #digitalsovereignty

    #ai #llm #llamacpp #wllama #stablediffusion #qwen
  2. Thomas @[email protected] ·

    New week, new slides: Run LLMs Locally

    Added OpenAI's Privacy-Filter model using privacy-filter.cpp.
    New slides about economics and hardware used by the Hugging Face community.

    codeberg.org/thbley/talks/raw/

    #ai #llm #llamacpp #wllama #stablediffusion #qwen #localai #gemma4 #webgpu #opencode #huggingface #privacy #digitalsovereignty

    #ai #llm #llamacpp #wllama #stablediffusion #qwen
  3. Thomas @[email protected] ·

    After speaking at MLCon in Munich, I'm happy to announce my next talks about "Run LLMs locally":

    Agentic Web Week Munich, October 2026

    AI Native Software Week Berlin, November 2026

    #localai #llamacpp #wllama #webgpu #opencode

    #localai #llamacpp #wllama #webgpu #opencode
  4. Thomas @[email protected] ·

    Update, more slides: Run LLMs Locally

    I added sandboxing of OpenCode and llama.cpp with nono and Landlock.
    And a new slide to describe jailbreaks with DeepInception.

    codeberg.org/thbley/talks/raw/

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3 #glm #localai #gemma4 #webgpu #opencode

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3
  5. Thomas @[email protected] ·

    Update, more slides: Run LLMs Locally

    I added sandboxing of OpenCode and llama.cpp with nono and Landlock.
    And a new slide to describe jailbreaks with DeepInception.

    codeberg.org/thbley/talks/raw/

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3 #glm #localai #gemma4 #webgpu #opencode

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3
  6. Thomas @[email protected] ·

    New week, new slides: Run LLMs Locally

    I added virtualization of OpenCode with Matchlock and Firecracker microVMs,
    containerization of OpenCode and llama.cpp with Docker
    and a new slide for indirect prompt injection attacks.
    Matchlock is a great project for sandboxing, bringing the advantages of containers to virtual machines.

    codeberg.org/thbley/talks/raw/

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3 #glm #localai #gemma4 #webgpu #opencode #firecracker #docker

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3
  7. Thomas @[email protected] ·

    New week, new slides: Run LLMs Locally

    I added virtualization of OpenCode with Matchlock and Firecracker microVMs,
    containerization of OpenCode and llama.cpp with Docker
    and a new slide for indirect prompt injection attacks.
    Matchlock is a great project for sandboxing, bringing the advantages of containers to virtual machines.

    codeberg.org/thbley/talks/raw/

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3 #glm #localai #gemma4 #webgpu #opencode #firecracker #docker

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3
  8. Thomas @[email protected] ·

    New week, new slides and small updates: Run LLMs Locally

    Added an example to create Mermaid diagrams in llama.cpp UI.
    Added QAT (Quantization-Aware Training) variants of Gemma 4 which are 50 percent faster in token generation with my local setup.
    Added definitions for Deterministic and Probabilistic results.

    codeberg.org/thbley/talks/raw/

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3 #glm #localai #gemma4 #webgpu #opencode #mtp #webassembly #mellum2

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3
  9. Thomas @[email protected] ·

    New week, new slides and small updates: Run LLMs Locally

    Added an example to create Mermaid diagrams in llama.cpp UI.
    Added QAT (Quantization-Aware Training) variants of Gemma 4 which are 50 percent faster in token generation with my local setup.
    Added definitions for Deterministic and Probabilistic results.

    codeberg.org/thbley/talks/raw/

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3 #glm #localai #gemma4 #webgpu #opencode #mtp #webassembly #mellum2

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3
  10. Thomas @[email protected] ·

    New week, beautiful new slides: Run LLMs Locally

    Now with Mellum2 from JetBrains!
    A very fast coding model, requires only 10 GB RAM.

    I also added LFM 2.5 from LiquidAI, updated translations with HY-MT2 from Tencent, added examples for wllama using re-ranking and structured output
    and added thinking_budget_tokens to the curl examples.

    codeberg.org/thbley/talks/raw/

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3 #glm #localai #gemma4 #webgpu #opencode #mtp #webassembly #jetbrains #mellum2

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3
  11. Thomas @[email protected] ·

    New week, beautiful new slides: Run LLMs Locally

    Now with Mellum2 from JetBrains!
    A very fast coding model, requires only 10 GB RAM.

    I also added LFM 2.5 from LiquidAI, updated translations with HY-MT2 from Tencent, added examples for wllama using re-ranking and structured output
    and added thinking_budget_tokens to the curl examples.

    codeberg.org/thbley/talks/raw/

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3 #glm #localai #gemma4 #webgpu #opencode #mtp #webassembly #jetbrains #mellum2

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3
  12. Thomas @[email protected] ·

    New week, more slides: Run LLMs Locally

    Now including wllama to run GGUF models inside your browser!

    wllama uses llama.cpp, WebAssembly and WebGPU, bringing a completely new experience of LLMs into the web.
    It has no 4 GB limitation and is faster than Transformers.js.

    I also added translations using the HY-MT model from Tencent.

    codeberg.org/thbley/talks/raw/

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3 #glm #localai #gemma4 #webgpu #opencode #mtp #webassembly

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3
  13. Thomas @[email protected] ·

    New week, more slides: Run LLMs Locally

    Now including wllama to run GGUF models inside your browser!

    wllama uses llama.cpp, WebAssembly and WebGPU, bringing a completely new experience of LLMs into the web.
    It has no 4 GB limitation and is faster than Transformers.js.

    I also added translations using the HY-MT model from Tencent.

    codeberg.org/thbley/talks/raw/

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3 #glm #localai #gemma4 #webgpu #opencode #mtp #webassembly

    #ai #llm #llamacpp #wllama #stablediffusion #qwen3
  14. Habr @[email protected] ·

    Некорпоративный Хабр: семантический поиск и фильтрация по структурированным полям

    Классический RAG индексирует исходный текст документа, предварительно разбивая на фрагменты. Потом рассчитывает векторное представление фрагментов и сохраняет их векторные представления в базу данных для поиска. Это дает возможность искать по сходству фрагментов текста и поискового запроса пользователя, но не дает возможность искать по более высокоуровневым резюме и смыслам, темам поднятым в тексте и прочему. Также не помогает с аналитикой по содержимому. Бесплатный проект text-metadata-generator позволяет выполнять запросы к LLM по каждому документу из коллекции документов, результаты вывода LLM проверяются по JSON схеме. Зачем может пригодиться эта программа и подход со структурированием текстовой информации своя библиотека с каталогом - поиск по локальным документам с использованием комбинации SQL предикатов и семантического поиска аналитика по документам, возможность находить новое в текстах: комбинируя структурированные поля созданные LLM из исходного текста, и находя закономерности с уже существующими в документе метаданными. Например, связывая с рейтингом признак NSFW, тон повествования, полноту содержания итп. разгрести “авгиевы конюшни” личных заметок в Obsidian или git репозитарии с Markdown файлами Рассмотрим как работает данный подход на 13275 статьях с Хабра, а также текстах трех песнен…

    habr.com/ru/articles/1036594/

    #семантический_поиск #структурирование_инфомации #локальный_поиск_по_документам #эмбеддинги #duckdb_wasm #wllama #llmприложения #sql #rag #llm

    #llm #rag #sql #llmприложения #wllama #duckdb_wasm
  15. Habr @[email protected] ·

    HalChatLocalAI: как я встроил офлайн-ИИ прямо в мессенджер

    Большинство ИИ-ассистентов работают в облаке. А я сделал локальный — прямо внутри мессенджера HalChat .

    habr.com/ru/articles/966888/

    #ИИ #Локальный_ИИ #HalChat #HalSM #WLLama #LLM #приватность #javascript #чат_бот #экосистема

    #экосистема #чат_бот #javascript #приватность #llm #wllama
  16. Alessio Pomaro @[email protected] ·

    🧠 #Wllama è un progetto che permette di eseguire l'inferenza del modello di linguaggio direttamente nel browser.
    👉 Senza backend e senza #GPU. Il tutto basato su #WebAssembly SIMD.
    ⚙️ Con tutte le limitazioni del caso, ma per certi tipi di applicazioni semplici è un approccio davvero interessante.
    🔗 Il progetto: github.com/ngxson/wllama

    ___
    ✉️ 𝗦𝗲 𝘃𝘂𝗼𝗶 𝗿𝗶𝗺𝗮𝗻𝗲𝗿𝗲 𝗮𝗴𝗴𝗶𝗼𝗿𝗻𝗮𝘁𝗼/𝗮 𝘀𝘂 𝗾𝘂𝗲𝘀𝘁𝗲 𝘁𝗲𝗺𝗮𝘁𝗶𝗰𝗵𝗲, 𝗶𝘀𝗰𝗿𝗶𝘃𝗶𝘁𝗶 𝗮𝗹𝗹𝗮 𝗺𝗶𝗮 𝗻𝗲𝘄𝘀𝗹𝗲𝘁𝘁𝗲𝗿: bit.ly/newsletter-alessiopomar

    #AI #GenAI #GenerativeAI

    #wllama #gpu #webassembly #ai #genai #generativeai
  17. Alessio Pomaro @[email protected] ·

    🧠 #Wllama è un progetto che permette di eseguire l'inferenza del modello di linguaggio direttamente nel browser.
    👉 Senza backend e senza #GPU. Il tutto basato su #WebAssembly SIMD.
    ⚙️ Con tutte le limitazioni del caso, ma per certi tipi di applicazioni semplici è un approccio davvero interessante.
    🔗 Il progetto: github.com/ngxson/wllama

    ___
    ✉️ 𝗦𝗲 𝘃𝘂𝗼𝗶 𝗿𝗶𝗺𝗮𝗻𝗲𝗿𝗲 𝗮𝗴𝗴𝗶𝗼𝗿𝗻𝗮𝘁𝗼/𝗮 𝘀𝘂 𝗾𝘂𝗲𝘀𝘁𝗲 𝘁𝗲𝗺𝗮𝘁𝗶𝗰𝗵𝗲, 𝗶𝘀𝗰𝗿𝗶𝘃𝗶𝘁𝗶 𝗮𝗹𝗹𝗮 𝗺𝗶𝗮 𝗻𝗲𝘄𝘀𝗹𝗲𝘁𝘁𝗲𝗿: bit.ly/newsletter-alessiopomar

    #AI #GenAI #GenerativeAI

    #wllama #gpu #webassembly #ai #genai #generativeai