#ollama — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #ollama, aggregated by home.social.
-
Сколько токенов в секунду выдадут новые Mac mini на M6 и M5 Pro. Прогноз до того, как железо приехало
Через несколько дней Apple начнёт отгружать Mac mini на M6 и M5 Pro и Mac Studio на M5 Max и M5 Ultra. Замеров локальных моделей на них ещё ни у кого нет, а прикинуть скорость можно уже сейчас. В прошлой статье я замерил шесть моделей на базовом M4 и вывел правило через полосу памяти с КПД 0.83, которое сошлось с замером в пределах двух процентов. Здесь применяю его к новым чипам и публикую цифры до того, как железо приехало. Потом замерю и покажу, где ошибся. Коротко. M6 для локальных моделей не апгрейд, прирост 25 до 40 процентов и тот же потолок 8B. M5 Pro первый mini, где 32B работает на рабочей скорости. M5 Max меняет класс, 70B на 13 токенах в секунду. Отдельно про ловушки младших конфигураций, у M6 с 16 ГБ и M5 Max с 36 ГБ полоса памяти заметно ниже.
https://habr.com/ru/articles/1083322/
#mac_mini #m6 #m5_pro #ollama #llm #apple_silicon #локальные_модели #бенчмарк
-
🏠 Open WebUI: ChatGPT UX on your own infrastructure!
Docker, RAG, multi-user auth, any backend behind one UI.
📖 Read: https://devopstales.github.io/ai/open-webui-self-hosted/?utm_source=mastodon&utm_medium=social
-
🏠 Open WebUI: ChatGPT UX on your own infrastructure!
Docker, RAG, multi-user auth, any backend behind one UI.
📖 Read: https://devopstales.github.io/ai/open-webui-self-hosted/?utm_source=mastodon&utm_medium=social
-
🏠 Open WebUI: ChatGPT UX on your own infrastructure!
Docker, RAG, multi-user auth, any backend behind one UI.
📖 Read: https://devopstales.github.io/ai/open-webui-self-hosted/?utm_source=mastodon&utm_medium=social
-
🏠 Open WebUI: ChatGPT UX on your own infrastructure!
Docker, RAG, multi-user auth, any backend behind one UI.
📖 Read: https://devopstales.github.io/ai/open-webui-self-hosted/?utm_source=mastodon&utm_medium=social
-
🏠 Open WebUI: ChatGPT UX on your own infrastructure!
Docker, RAG, multi-user auth, any backend behind one UI.
📖 Read: https://devopstales.github.io/ai/open-webui-self-hosted/?utm_source=mastodon&utm_medium=social
-
My harness, or whatever that means
That's one of those words that, one year ago without context in a question, it could get you off guard. Today it appears to be pretty common in conversations. Of course I'm talking about the tools one uses to interact with LLMs and manage agent execution. Even though in the past I've written about my preference for local AI and even shared my setup at the time, I don't give that much usage to any AI apart from using it as a coding assistant. I see value in things like asking it to review […] -
My harness, or whatever that means
That's one of those words that, one year ago without context in a question, it could get you off guard. Today it appears to be pretty common in conversations. Of course I'm talking about the tools one uses to interact with LLMs and manage agent execution. Even though in the past I've written about my preference for local AI and even shared my setup at the time, I don't give that much usage to any AI apart from using it as a coding assistant. I see value in things like asking it to review […] -
My harness, or whatever that means
That's one of those words that, one year ago without context in a question, it could get you off guard. Today it appears to be pretty common in conversations. Of course I'm talking about the tools one uses to interact with LLMs and manage agent execution. Even though in the past I've written about my preference for local AI and even shared my setup at the time, I don't give that much usage to any AI apart from using it as a coding assistant. I see value in things like asking it to review […] -
My harness, or whatever that means
That's one of those words that, one year ago without context in a question, it could get you off guard. Today it appears to be pretty common in conversations. Of course I'm talking about the tools one uses to interact with LLMs and manage agent execution. Even though in the past I've written about my preference for local AI and even shared my setup at the time, I don't give that much usage to any AI apart from using it as a coding assistant. I see value in things like asking it to review […] -
My harness, or whatever that means
That's one of those words that, one year ago without context in a question, it could get you off guard. Today it appears to be pretty common in conversations. Of course I'm talking about the tools one uses to interact with LLMs and manage agent execution. Even though in the past I've written about my preference for local AI and even shared my setup at the time, I don't give that much usage to any AI apart from using it as a coding assistant. I see value in things like asking it to review […]