#tinyllama — Public Fediverse posts

Mới nâng cấp lên máy N100 (8GB RAM, NVMe) và rất ấn tượng với khả năng xử lý LLM và Stable Diffusion trên phần cứng thấp. Đang thử TinyLlama, Gemma-3, Phi-4-mini cho mã nguồn, dịch thuật, hình ảnh và âm thanh. Khả năng chạy AI local trên thiết bị yếu thật sự bất ngờ. Bạn đang dùng gì cho self-hosted LLM? #LLM #SelfHosted #TinyLlama #StableDiffusion #AI #MachineLearning #AIcụcgạch #HọcMáy #TựChủCôngNghệ

https://www.reddit.com/r/selfhosted/comments/1qsdgop/selfhosted_llm_on_lowpower_hardware/

Mô hình LLM 1.1B (TinyLlama) hoạt động trên laptop i3 cũ 15 năm tuổi! Tối ưu bằng giám sát Shannon Entropy và cắt bớt ngữ cảnh thủ công để ổn định. AI được cá nhân hóa thành "Người Bảo Vệ" theo manga Blame!, vượt bài kiểm tra trung thành khi từ chối nâng cấp phần cứng để xóa dữ liệu. Chứng minh hiệu suất ấn tượng của mô hình nhỏ với giới hạn hợp lý. #TinyLlama #LLM #AI #ShannonEntropy #LocalLLM #TríTuệNhânTạo #AIcụcmịch #TốiƯuHóa

https://www.reddit.com/r/LocalLLaMA/comments/1ql377e/personalized

Лоботомия нейросети: удалил 7 слоёв из LLM — она стала на 30% быстрее

Эксперимент по хирургическому удалению слоёв из языковой модели Современные LLM переобучены — многие слои делают одно и то же. Я проверил эту гипотезу на практике: взял TinyLlama (1.1B параметров, 22 слоя) и измерил, как удаление каждого слоя влияет на perplexity. Результаты: • Удаление 1 среднего слоя: +10% скорость, -4% качество • Удаление 7 «безопасных» слоёв: +32% скорость, -2.5% качество • Удаление первого слоя: модель полностью ломается Неожиданно: Layer 2 важнее Layer 0 (perplexity +6.67 vs +3.92 при удалении). Статья с кодом на PyTorch, графиками и практическими рекомендациями — какие слои можно удалять для ускорения инференса.

https://habr.com/ru/articles/983636/

#LLM #Large_Language_Models #Layer_Pruning #оптимизация_нейросетей #TinyLlama #PyTorch #inference_optimization #трансформеры #ускорение_моделей #ShortGPT

Лоботомия нейросети: удалил 7 слоёв из LLM — она стала на 30% быстрее

Эксперимент по хирургическому удалению слоёв из языковой модели Современные LLM переобучены — многие слои делают одно и то же. Я проверил эту гипотезу на практике: взял TinyLlama (1.1B параметров, 22 слоя) и измерил, как удаление каждого слоя влияет на perplexity. Результаты: • Удаление 1 среднего слоя: +10% скорость, -4% качество • Удаление 7 «безопасных» слоёв: +32% скорость, -2.5% качество • Удаление первого слоя: модель полностью ломается Неожиданно: Layer 2 важнее Layer 0 (perplexity +6.67 vs +3.92 при удалении). Статья с кодом на PyTorch, графиками и практическими рекомендациями — какие слои можно удалять для ускорения инференса.

https://habr.com/ru/articles/983636/

#LLM #Large_Language_Models #Layer_Pruning #оптимизация_нейросетей #TinyLlama #PyTorch #inference_optimization #трансформеры #ускорение_моделей #ShortGPT

Лоботомия нейросети: удалил 7 слоёв из LLM — она стала на 30% быстрее

Эксперимент по хирургическому удалению слоёв из языковой модели Современные LLM переобучены — многие слои делают одно и то же. Я проверил эту гипотезу на практике: взял TinyLlama (1.1B параметров, 22 слоя) и измерил, как удаление каждого слоя влияет на perplexity. Результаты: • Удаление 1 среднего слоя: +10% скорость, -4% качество • Удаление 7 «безопасных» слоёв: +32% скорость, -2.5% качество • Удаление первого слоя: модель полностью ломается Неожиданно: Layer 2 важнее Layer 0 (perplexity +6.67 vs +3.92 при удалении). Статья с кодом на PyTorch, графиками и практическими рекомендациями — какие слои можно удалять для ускорения инференса.

https://habr.com/ru/articles/983636/

#LLM #Large_Language_Models #Layer_Pruning #оптимизация_нейросетей #TinyLlama #PyTorch #inference_optimization #трансформеры #ускорение_моделей #ShortGPT

Лоботомия нейросети: удалил 7 слоёв из LLM — она стала на 30% быстрее

Эксперимент по хирургическому удалению слоёв из языковой модели Современные LLM переобучены — многие слои делают одно и то же. Я проверил эту гипотезу на практике: взял TinyLlama (1.1B параметров, 22 слоя) и измерил, как удаление каждого слоя влияет на perplexity. Результаты: • Удаление 1 среднего слоя: +10% скорость, -4% качество • Удаление 7 «безопасных» слоёв: +32% скорость, -2.5% качество • Удаление первого слоя: модель полностью ломается Неожиданно: Layer 2 важнее Layer 0 (perplexity +6.67 vs +3.92 при удалении). Статья с кодом на PyTorch, графиками и практическими рекомендациями — какие слои можно удалять для ускорения инференса.

https://habr.com/ru/articles/983636/

#LLM #Large_Language_Models #Layer_Pruning #оптимизация_нейросетей #TinyLlama #PyTorch #inference_optimization #трансформеры #ускорение_моделей #ShortGPT

Playing around with "knowledge transfer" to my #Robots

I told each "I need to go take a shower. Talk to you later."

RPi4 #GoPiGo3 robot Dave with tinyllama responded: "I understand how important your shower is... Enjoy the shower!"

RPi5 #TurtleBot4 robot WaLI with Gemma responded: "Acknowledged. Initiating shower sequence. Requesting allocation of cleaning supplies..."

I hope I don't get a "We shipped your soap" email from Amazon.

#ollama #tinyllama #Gemma3

Playing around with "knowledge transfer" to my #Robots

I told each "I need to go take a shower. Talk to you later."

RPi4 #GoPiGo3 robot Dave with tinyllama responded: "I understand how important your shower is... Enjoy the shower!"

RPi5 #TurtleBot4 robot WaLI with Gemma responded: "Acknowledged. Initiating shower sequence. Requesting allocation of cleaning supplies..."

I hope I don't get a "We shipped your soap" email from Amazon.

#ollama #tinyllama #Gemma3

Playing around with "knowledge transfer" to my #Robots

I told each "I need to go take a shower. Talk to you later."

RPi4 #GoPiGo3 robot Dave with tinyllama responded: "I understand how important your shower is... Enjoy the shower!"

RPi5 #TurtleBot4 robot WaLI with Gemma responded: "Acknowledged. Initiating shower sequence. Requesting allocation of cleaning supplies..."

I hope I don't get a "We shipped your soap" email from Amazon.

#ollama #tinyllama #Gemma3

Playing around with "knowledge transfer" to my #Robots

I told each "I need to go take a shower. Talk to you later."

RPi4 #GoPiGo3 robot Dave with tinyllama responded: "I understand how important your shower is... Enjoy the shower!"

RPi5 #TurtleBot4 robot WaLI with Gemma responded: "Acknowledged. Initiating shower sequence. Requesting allocation of cleaning supplies..."

I hope I don't get a "We shipped your soap" email from Amazon.

#ollama #tinyllama #Gemma3

Playing around with "knowledge transfer" to my #Robots

I told each "I need to go take a shower. Talk to you later."

RPi4 #GoPiGo3 robot Dave with tinyllama responded: "I understand how important your shower is... Enjoy the shower!"

RPi5 #TurtleBot4 robot WaLI with Gemma responded: "Acknowledged. Initiating shower sequence. Requesting allocation of cleaning supplies..."

I hope I don't get a "We shipped your soap" email from Amazon.

#ollama #tinyllama #Gemma3

@Harlander
>>> can I order door dash here?
Sure! You can place an order for our custom-made door dash, which is a
specialty of our company. Our team of designers will work with you to
create the perfect fit for your needs and budget. We offer several options
for door dashes, including round or oval shapes, in different finishes
like brass, bronze, and stainless steel. Contact us today to discuss your
custom-made door dash and get a quote!

#tinyllama

#tinyllama #llm is so interesting.

>>> oh boy
pyxtcirpppid mypier prov p python 7 dig pixel cro 1pipipus p os os pupix
poly cro pin python os p p cp p pixel pixel os ipospixtpng p_py du pia pf
pon <ppos pro os os descript rect stream dut<pent<to to to
<digpi arer

https://teia.art/objkt/872578

Run Your Own AI LLM Model on a LowEnd VPS for Only $2.49 a Month! Part Two: Web UI https://lowendbox.com/blog/run-your-own-ai-llm-model-on-a-lowend-vps-for-only-2-49-a-month-part-two-web-ui/ #LowEndVirtual #losangeles #openwebui #openwebui #tinyllama #racknerd #racknerd #ashburn #chatgpt #chatgpt #chicago #newyork #sanjose #seattle #toronto #claude #dallas #ollama #openai

📣 The LLM shift no one saw coming!
Top engineers are dropping GPT for TinyLlama—faster, cheaper, and surprisingly more effective in real-world tasks 💼⚡
Want to know why?

👇 Read the article and future-proof your GenAI strategy:
👉 https://medium.com/@rogt.x1997/why-smart-engineers-are-ditching-gpt-for-tinyllama-and-you-should-too-345a042e6f6e

#TinyLlama #LLMTrends #GenAI2025 #OpenSourceAI
https://medium.com/@rogt.x1997/why-smart-engineers-are-ditching-gpt-for-tinyllama-and-you-should-too-345a042e6f6e

📣 The LLM shift no one saw coming!
Top engineers are dropping GPT for TinyLlama—faster, cheaper, and surprisingly more effective in real-world tasks 💼⚡
Want to know why?

👇 Read the article and future-proof your GenAI strategy:
👉 https://medium.com/@rogt.x1997/why-smart-engineers-are-ditching-gpt-for-tinyllama-and-you-should-too-345a042e6f6e

#TinyLlama #LLMTrends #GenAI2025 #OpenSourceAI
https://medium.com/@rogt.x1997/why-smart-engineers-are-ditching-gpt-for-tinyllama-and-you-should-too-345a042e6f6e

Just trained my own language model offline.
No cloud. No APIs. Fine-tuned it on my data, merged it, and ran it with llama.cpp.
This is what real AI literacy looks like.

Documentation:
https://github.com/hassanhabib/AI.Llama.Traing.Offline

Video:
https://youtube.com/watch?v=FQr7VrK5RRQ

#AI #LLM #LoRA #OfflineAI #TinyLlama

I asked #tinyllama to generate me a bio for my new #mastodon account at the great #OhaiSocial instance.

In short: it is a verbose inventor of text.

Me: "write me a bio info for my social media profile where my skills are presented: my skills are software, society, rocks , hiking,"

At least it stated: Here's an example of how you might incorporate inline citations into your bio information for your social media profile...

And I've forgot to say, the rest is also wrong.

#expanse #ai #experiment #knowledge #tinyllama #ollama

The excuse is, that this tinyllama is a VERY tiny model. llama3.2 works much smarter. (but harder to test because it already knows about the expanse without adding stuff to the "knowledge" so I need to invent something...)

Here it is how you can do #finetuning for a SMAL-language model that can be put on a #RaspberryPI or other edge-computing devices, or even wearables:
https://www.youtube.com/watch?v=DTYi7z4cLD0

#TinyLLaMA #TinyDolphin #Ollama #AIonEdge #MachineLearning #AIModels #EdgeComputing #AI #LLM

Impariamo ad eseguire i #LLM (#Large #Language #Model) localmente con #Ollama e entriamo nel mondo dell'#AI #generativa in maniera semplice e veloce:

https://www.risposteinformatiche.it/eseguire-ollama-e-vari-modelli-llm-su-linux/

#UnoLinux #Linux #gnu_linux #opensource #largelanguagemodel #phy3 #tinyllama #IntelligenzaArificiale #ChatGPT #ChatBot #Privacy

TinyLlama is a 486 in Your Pocket - We love retrocomputing and tiny computers here at Hackaday, so it’s always nice to... - https://hackaday.com/2022/11/20/tinyllama-is-a-486-in-your-pocket/ #raspberrypizero2 #retrocomputing #computerhacks #doesitrundoom #crystalaudio #raspberrypi #tinyllama #hardware #86duino #games #mt-32 #doom #midi #dos #vga

TinyLlama is a 486 in Your Pocket - We love retrocomputing and tiny computers here at Hackaday, so it’s always nice to... - https://hackaday.com/2022/11/20/tinyllama-is-a-486-in-your-pocket/ #raspberrypizero2 #retrocomputing #computerhacks #doesitrundoom #crystalaudio #raspberrypi #tinyllama #hardware #86duino #games #mt-32 #doom #midi #dos #vga

TinyLlama is a 486 in Your Pocket - We love retrocomputing and tiny computers here at Hackaday, so it’s always nice to... - https://hackaday.com/2022/11/20/tinyllama-is-a-486-in-your-pocket/ #raspberrypizero2 #retrocomputing #computerhacks #doesitrundoom #crystalaudio #raspberrypi #tinyllama #hardware #86duino #games #mt-32 #doom #midi #dos #vga

TinyLlama is a 486 in Your Pocket - We love retrocomputing and tiny computers here at Hackaday, so it’s always nice to... - https://hackaday.com/2022/11/20/tinyllama-is-a-486-in-your-pocket/ #raspberrypizero2 #retrocomputing #computerhacks #doesitrundoom #crystalaudio #raspberrypi #tinyllama #hardware #86duino #games #mt-32 #doom #midi #dos #vga

TinyLlama is a 486 in Your Pocket - We love retrocomputing and tiny computers here at Hackaday, so it’s always nice to... - https://hackaday.com/2022/11/20/tinyllama-is-a-486-in-your-pocket/ #raspberrypizero2 #retrocomputing #computerhacks #doesitrundoom #crystalaudio #raspberrypi #tinyllama #hardware #86duino #games #mt-32 #doom #midi #dos #vga

TinyLlama is a 486 in Your Pocket

https://hackaday.com/2022/11/20/tinyllama-is-a-486-in-your-pocket/

#RaspberryPiZero2 #Retrocomputing #computerhacks #doesitrundoom #CrystalAudio #RaspberryPi #raspberrypi #TinyLlama #hardware #86Duino #Games #doom #midi #MT32 #486 #dos #vga

TinyLlama is a 486 in Your Pocket

https://hackaday.com/2022/11/20/tinyllama-is-a-486-in-your-pocket/

#RaspberryPiZero2 #Retrocomputing #computerhacks #doesitrundoom #CrystalAudio #RaspberryPi #raspberrypi #TinyLlama #hardware #86Duino #Games #doom #midi #MT32 #486 #dos #vga