#runpod — Public Fediverse posts

🚀 Running #OCR at scale with a #Vision #LLM for $0.49/hour

Just deployed dots.ocr (3B parameter Vision LLM by RedNote) on a single #RTX A6000 (48GB VRAM) via #RunPod. The results are great:

https://github.com/rednote-hilab/dots.ocr

#ai #opensource

📄 The Setup
- Upload any #PDF → server converts each page to an image (PyMuPDF)
- Images are sent in parallel to #vLLM (continuous batching)
- The Vision LLM reads each page and returns clean Markdown

🧵 👇

🚀 Running #OCR at scale with a #Vision #LLM for $0.49/hour

Just deployed dots.ocr (3B parameter Vision LLM by RedNote) on a single #RTX A6000 (48GB VRAM) via #RunPod. The results are great:

https://github.com/rednote-hilab/dots.ocr

#ai #opensource

📄 The Setup
- Upload any #PDF → server converts each page to an image (PyMuPDF)
- Images are sent in parallel to #vLLM (continuous batching)
- The Vision LLM reads each page and returns clean Markdown

🧵 👇

🚀 Running #OCR at scale with a #Vision #LLM for $0.49/hour

Just deployed dots.ocr (3B parameter Vision LLM by RedNote) on a single #RTX A6000 (48GB VRAM) via #RunPod. The results are great:

https://github.com/rednote-hilab/dots.ocr

#ai #opensource

📄 The Setup
- Upload any #PDF → server converts each page to an image (PyMuPDF)
- Images are sent in parallel to #vLLM (continuous batching)
- The Vision LLM reads each page and returns clean Markdown

🧵 👇

🚀 Running #OCR at scale with a #Vision #LLM for $0.49/hour

Just deployed dots.ocr (3B parameter Vision LLM by RedNote) on a single #RTX A6000 (48GB VRAM) via #RunPod. The results are great:

https://github.com/rednote-hilab/dots.ocr

#ai #opensource

📄 The Setup
- Upload any #PDF → server converts each page to an image (PyMuPDF)
- Images are sent in parallel to #vLLM (continuous batching)
- The Vision LLM reads each page and returns clean Markdown

🧵 👇

🚀 Running #OCR at scale with a #Vision #LLM for $0.49/hour

Just deployed dots.ocr (3B parameter Vision LLM by RedNote) on a single #RTX A6000 (48GB VRAM) via #RunPod. The results are great:

https://github.com/rednote-hilab/dots.ocr

#ai #opensource

📄 The Setup
- Upload any #PDF → server converts each page to an image (PyMuPDF)
- Images are sent in parallel to #vLLM (continuous batching)
- The Vision LLM reads each page and returns clean Markdown

🧵 👇

So, #tech question targeting #cloudGaming as well as #selfHosting / #selfHosted:

I've been looking around at GPU-on-demand providers and came across a number of decent offerings, currently favouring #RunPod (https://runpod.io).

However, storage is as always the killing blow to any self-run, cloud-hosted game streaming setup. I can live with as low as 250gb but my budget frame is around $15/month max including GPU hours (we're talking about 15-20h/month max) -- outside of that and you end up in GFN territory which I'm not going to pay.

Talking to some friends last night got me on an interesting track though, what if you host a RunPod with only session storage (which AFAIK grows dynamically at no cost but is also lost once the system is shut down) and backup / restore that session storage on-the-fly to a provider like #Wasabi?

I checked a few providers in that regard and Wasabi seems to be the only one to have no egress fees as long as you play fair. Big question being is "download 250gb about 4-6 times a month in one go" still fair?

Are there alternatives?

I'm all ears..

And no, "get a gaming PC" doesn't cut it - for one I want to be able to stream my games via moonlight and parsec and for another the price of even a low-end gaming rig to use for this covers the cost of a cloud-hosted environment for the next decade.

So, #tech question targeting #cloudGaming as well as #selfHosting / #selfHosted:

I've been looking around at GPU-on-demand providers and came across a number of decent offerings, currently favouring #RunPod (https://runpod.io).

However, storage is as always the killing blow to any self-run, cloud-hosted game streaming setup. I can live with as low as 250gb but my budget frame is around $15/month max including GPU hours (we're talking about 15-20h/month max) -- outside of that and you end up in GFN territory which I'm not going to pay.

Talking to some friends last night got me on an interesting track though, what if you host a RunPod with only session storage (which AFAIK grows dynamically at no cost but is also lost once the system is shut down) and backup / restore that session storage on-the-fly to a provider like #Wasabi?

I checked a few providers in that regard and Wasabi seems to be the only one to have no egress fees as long as you play fair. Big question being is "download 250gb about 4-6 times a month in one go" still fair?

Are there alternatives?

I'm all ears..

And no, "get a gaming PC" doesn't cut it - for one I want to be able to stream my games via moonlight and parsec and for another the price of even a low-end gaming rig to use for this covers the cost of a cloud-hosted environment for the next decade.

So, #tech question targeting #cloudGaming as well as #selfHosting / #selfHosted:

I've been looking around at GPU-on-demand providers and came across a number of decent offerings, currently favouring #RunPod (https://runpod.io).

However, storage is as always the killing blow to any self-run, cloud-hosted game streaming setup. I can live with as low as 250gb but my budget frame is around $15/month max including GPU hours (we're talking about 15-20h/month max) -- outside of that and you end up in GFN territory which I'm not going to pay.

Talking to some friends last night got me on an interesting track though, what if you host a RunPod with only session storage (which AFAIK grows dynamically at no cost but is also lost once the system is shut down) and backup / restore that session storage on-the-fly to a provider like #Wasabi?

I checked a few providers in that regard and Wasabi seems to be the only one to have no egress fees as long as you play fair. Big question being is "download 250gb about 4-6 times a month in one go" still fair?

Are there alternatives?

I'm all ears..

And no, "get a gaming PC" doesn't cut it - for one I want to be able to stream my games via moonlight and parsec and for another the price of even a low-end gaming rig to use for this covers the cost of a cloud-hosted environment for the next decade.

So, #tech question targeting #cloudGaming as well as #selfHosting / #selfHosted:

I've been looking around at GPU-on-demand providers and came across a number of decent offerings, currently favouring #RunPod (https://runpod.io).

However, storage is as always the killing blow to any self-run, cloud-hosted game streaming setup. I can live with as low as 250gb but my budget frame is around $15/month max including GPU hours (we're talking about 15-20h/month max) -- outside of that and you end up in GFN territory which I'm not going to pay.

Talking to some friends last night got me on an interesting track though, what if you host a RunPod with only session storage (which AFAIK grows dynamically at no cost but is also lost once the system is shut down) and backup / restore that session storage on-the-fly to a provider like #Wasabi?

I checked a few providers in that regard and Wasabi seems to be the only one to have no egress fees as long as you play fair. Big question being is "download 250gb about 4-6 times a month in one go" still fair?

Are there alternatives?

I'm all ears..

And no, "get a gaming PC" doesn't cut it - for one I want to be able to stream my games via moonlight and parsec and for another the price of even a low-end gaming rig to use for this covers the cost of a cloud-hosted environment for the next decade.

So, #tech question targeting #cloudGaming as well as #selfHosting / #selfHosted:

I've been looking around at GPU-on-demand providers and came across a number of decent offerings, currently favouring #RunPod (https://runpod.io).

However, storage is as always the killing blow to any self-run, cloud-hosted game streaming setup. I can live with as low as 250gb but my budget frame is around $15/month max including GPU hours (we're talking about 15-20h/month max) -- outside of that and you end up in GFN territory which I'm not going to pay.

Talking to some friends last night got me on an interesting track though, what if you host a RunPod with only session storage (which AFAIK grows dynamically at no cost but is also lost once the system is shut down) and backup / restore that session storage on-the-fly to a provider like #Wasabi?

I checked a few providers in that regard and Wasabi seems to be the only one to have no egress fees as long as you play fair. Big question being is "download 250gb about 4-6 times a month in one go" still fair?

Are there alternatives?

I'm all ears..

And no, "get a gaming PC" doesn't cut it - for one I want to be able to stream my games via moonlight and parsec and for another the price of even a low-end gaming rig to use for this covers the cost of a cloud-hosted environment for the next decade.

Runpod, khởi nghiệp AI cloud từ Reddit 2022, hôm nay đạt $120M ARR và 500k nhà phát triển. Nền tảng cung cấp GPU “gần như tại chỗ” với bảo mật, mở rộng serverless, API đơn giản, không hợp đồng dài hạn, hỗ trợ H100. Cảm ơn cộng đồng! #AI #Cloud #GPU #Startup #Runpod #CôngNghệ #KhởiNghiệp

https://www.reddit.com/r/LocalLLaMA/comments/1qib2ks/runpod_hits_120m_arr_four_years_after_launching/

https://www.europesays.com/uk/703561/ AI cloud startup Runpod hits $120M in ARR — and it started with a Reddit post   #AI #AIDataCenter #ArtificialIntelligence #Exclusive #runpod #Technology #UK #UnitedKingdom

AI cloud startup Runpod hits $120M in ARR — and it started with a Reddit post

Runpod, an AI app hosting platform that launched four years ago, has hit a $120 million annual revenue…
#NewsBeep #News #Artificialintelligence #AI #AIdatacenter #ArtificialIntelligence #Exclusive #runpod #Technology #UK #UnitedKingdom
https://www.newsbeep.com/uk/376228/

AI cloud startup Runpod hits $120M in ARR — and it started with a Reddit post

Runpod, an AI app hosting platform that launched four years ago, has hit a $120 million annual revenue…
#NewsBeep #News #Artificialintelligence #AI #AIdatacenter #ArtificialIntelligence #AU #Australia #Exclusive #runpod #Technology
https://www.newsbeep.com/au/418885/

90% of LLM fine-tuning fails.
Not because of models — but how we train them ⚠️

This breaks it down, simply 🧠⚡
https://medium.com/@rogt.x1997/why-90-of-llm-fine-tuning-fails-how-runpod-fixes-it-e951c9aa97af

#GenAI #FineTuning #RunPod
https://medium.com/@rogt.x1997/why-90-of-llm-fine-tuning-fails-how-runpod-fixes-it-e951c9aa97af

90% of LLM fine-tuning fails.
Not because of models — but how we train them ⚠️

This breaks it down, simply 🧠⚡
https://medium.com/@rogt.x1997/why-90-of-llm-fine-tuning-fails-how-runpod-fixes-it-e951c9aa97af

#GenAI #FineTuning #RunPod
https://medium.com/@rogt.x1997/why-90-of-llm-fine-tuning-fails-how-runpod-fixes-it-e951c9aa97af

Chạy Kyutai Unmute trên máy chủ Runpod L40s với 1 GPU. Dự án cho phép chạy trực tiếp trên thiết bị iOS thông qua kết nối WebRTC tương thích OpenAI. #KyutaiUnmute #Runpod #LLaMA #TríTuệNhânTạo #AI

https://www.reddit.com/r/LocalLLaMA/comments/1okvyud/run_kyutai_unmute_on_a_runpod_l40s_singlegpu/

🚀 Tired of burning weekends fixing infra? RunPod 2025 makes GPU deploys boring (in the best way). Pods, endpoints & MCP turn ideas into live projects faster than ever. ⚡

👉 Read the full guide:
https://medium.com/@rogt.x1997/pods-endpoints-and-a-smoother-future-the-hidden-simplicity-of-runpod-f9bace9e1a8c

#RunPod #AIInfra #AIBuilders
https://medium.com/@rogt.x1997/pods-endpoints-and-a-smoother-future-the-hidden-simplicity-of-runpod-f9bace9e1a8c

🚀 Tired of burning weekends fixing infra? RunPod 2025 makes GPU deploys boring (in the best way). Pods, endpoints & MCP turn ideas into live projects faster than ever. ⚡

👉 Read the full guide:
https://medium.com/@rogt.x1997/pods-endpoints-and-a-smoother-future-the-hidden-simplicity-of-runpod-f9bace9e1a8c

#RunPod #AIInfra #AIBuilders
https://medium.com/@rogt.x1997/pods-endpoints-and-a-smoother-future-the-hidden-simplicity-of-runpod-f9bace9e1a8c

🚀 Tired of burning weekends fixing infra? RunPod 2025 makes GPU deploys boring (in the best way). Pods, endpoints & MCP turn ideas into live projects faster than ever. ⚡

👉 Read the full guide:
https://medium.com/@rogt.x1997/pods-endpoints-and-a-smoother-future-the-hidden-simplicity-of-runpod-f9bace9e1a8c

#RunPod #AIInfra #AIBuilders
https://medium.com/@rogt.x1997/pods-endpoints-and-a-smoother-future-the-hidden-simplicity-of-runpod-f9bace9e1a8c

🚀 Tired of burning weekends fixing infra? RunPod 2025 makes GPU deploys boring (in the best way). Pods, endpoints & MCP turn ideas into live projects faster than ever. ⚡

👉 Read the full guide:
https://medium.com/@rogt.x1997/pods-endpoints-and-a-smoother-future-the-hidden-simplicity-of-runpod-f9bace9e1a8c

#RunPod #AIInfra #AIBuilders
https://medium.com/@rogt.x1997/pods-endpoints-and-a-smoother-future-the-hidden-simplicity-of-runpod-f9bace9e1a8c

🚨 Trained a GPT-style model for just $0.80 in 90 mins.
🤯 No GPU farm. No million-dollar lab. Just LoRA + RunPod magic.
This changes everything for indie devs, students & lean startups.
👇 Read the future of fine-tuning here:
https://medium.com/@rogt.x1997/lora-runpod-the-0-80-ai-revolution-you-cant-afford-to-ignore-c14c2ed857a9
#LoRA #AIRevolution #RunPod
https://medium.com/@rogt.x1997/lora-runpod-the-0-80-ai-revolution-you-cant-afford-to-ignore-c14c2ed857a9

🚨 Trained a GPT-style model for just $0.80 in 90 mins.
🤯 No GPU farm. No million-dollar lab. Just LoRA + RunPod magic.
This changes everything for indie devs, students & lean startups.
👇 Read the future of fine-tuning here:
https://medium.com/@rogt.x1997/lora-runpod-the-0-80-ai-revolution-you-cant-afford-to-ignore-c14c2ed857a9
#LoRA #AIRevolution #RunPod
https://medium.com/@rogt.x1997/lora-runpod-the-0-80-ai-revolution-you-cant-afford-to-ignore-c14c2ed857a9

💻 Ever wondered how startups are training 70B parameter models for under $10?

This is your backstage pass to the AI cloud revolution:
• 64 H100s
• 75% cost savings
• 240K tokens per dollar
⚙️ RunPod is quietly powering the next wave of GenAI breakthroughs.

🔥 Read the full case study now:
👉 https://medium.com/@rogt.x1997/why-64-h100s-on-runpod-beat-hyperscalers-and-how-one-startup-slashed-65-of-their-ai-costs-ba251302015e
#LLM #RunPod #GPUCloud #GenAI #TokenEconomy #Mistral
https://medium.com/@rogt.x1997/why-64-h100s-on-runpod-beat-hyperscalers-and-how-one-startup-slashed-65-of-their-ai-costs-ba251302015e

💻 Ever wondered how startups are training 70B parameter models for under $10?

This is your backstage pass to the AI cloud revolution:
• 64 H100s
• 75% cost savings
• 240K tokens per dollar
⚙️ RunPod is quietly powering the next wave of GenAI breakthroughs.

🔥 Read the full case study now:
👉 https://medium.com/@rogt.x1997/why-64-h100s-on-runpod-beat-hyperscalers-and-how-one-startup-slashed-65-of-their-ai-costs-ba251302015e
#LLM #RunPod #GPUCloud #GenAI #TokenEconomy #Mistral
https://medium.com/@rogt.x1997/why-64-h100s-on-runpod-beat-hyperscalers-and-how-one-startup-slashed-65-of-their-ai-costs-ba251302015e

💻 Ever wondered how startups are training 70B parameter models for under $10?

This is your backstage pass to the AI cloud revolution:
• 64 H100s
• 75% cost savings
• 240K tokens per dollar
⚙️ RunPod is quietly powering the next wave of GenAI breakthroughs.

🔥 Read the full case study now:
👉 https://medium.com/@rogt.x1997/why-64-h100s-on-runpod-beat-hyperscalers-and-how-one-startup-slashed-65-of-their-ai-costs-ba251302015e
#LLM #RunPod #GPUCloud #GenAI #TokenEconomy #Mistral
https://medium.com/@rogt.x1997/why-64-h100s-on-runpod-beat-hyperscalers-and-how-one-startup-slashed-65-of-their-ai-costs-ba251302015e

💻 Ever wondered how startups are training 70B parameter models for under $10?

This is your backstage pass to the AI cloud revolution:
• 64 H100s
• 75% cost savings
• 240K tokens per dollar
⚙️ RunPod is quietly powering the next wave of GenAI breakthroughs.

🔥 Read the full case study now:
👉 https://medium.com/@rogt.x1997/why-64-h100s-on-runpod-beat-hyperscalers-and-how-one-startup-slashed-65-of-their-ai-costs-ba251302015e
#LLM #RunPod #GPUCloud #GenAI #TokenEconomy #Mistral
https://medium.com/@rogt.x1997/why-64-h100s-on-runpod-beat-hyperscalers-and-how-one-startup-slashed-65-of-their-ai-costs-ba251302015e

💻 Ever wondered how startups are training 70B parameter models for under $10?

This is your backstage pass to the AI cloud revolution:
• 64 H100s
• 75% cost savings
• 240K tokens per dollar
⚙️ RunPod is quietly powering the next wave of GenAI breakthroughs.

🔥 Read the full case study now:
👉 https://medium.com/@rogt.x1997/why-64-h100s-on-runpod-beat-hyperscalers-and-how-one-startup-slashed-65-of-their-ai-costs-ba251302015e
#LLM #RunPod #GPUCloud #GenAI #TokenEconomy #Mistral
https://medium.com/@rogt.x1997/why-64-h100s-on-runpod-beat-hyperscalers-and-how-one-startup-slashed-65-of-their-ai-costs-ba251302015e

🚨 Breaking the $3000 Barrier
Train AI like a pro—for under $10. LoRA + RunPod is reshaping GenAI creation from your dorm room, café, or coworking desk.
No gatekeepers. Just GPUs and grit.
⚡ #LoRA #RunPod #GenAI #AI4Everyone #AIDemocratization
👉
https://medium.com/@rogt.x1997/from-3-000-to-10-how-lora-runpod-shrink-ai-fine-tuning-costs-by-99-7-7a66d5181fac

I wrote a blog post about running coding assistants on your own infrastructure. It will be useful but not as much as Claude3.7 - and it costs ten times more.

https://davidgrajal.com/2025/03/24/202502_vllm_coding/

#ai #aider #cline #mcp #runpod

I wrote a blog post about running coding assistants on your own infrastructure. It will be useful but not as much as Claude3.7 - and it costs ten times more.

https://davidgrajal.com/2025/03/24/202502_vllm_coding/

#ai #aider #cline #mcp #runpod

I wrote a blog post about running coding assistants on your own infrastructure. It will be useful but not as much as Claude3.7 - and it costs ten times more.

https://davidgrajal.com/2025/03/24/202502_vllm_coding/

#ai #aider #cline #mcp #runpod

I wrote a blog post about running coding assistants on your own infrastructure. It will be useful but not as much as Claude3.7 - and it costs ten times more.

https://davidgrajal.com/2025/03/24/202502_vllm_coding/

#ai #aider #cline #mcp #runpod

I wrote a blog post about running coding assistants on your own infrastructure. It will be useful but not as much as Claude3.7 - and it costs ten times more.

https://davidgrajal.com/2025/03/24/202502_vllm_coding/

#ai #aider #cline #mcp #runpod

Just finished an article on setting up a privately hosted #AI #chatbot using #traefik #docker #librechat and #ollama built on a #runpod GPU backend. #skynet

https://nuxnik.com/private-ai-chatbot/

Just finished an article on setting up a privately hosted #AI #chatbot using #traefik #docker #librechat and #ollama built on a #runpod GPU backend. #skynet

https://nuxnik.com/private-ai-chatbot/

Just finished an article on setting up a privately hosted #AI #chatbot using #traefik #docker #librechat and #ollama built on a #runpod GPU backend. #skynet

https://nuxnik.com/private-ai-chatbot/

Just finished an article on setting up a privately hosted #AI #chatbot using #traefik #docker #librechat and #ollama built on a #runpod GPU backend. #skynet

https://nuxnik.com/private-ai-chatbot/

Just finished an article on setting up a privately hosted #AI #chatbot using #traefik #docker #librechat and #ollama built on a #runpod GPU backend. #skynet

https://nuxnik.com/private-ai-chatbot/

Out early for a Goal Pace run as part of my training for the #GreatSouthRun in October, and a great start to the day 💪🏃‍♂️👍😊

I'm fundraising for UK Sepsis Trust. Check out my Just Giving page and please donate if you can. Thank you! #JustGiving https://www.justgiving.com/page/swinders-sepsis-gsr2024

#garmin #beatyesterday #therunningcommunity
#redfoxrunclub #teamr40plus #running #therunningchannel #runpod #runpodrunclub

Out early for a Goal Pace run as part of my training for the #GreatSouthRun in October, and a great start to the day 💪🏃‍♂️👍😊

I'm fundraising for UK Sepsis Trust. Check out my Just Giving page and please donate if you can. Thank you! #JustGiving https://www.justgiving.com/page/swinders-sepsis-gsr2024

#garmin #beatyesterday #therunningcommunity
#redfoxrunclub #teamr40plus #running #therunningchannel #runpod #runpodrunclub

Out early for a Goal Pace run as part of my training for the #GreatSouthRun in October, and a great start to the day 💪🏃‍♂️👍😊

I'm fundraising for UK Sepsis Trust. Check out my Just Giving page and please donate if you can. Thank you! #JustGiving https://www.justgiving.com/page/swinders-sepsis-gsr2024

#garmin #beatyesterday #therunningcommunity
#redfoxrunclub #teamr40plus #running #therunningchannel #runpod #runpodrunclub

Furthermore:
The pod:
1 x RTX 4000 Ada
9 vCPU 50 GB RAM

#runpod #ollama #ai

Furthermore:
The pod:
1 x RTX 4000 Ada
9 vCPU 50 GB RAM

#runpod #ollama #ai

Furthermore:
The pod:
1 x RTX 4000 Ada
9 vCPU 50 GB RAM

#runpod #ollama #ai

At the moment I am translating lyrics. Running it locally to translate non-English lyrics I get a response time of anywhere around 60 seconds per lyric. on the #runpod around 5 seconds.
My hardware is a #minisforum with 32 gb memory, no gpu.
Only weirdness I notice that it translated 'zonde' (in this context meaning 'a waste' as 'sinful'

At the moment I am translating lyrics. Running it locally to translate non-English lyrics I get a response time of anywhere around 60 seconds per lyric. on the #runpod around 5 seconds.
My hardware is a #minisforum with 32 gb memory, no gpu.
Only weirdness I notice that it translated 'zonde' (in this context meaning 'a waste' as 'sinful'

At the moment I am translating lyrics. Running it locally to translate non-English lyrics I get a response time of anywhere around 60 seconds per lyric. on the #runpod around 5 seconds.
My hardware is a #minisforum with 32 gb memory, no gpu.
Only weirdness I notice that it translated 'zonde' (in this context meaning 'a waste' as 'sinful'

2nd day I am running #ollama on a #runpod pod.
Not that hard to set up (once you know)
1) create a pod of your liking (but it should be a gpu pod) I used the latest RunPod Pytorch as a template 2.2.10
2) add port 11434 to exposed ports
3) add OLLAMA_HOST: 0.0.0.0
4) Start it up and ssh into it
(assuming you have the keys added as needed.
5) run the install script from https://ollama.ai
6) ollama serve &
7) ollama pull [the models you want]

2nd day I am running #ollama on a #runpod pod.
Not that hard to set up (once you know)
1) create a pod of your liking (but it should be a gpu pod) I used the latest RunPod Pytorch as a template 2.2.10
2) add port 11434 to exposed ports
3) add OLLAMA_HOST: 0.0.0.0
4) Start it up and ssh into it
(assuming you have the keys added as needed.
5) run the install script from https://ollama.ai
6) ollama serve &
7) ollama pull [the models you want]