Live and recent posts from across the Fediverse tagged #deepspeed, aggregated by home.social.
🧰 LLM Engineer Toolkit is a curated list of 120+ #opensource #LLM libraries, sorted by category across the whole development lifecycle. Each entry gets a one-line description and a link. 10.9k stars, Apache-2.0 license
🎯 Training and fine-tuning: #Unsloth, #PEFT, #TRL, #Axolotl, #LitGPT, #DeepSpeed and #torchtune
🧵👇
Megatron-DeepSpeed 3D Parallelism: Scaling 100B+ LLM Training
A 400B model needs ~800GB just for weights. Factor in optimizer states, and memory hits terabytes.
Our SRE guide covers:
⚙️ Topology: TP (NVLink) + PP + ZeRO-DP math
⚖️ Memory: ZeRO-2 vs ZeRO-3 trade-offs
⚡ Network: NCCL RoCEv2 & GPUDirect RDMA tuning
📜 Slurm: Multi-node torchrun scripts & traps
https://www.servermo.com/howto/setup-megatron-deepspeed-3d-parallelism/