#techdilemmas — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #techdilemmas, aggregated by home.social.
-
Planning to locally host an LLM 🤖. Having a lot of internal struggles figuring out the best price curve.
A used NVIDIA Jetson Xavier NX can be acquired for ~$250 💰.
A 4060 w/ 16gb for ~$400 💳.Not sure I really want to do either 🤔. Waiting for NPU support on Transformer models may be a better approach.
#LLM #LocalHosting #NVIDIA #AI #MachineLearning #NPU #Transformers #TechDilemmas #HardwareChoices