#techdilemmas — Public Fediverse posts on home.social

Planning to locally host an LLM 🤖. Having a lot of internal struggles figuring out the best price curve.

A used NVIDIA Jetson Xavier NX can be acquired for ~$250 💰.
A 4060 w/ 16gb for ~$400 💳.

Not sure I really want to do either 🤔. Waiting for NPU support on Transformer models may be a better approach.

#llm #localhosting #nvidia #ai #machinelearning #npu

Planning to locally host an LLM 🤖. Having a lot of internal struggles figuring out the best price curve.

A used NVIDIA Jetson Xavier NX can be acquired for ~$250 💰.
A 4060 w/ 16gb for ~$400 💳.

Not sure I really want to do either 🤔. Waiting for NPU support on Transformer models may be a better approach.

#llm #localhosting #nvidia #ai #machinelearning #npu

Planning to locally host an LLM 🤖. Having a lot of internal struggles figuring out the best price curve.

A used NVIDIA Jetson Xavier NX can be acquired for ~$250 💰.
A 4060 w/ 16gb for ~$400 💳.

Not sure I really want to do either 🤔. Waiting for NPU support on Transformer models may be a better approach.

#llm #localhosting #nvidia #ai #machinelearning #npu

Planning to locally host an LLM 🤖. Having a lot of internal struggles figuring out the best price curve.

A used NVIDIA Jetson Xavier NX can be acquired for ~$250 💰.
A 4060 w/ 16gb for ~$400 💳.

Not sure I really want to do either 🤔. Waiting for NPU support on Transformer models may be a better approach.

#hardwarechoices #techdilemmas #transformers #npu #machinelearning #ai

Planning to locally host an LLM 🤖. Having a lot of internal struggles figuring out the best price curve.

A used NVIDIA Jetson Xavier NX can be acquired for ~$250 💰.
A 4060 w/ 16gb for ~$400 💳.

Not sure I really want to do either 🤔. Waiting for NPU support on Transformer models may be a better approach.

#llm #localhosting #nvidia #ai #machinelearning #npu