Dilshan R Jayakody’s Web Log

In today's software development landscape, local Large Language Models (LLMs) offer unprecedented privacy, zero latency, and freedom from subscription fees. While I generally avoid purely " vibe coding " for my projects, having robust inline code completion and chat directly integrated into IDEs speeds up development significantly. However, consumer GPUs with high VRAM often cost enough that paying for cloud inference ends up being the more economical option for most developers. To overcome this, I recently completed constructing a custom, dual-rack compute node specifically designed to host LLMs and compile complex codebases without breaking the bank. Here is the full engineering breakdown of how I designed, assembled, and configured this dual Tesla V100 machine from scratch. Fully assembled rig. The core philosophy behind this build was maximizing compute performance and Tensor VRAM density while strictly controlling costs. Consumer-grade GPUs with large VRAM, su...