I run OpenWebUI at home to provide a fronend to run a variety of language models.

This runs on a whitebox Supermicro build that has two 12G Nvidia A2000 GPUs for a total of 24G of video memory and 128G of RAM. If at all possible it's best to keep models within the 24G limit to keep token speeds at a moderate pace. I use LLMs to help with some basic scripting & programming tasks but do not allow it direct access to my machines with something like OpenClaw. I prefer to read and understand the code the LLM provides before I run anything.

Why A2000? It is a strange choice of GPU, I picked them primarily because they run off of the slot-provided 75w of power and do not need any power connectors. Additionally they are half-height and dual slot which keeps them low-profile and quiet. My lab was designed with power efficiency and noise as the primary concerns.