I use llama-swap with a couple of different models, one 8B as a general chat bot, with offloading to several MoE-models for specific workloads. It’s all run on an old dell xps with 64GB ram.
Mine is used for code reviews, general questions, coding assistance and for anything else I can think of.
Weekly update and restart of my servers. Also planning migration of my k3s cluster from Archlinux to NixOS.