Just get the cheapest OEM, Bosgame M5. I know “cheapest” is in quotes. But again, it is the cheapest 128GB unified vram device you can get right now. I got mine at pre-orders, so yes, more than a year and a half ago, prices were a steal. Also, great desktop (mini) PC overall and very efficient, even if you don’t want to do AI.

Try gufo with qwen 3.8 flash next. Much much better than 3.6-35B.

It depends on your hardware. But right now the “cheapest” hardware you can buy for a usable setup, is the strix halo (128GB), and run gufo with qwen 3.8 flash next (Q4XL). It is sonnet 5 level. Very good speed (both prefill 1000+ and decode 50+), with up to 256k context.

I have it set up with omp + searxng. Sometimes with hermes. Really good! It changes the game for local setup.

Sounds like a first step for a community meeting focused on 1 topic: helping OP set up reverse proxy. Honestly this sounds fun. I hope it went well!

Great project. One risk I see (and faced in the past) is dying sdcards. They hold up much less overtime compared to ssds. Did you face this before? I have gotten a bunch a corruption on my dbs, and even filesystems (ext4). SDcards are vulnerable especially if running 24/7. I wonder if there are better tiers or better solutions.