Since you obviously can’t use unified RAM, VRAM is going to make the most notable difference in locally run LLMs. But even if you do get a more powerful GPU, unless you drop thousands of dollars on one with more VRAM (on top of other components you may need to upgrade as well), you are, unfortunately, not going to have anything comparable to frontier cloud models.

The $20 a month is going to be far more affordable. Again, this is all by design so that you are locked into cloud models once you get hooked onto LLMs.

I think most don’t even try, which I don’t blame. I had to essentially put in a business proposal for my work Linux computer.

Just like you, I’m crammed it in :P

I built two of these, one in 2021 for a client and another in 2023 IIRC, back when SSDs and RAM weren’t outrageously priced, so it’s got an NVMe m.2 SSD and 64 GB of RAM.

It’s got an AIO cooler, and I don’t think anything other than that or a stock fan would fit in there. It had a 3060 12 GB but it has died :/ so it’s currently got a 2060 SUPER, which doesn’t fit as well (mainly because the power port points towards the front of the case instead of up.)

With the discrete GPU and AIO cooler, I wouldn’t have been able to fit a SATA SSD in there.

Both of these were far from the most painful builds I’ve done.

I have that exact same case for my work computer!

It has no multiplayer beyond co-op and it has horrible performance issues on PC - Windows or Linux. I refunded it on Steam.

Even the console versions have bad bugs - random disconnects, you can’t go both ways around the island on The Silent Cartographer, and you can’t rebind buttons that are already bound without the game resetting after closing the game - ask me how I know all this.

This might not be a very helpful response, but have you tried getting any Windows software that accomplishes this task running on Linux via WINE?