That’s a pretty good way to describe those kinds of models. Somewhere between rubber duck debugging and a really lazy intern. Better than an algorithm, faster than a human, but less trustworthy than an intern. It’s great if you know how and when to use it. Maybe it’ll become an intern soon enough, but for now let’s focus on decompiling adobe into bankruptcy.

Yeah running in the background type beat. Not gonna trust this with anything important but it’s great to have someone just do shit for you or get a second opinion.

That’s gonna be quick af with the same models.

Oh damn what a useful link. Gonna check out those orchestrators, maybe I can replace my janky ass proxy.

MCP is a really dumb server that just takes requests and sends back data.

Somewhere in your frontend settings (Open-WebUI has it under “integrations” I believe, some backends can set these things too) you see a section called “MCP server”. You need to keep your MCP server running somewhere (it’s very slim, so not a huge deal usually) and then in those settings you put your IP and port for the server, for example “http://192.168.1.15:8080/”. Usually if you did it right (sometimes you can check if it works in the settings), you can ask the model if it has access to the tools and it will tell you.

What’s happening in the background is that every conversation the AI does, it gets a short preamble with some context about what it can do, and if you connect web search, custom skills or an MCP server (it’s just skills but they are run through their own server basically), the AI sees all that in this preamble if it’s set up correctly. And then when it feels like it needs to use those tools, it will check the description on how to do that, and then it uses them.

1070ti and some hexacore something cpu that’s alright with around 24gb ddr4, I think one of the slower speeds. I also got a faster PC with another llama.cpp instance and I vibe coded a quick and dirty proxy which I can use to switch between the different backends. It’s genuinely useful because I got code editors but also my Open-WebUI using it.

Sounds elaborate but generally it’s all pretty primitive.

Also fyi long preloading steps because my vram is obv not good enough to hold the full model and kv cache and stuff, generation at about 15t/s tops.

My experience is right now we’re at the stage where it’s a good assistant but not great for actually taking over bigger tasks completely.

Am looking forward to:

• experimenting with existing non-uniform quants; I believe those quants are the next step to more bang for your buck on those low-grade consumer rigs
• hoping for new kinds of quants to come out and be more widely adopted
• maybe a more universal or consistent harness that is less prone to broken tool-calls etc.
• better models, specifically a new version for 35b a3b.

• Qwen3.6 35b a3b Q4_K_M running on llama.cpp
• Open-WebUI as a frontend
• integrations with an embedding model and searxng & crawl4ai
• doing lots of different stuff with it, asking basic stupid questions, recipes, tech stuff, etc

every win should be celebrated

Yeah we could have some more of that here on Lemmy because people tend to be quite negative.

Honestly that’s awesome, I didn’t manage to get grafana working when I tried exactly this, I’ll definitely check out any guides you used ^^

This is exactly why I believe yelling at vibe coders is unhelpful, and giving them pointers on how to do it right is way more effective. Why bully them when you can just make lots of them become genuinely good coders?

Oh God quickly disperse before gimp gang pulls up

Lots of people chimed in regarding battery, which is a legitimate issue.

Another thing though is ARM. Phones usually have a different processor architecture and this architecture used to be incompatible with lots of server software. I remember being pissed at because I couldn’t even host a TeamSpeak Server on it because it wasn’t compatible. Now this was a while back and I’ve heard compatibility has gotten better but still, do some research beforehand.

WSL?

Maybe they found a new funny and quirky way of violating privacy.

It literally was. This happened only minutes before the shooting and the photographer testified in court, describing how he accidentally stumbled upon them and snapping a picture, and then about subsequently becoming a witness to the shooting.

If this was sarcasm, maybe not in great taste considering just a week ago the killer was finally confirmed and for 30 years there was no justice, just violence about it.

So they didn’t even backpedal? Rip windows I guess and probably microslop too if current trends continue.

I second that Optimus sucks a bit still. That said, I have an older laptop with a 1060 mobile and that one runs smooth af.

For like 80% of the hardware I would assume things work without much hassle.

For the 20% there are more or less reliable workarounds.

But the only thing you need to know: these 10 years Linux’ drivers issues really have improved in a major way. If you ask whether it’s worth it to try again, I would say yes. Not a guarantee but a new roll of the dice.

Admittedly voice chat has bunch of quirks still in fluxer but to me it seems the most promising. They are even discussing federated servers which would be insane.

What a bad day to have a brain.

Interesting, I might check that out.

Now this is interesting to say the least.