I’ve been using a local Qwen3.8 model as a NixOS sysadmin lately for a non-critical VPS. So far, it has been working surprisingly well! I guess it’s easier for AI agents to gather context when the whole system and apps are defined in code, and immutable when applied.
I’ve been wanting to do this, even as a testbed. I just want computers to do cool shit, and am getting tired of being a Linux sysadmin in my free time. How large of a model do you have assigned to this?
I was fortunate to have bought my graphics card almost a year ago, and for a nice discount (a Black Friday sale, IIRC). I’d love to buy another, but they now cost >2x the price I paid.
FWIW, you can probably run it on CPU if you have enough RAM. I assume it will be really slow though.
Might also be worth trying Qwen3.6 35B-A3B. It’s not as “smart” as Qwen3.8 27B, but can be much faster on limited resources.
I’ve been using a local Qwen3.8 model as a NixOS sysadmin lately for a non-critical VPS. So far, it has been working surprisingly well! I guess it’s easier for AI agents to gather context when the whole system and apps are defined in code, and immutable when applied.
I’ve been wanting to do this, even as a testbed. I just want computers to do cool shit, and am getting tired of being a Linux sysadmin in my free time. How large of a model do you have assigned to this?
Just Qwen3.8-27B. That little model is surprisingly smart.
NixOS also gives you the ability to roll back to previous config versions. I have not needed to do that (yet), but it reduces some risk.
“Just” and a 27B model don’t exactly belong in the same sentence if you ask my wallet. 💸
Yeahhh sorry 😕
I was fortunate to have bought my graphics card almost a year ago, and for a nice discount (a Black Friday sale, IIRC). I’d love to buy another, but they now cost >2x the price I paid.
FWIW, you can probably run it on CPU if you have enough RAM. I assume it will be really slow though.
Might also be worth trying Qwen3.6 35B-A3B. It’s not as “smart” as Qwen3.8 27B, but can be much faster on limited resources.