I have no experience with HPC clusters, just dropping in to contribute my opinions on the NixOS stuff.

I won’t be actively maintaining the cluster at all times and find it nice that if someone needs some software I can point them to the configuration and tell them to modify it and then just update the system.

This sounds good in theory, but in practice few people will have the know-how or patience to make changes to your config.

I read that NixOs can have problems with cuda drivers and custom compiled software, which would definitely be a deal breaker.

NixOS is very flexible and you can certainly bake custom compiled software into the builds, but expect a steep learning curve.

I love NixOS but I wouldn’t commend it to people who are at a point in their sysadmin trajectory where they haven’t heard of Ansible. (It’s not just that Nix has a learning curve, it’s that you’re seriously screwed if you can’t tell the difference between a Nix problem and a regular package / kernel issue.)

OP, use Ansible.

Absolutely! I’ll give you an example. In the NixOS config for my desktop I have the lines:

{
  environment.systemPackages = with pkgs; [
    firefox
    ...
  ];
}

So Firefox is installed every time I build a system with this config. This is just like apt-get install firefox in that very user can use it after installation. The config lives in the respective user’s dotfiles (.config/mozilla/firefox) and will of course survive reboots.

What I chose to do additionally (but this is in no way required!) is a home-manager config for my main account with the lines:

{
  programs.firefox = {
    enable = true;
    policies = {
      DisableFirefoxAccounts = true;
      DisablePocket = true;
      DisableTelemetry = true;
      DownloadDirectory = "${config.home.homeDirectory}/tmp";
      OfferToSaveLogins = false;
      ...
  }
  ...
}

This is a declarative configuration that basically handles my dotfiles (profiles, extensions, themes, …) for me. I think you have the impression that this is mandatory, but it is really a very specific behavior through the home-manager module, but you can absolutely run NixOS without it.

I’m not the type to put my dotfiles in git, though.

That’s what I’m saying, you don’t have to! Just install the package (like neovim or whatever) through NixOS and it will use your ad-hoc dotfiles like it would on any other distro. For a lot of stuff you can make use of declarative NixOS options (programs.neovim = { ... };), but you don’t have to, except for really basic system stuff like networking I guess.

Pre-compiled, non-system binaries.

Gotcha. There’s several ways do do this on NixOS (steam-run works like a charm!), but I’ll concede that there’s an extra step involved here that you don’t have to do on other distros.

That, and seeing talk about how great Nix is but also people having trouble later on too (major updates? bleeding edge woes?).

There’s a learning curve for sure, but I haven’t looked back or experienced any major issues (where I hadn’t shot myself in the foot) since 22.11.

Give NixOS another look. It’s no problem to run a declarative system as a base, and plain old dotfiles on top if you want, especially for user stuff. Not sure what you mean by “non-packaged executables” exactly, but I don’t see how NixOS would give you a disadvantage here. Heoric works fine as a Steam alterntative.

Every couple of years I give KDE Plasma a spin, and I’ve found it disappointing every time. Nothing terrible, just a lot of tiny bugs and inconveniences. I find it strange that it gets recommended as a beginner DE so often just because the layout resembles Windows a bit more. Gnome is so much more consistent and aesthetically pleasing… not that I use it, but it’s always my strong recommendation for beginners.

Good luck on your Linux journey!

Speaking of arbitrary Unicode characters… what’s with the þ, out of interest?