In the marketplace of developer skills, there’s a certain category that I’m long-term bearish on. They don’t warrant investing free time trying to level up. Devs should limit their exposure to the minimum necessary to do their job. I’m talking about tools like Docker, Kubernetes, and the intricate managed hosting platforms of Azure and AWS. These are tools that help manage

  • social problems caused by scaling organizations to thousands of engineers
  • existing complexity
  • laziness or desire to throw money at a problem rather than solve it

While they do solve real technical problems, I’ll argue here that there’s superior tooling for most use cases outside of big tech and legacy systems.

Containerize or Modularize?

We need deployment consistency and environment isolation, and containers solve those!

The 2016 essay Docker Considered Harmful explains how Linux primitives solve the same problems in a more robust way. One valid criticism of this is that Docker can get a junior developer productive more quickly without needing to learn about arcane system settings.

Fine, so we need an abstraction to work with.

Does the abstraction need to expose us to a bloated ecosystem prone to drift? What if it could evaluate to the precise, minimal, and correct system without introducing overhead? What if it’s a better Docker image builder than Docker’s image builder?

Enter Nix with its module system. Nix approaches the “dependency hell” problem from first principles. It requires a radically different mental model, which is itself a social organization problem. This model, by the way, is laid out best in Eelco Dolstra’s 2006 PHD Thesis (PDF).

When you think about

  • polyglot system with multiple language ecosystems working together
  • co-existing packages with conflicting dependencies (i.e dfferent versions of python)

do you get a headache thinking about all the setup? Well then you should probably swallow some Nix pills. No other tool can encapsulate cross-ecosystem practices and Linux expertise so cleanly. A simple configuration change like

services.postgres.enable = true;

is the tip of a carefully crafted reproducible iceberg. This line of code hides a host of system-level changes including

  • User and groups setup with correct isolation and permissions
  • Directory structure with ownership and permissions
  • Initialization, default template databases
  • SystemD service with correct dependencies, start order, lifecycle and recovery and much more!

It’s a massive setup process that would normally be done manually, maybe captured in brittle shell scripts, or containerized. But here it’s a one-liner, and it’s exposed for use as a declarative model that allows completely deterministic and atomic system upgrades!

So while containers provide language-agnostic deployment, Nix achieves the same universality at the package level while maintaining better reproducibility and lower overhead. You can even spit out a Docker image as an afterthought if that’s what you really need.

Cluster or Cluster*@!#?

We need service discovery, load balancing, and observability! Kubernetes provides these with a vast ecosystem of battle-tested tools, and its nice declarative operator pattern for nearly every infrastructure need.

The infrastructure problem is solved by creating different problems: cognitive overhead, ecosystem lock-in, long development cycles, and significant cost markup. There are compelling reasons to look for a better alternative.

Well, the largest package repo in the world (Nixpkgs) has nice modules defined for over 20,000 world-class open-source tools. With a 3-line change to your system configuration, you’re now building a robust set of tools:

services = {
  nginx.enable = true;      #Load balancing
  consul.enable = true;     #Service discovery
  prometheus.enable = true; #Observability
  #You'll want to configure these, of course.

But that’s not it. The Nix flake output schema specifies nixosConfigurations as a key. Note the plural. So you might do something like:

nixosConfigurations = {
  web = mkNode "web" {
    services.nginx.enable = true;
    services.nginx.virtualHosts."web" = {
      locations."/api/" = {
      proxyPass = "http://${cluster.db.ip}:5000/";
    };
    app.db_address = cluster.db.ip
  };

  db = mkNode "db" {
    services.postgresql.enable = true;
    services.postgresql.enableTCPIP = true;
    services.postgresql.authentication = ''
      host all all ${cluster.web.ip}/32 trust
      '';
    api.enable = true
  };
};

where the 2 nodes configurations make them aware of each other.

The same flake can even include

So the question then becomes, do you have a nest of messy legacy code and need to throw money at scaling it up? Or do you have the luxury of using the tools directly and scaling piecewise as needs arise?

Summary

It’s never going to be easy to retrofit legacy enterprise systems. But for designing new systems, the NixOS advantage is massive. Your entire system configuration, from kernel to application dependencies to infrastructure services, is a single, reproducible, version-controlled artifact. It allows for

  • One deployment mechanism
  • Deterministic deployments
  • Exact parity between development and production

and most importantly, a level of autonomy and sovereignty that you yield when your system is built on unstable abstractions.