Links for 2014-09-02

  • Nix: The Purely Functional Package Manager

    ‘a powerful package manager for Linux and other Unix systems that makes package management reliable and reproducible. It provides atomic upgrades and rollbacks, side-by-side installation of multiple versions of a package, multi-user package management and easy setup of build environments. ‘ Basically, this is a third-party open source reimplementation of Amazon’s (excellent) internal packaging system, using symlinks to versioned package directories to ensure atomicity and the ability to roll back. This is definitely the *right* way to build packages — I know what tool I’ll be pushing for, next time this question comes up. See also nixos.org for a Linux distro built on Nix.

    (tags: ops linux devops unix packaging distros nix nixos atomic upgrades rollback versioning)

This entry was posted in Uncategorized. Bookmark the permalink. Both comments and trackbacks are currently closed.

5 Comments

  1. Luca Bruno
    Posted September 3, 2014 at 11:04 | Permalink

    How do you know amazon is using this approach? What’s the source?

    • Posted September 3, 2014 at 12:23 | Permalink

      They used to use this approach — I don’t know if they still do, to be honest. I had the pleasure of using Amazon’s packaging system for several years, while I worked there.

      http://stackoverflow.com/a/12597919 has some more details from an ex-member of the Amazon Build Systems team.

      BTW, I should clarify — I don’t think that Nix was an intentional reimplementation, just that it takes a similar approach (and IMO it’s the right way to do it)…

  2. Nix
    Posted September 3, 2014 at 14:59 | Permalink

    The unusual thing about NixOS (which I am not associated with — a coincidence of names, decades apart) is not the symlink-farm approach, which has been used by many other systems (GNU stow, graft, depot, etc), but rather the functional language and system wrapped around it. This has problems — notably with programs like various parts of both KDE and GNOME that assume they can symlink-chase, realpath(), then walk up a directory level or two and look at other packages — but they are survivable with minor patching.

    But the scheme has further disadvantages unique to itself, principal among them that it does not know anything about ABI stability — rather than assuming Unix SONAME rules (slightly unreliable and prone to the occasional crash-inducing FN) or attempting to automatically verify ABI compatibility (prone to FPs but still acceptable for this purpose, since 99% of the time it is right) it assumes that all users of a shared library must be rebuilt whenever that shared library is, even if just for a minor bugfix. This is downright crazy: a lot of critical libraries (like libX11 or glibc) have strong ABI compatibility rules, but without special-casing on a per-package basis NixOS doesn’t know this, and triggers a massive rebuild of the world whenever one of these is rebuilt (for, say, a crucial security fix you’d rather have now without waiting for the world to rebuild).

    But it’s a very nice scheme nonetheless. It says something about how nice the scheme is that it’s usable despite this fault.

    (Disclaimer: I haven’t used it for a year or two now. It might have fixed the ABI thing, but from what I hear it hasn’t.)

    • Posted September 3, 2014 at 15:40 | Permalink

      depot! Thanks Nix, I was trying to remember what that tool was called.

  3. Luca Bruno
    Posted September 3, 2014 at 15:09 | Permalink

    It’s not true that you necessarily have to wait for a security issue: https://nixos.org/wiki/Security_Updates . There’s no security channel in nixos because of lack of manpower, but certainly for critical non-ABI breaking updates it’s possible to not rebuild the world. As you say it might be crazy, but until now I was able to run any program perfectly fine even on non-NixOS systems. The rebuild-the-world is certainly not nice, but it guarantees certain properties that are otherwise hard to check manually on every update. Also, automatic ABI compatibility is only valid for elf binaries. Good luck with scripts or other kind of binaries.

    That said, nixos certainly is not perfect… in my opinion every system until now has its own downsides. However after running for several months nixos on both desktop and server, I feel satisfied so far.