NixOS: A purely functional Linux distribution

My last blog post was about the Nix package manager, a package manager with some distinct features, such as multiple versions/variants packages which are stored in isolation from each other in a Nix store, the Nix expression language and atomic upgrades and rollbacks. I also briefly mentioned NixOS, a Linux distribution using the Nix package manager as a basis. In this blog post I will discuss the concepts of NixOS more thoroughly.

It's probably already very obvious that NixOS uses the Nix package manager for installing, upgrading and removing software components. In fact, except for end-user software such as Mozilla Firefox and OpenOffice.org, development tools such as GCC, Binutils, also all operating system parts such as the Linux kernel, system configuration files and kernel modules are all managed by the Nix package manager and stored in isolation in the Nix store.

Because of the fact that Nix stores all packages in a special directory: /nix/store, the NixOS distribution does not have standard FHS directories such as: /usr and /lib and only a minimal /etc and /bin. The /bin directory only contains a symlink: /bin/sh pointing to the bash shell. This is required in order to make glibc's system() function working properly. I have uploaded a screenshot (seen above) in which a terminal is shown, listing the directory contents of the root filesystem, /bin folder and some other files. As you may notice, certain directories commonly found on other distributions don't exist, and the /bin/sh points to the bash shell which is also stored somewhere in the Nix store.

The fact that Nix is used instead of a conventional package manager such as RPM or dpkg, does probably not sound so exciting to most users. Actually, NixOS is more then just a Linux distribution managed by Nix. The main thing that sets NixOS apart from conventional Linux distributions is that system configurations are also configured with declarative specifications just Nix like packages. Instead of manually installing packages and manually editing configurations files, such as an Apache web server configuration, everything is configured from a single declarative specification, invoking functions which generate configuration files and install required packages.

{pkgs, ...}:

{                                                                                                                                                            
  boot.loader.grub.device = "/dev/sda";
                                                
  fileSystems = [
    { mountPoint = "/";
      device = "/dev/sda2";
    }
  ];

  swapDevices = [ { device = "/dev/sda1"; } ];
  
  services = {
    openssh.enable = true;

    httpd = {
      enable = true;
      documentRoot = "/var/www";
      adminAddr = "admin@localhost";
    };

    xserver = {
      enable = true;
      
      desktopManager = {
        default = "kde4";
        kde4 = { enable = true; };
      };
      
      displayManager = {
        kdm = { enable = true; };
        slim = { enable = false; };
      };
    };
  };
  
  environment.systemPackages = [
    pkgs.mc
    pkgs.subversion
    pkgs.firefox
  ];
}

The code fragment above shows an example of a NixOS configuration. As you may notice, this configuration file contains all kinds of settings for a particular system, such as the MBR partition for the GRUB bootloader, filesystems (such as the root and swap partition), system services (OpenSSH, Apache Webserver and X.Org configured to use the KDE desktop and the KDM login manager). Also, system packages are defined here such as Mozilla Firefox and Midnight Commander, which are automatically in the PATH of all users and also appear automatically in the KDE menu if desktop files are included.

By storing a configuration file in /etc/nixos/configuration.nix and by running: nixos-rebuild switch a complete system configuration is built by the Nix package manager and activated. Because of the fact that the Nix package manager is used, we can safely upgrade a complete system; this is because the complete system configuration and all its dependencies are safely stored in the Nix store and no files are overwritten or removed. Because of this we can also rollback to any previous configuration, which has not been garbage collected yet.

The figure above shows the GRUB boot loader used for NixOS. As you may notice, all the previous configurations of the system are shown, which are not garbage collected yet. The GRUB boot loader of NixOS allows you to boot into any previous configuration (You can also switch back to a previous configuration while running NixOS).

The last missing piece in the puzzle is how configuration files are handled in NixOS. In NixOS you don't manually edit configuration files. Instead, configuration files are generated by Nix expressions and stored in the Nix store (just like ordinary packages). Because they are generated by Nix expressions and they also may contain paths to Nix expressions containing binaries, this also has a nice implication. If a configuration file contains a Nix store path, Nix handles them as dependencies, and automatically includes them if a configuration is deployed somewhere.

Maybe you have become interested in trying out NixOS after reading this. ISOs can be obtained from the NixOS website: http://nixos.org/nixos. In NixOS quite a number of packages are supported. We also have end-user software available and the KDE desktop. Keep in mind that NixOS is an experimental Linux distribution, which may not offer everything you need. We only have a small community of ~25 developers maintaining it.

If you find some concepts of NixOS useful but you don't want to switch, you can also use the virtualization features of NixOS to automatically generate virtual machines running a particular NixOS configuration. You can even do this on regular Linux distributions supporting KVM having the Nix package manager installed.

One point of criticism I always receive is about the fact that we don't edit configuration files manually. I have to admit that this is a bit inconvenient, if you want to try out a particular feature of a service. But manually editing stuff becomes very inconvenient in a distributed setting. Probably everyone agrees that it is unfeasible in case of a configuration change, to manually update the configurations of every affected machine. One of the things we investigate in the PDS research project is expanding the Nix deployment vision to distributed systems.

There are some publications about NixOS available. The first prototype of NixOS was implemented in 2006 by Armijn Hemel for his masters thesis. Nicolas Pierron implemented the NixOS module system. Various publications about NixOS can be obtained from Eelco Dolstra's publications page.

The Nix package manager

In my previous blog post, which was about Pull Deployment of Services, I have explained what the project I'm working on as a PhD student is about. An import foundation is the Nix package manager, which serves as an important basis of my research.

There are dozens of papers and websites available describing Nix, so in this blog post I will not give another extensive description. Instead, I'm going to explain some of the details and the vision that we have about software deployment which sets the Nix approach apart from conventional approaches.

Nix is a package manager developed by Eelco Dolstra as part of his PhD research, which has similar purposes as tools such as RPM and dpkg commonly found in many Linux distributions, such as Red Hat enterprise Linux, Fedora, Debian and Ubuntu.

A package manager is basically a collection of software tools to automate the process of installing, upgrading, configuring, and removing software packages for an operating system in a consistent manner. In many Linux distributions, the package manager is one of the major factors that distinguish them from each other. Similarly, an experimental Linux distribution has been built around Nix, called NixOS using the Nix package manager as its basis.

One of the major differences of Nix compared to conventional deployment tooling, is the way Nix addresses packages (or components). Most tools use nominal dependency specifications, consisting of a name and version number, such as:

openssl-1.0.0c

Although it is a good thing to have these attributes to distinguish packages (which for instance isn't properly done for native Windows DLLs), this dependency specification mechanism has some limitations. A problem may occur when OpenSSL is compiled with a different version of GCC as an executable linking to it. A library compiled with an older version of GCC cannot always be linked to an executable compiled with a newer version of GCC, due to a different ABI. Moreover, OpenSSL also has dependencies on libraries, such as glibc. If OpenSSL is compiled against an incompatible older version of glibc as an executable, also problems may occur.

In Nix, we have different way of addressing components. We store components in a special directory called the Nix store in which every component is stored under an unique filename, such as:

/nix/store/xq2bfcqdsbrmfr8h5ibv7n1qb8xs5s79-openssl-1.0.0c

The former part of the component name: xq2bfcqdsbrmfr8h5ibv7n1qb8xs5s79 is a SHA256 hash code derived from all build time dependencies to build the component from source code. For example, if the same component is built with a different version of GCC, a different hash code is generated.

Using hash codes to address components offers us some nice advantages. Since every component has an unique filename, we can safely store multiple versions and variants next to each other, without overwriting each other. This also offers us the possibility to upgrade a system atomically. In case of an upgrade a new component is safely stored next to an existing one, and after the upgrade only a symlink pointing to the configuration by a user is changed.

The hash codes in the component names are also used to detect runtime dependencies of a component. This is done by scanning for occurrences of a hash inside a component. If a hash code is found (such as a path containing a Nix store component in an ELF header, or a shell script containing a Nix store path) Nix identifies these components as a runtime dependency. Although this technique sounds risky/scary, we have used this for many packages and it turns out to work quite well.

Another key feature is the Nix expression language, which we use to build packages, as shown in the code fragment below.

{stdenv, fetchurl}:

stdenv.mkDerivation {
  name = "hello-2.6";
  
  src = fetchurl {
    url = ftp://ftp.gnu.org/gnu/hello/hello-2.6.tar.gz;
    sha256 = "1h6fjkkwr7kxv0rl5l61ya0b49imzfaspy7jk9jas1fil31sjykl";
  };

  meta = {
    homepage = http://www.gnu.org/software/hello/manual/;
    license = "GPLv3+";
  };
}

Basically each package description is a function which takes some dependencies as arguments (in this case stdenv and fetchurl). In the body of the function we call the mkDerivation function describing a build action, which takes some arguments, such as a reference to the source code and other relevant build properties. In this expression we did not specify how to build this component. By default, Nix assumes that a packages is autotools based (i.e. ./configure; make; make install) if build instructions are omitted.

The expression described earlier cannot be used to build a package directly. The function must be called with the right arguments, i.e. a component must be composed. This is done in another expression, shown below:

rec {
  stdenv = ...;

  fetchurl = import ../build-support/fetchurl {
    inherit stdenv curl;
  };

  hello = import ../applications/misc/hello {
    inherit stdenv fetchurl;
  }

  ...
}

In this expression we call the function building the GNU Hello component (shown in the previous expression) with its required arguments. As you may notice, all the dependencies of the GNU Hello package are composed in this expression as well.

Now you may think that this dependency scheme of using hash codes to address components is a bit unhandy for end users. To solve this problem, we use Nix profiles. A Nix profile is basically an environment in the PATH of a user, containing symlinks indirectly referring to components in the Nix store. For example, by typing:

$ nix-env -i hello

The GNU Hello component is installed into the Nix profile of the current user. By typing the following instruction on the command-line:

$ hello

The GNU Hello component installed earlier is executed.

The last distinguishable feature I'd like to mention is that in Nix packages are not directly removed, but removed by a garbage collector. The Nix garbage collector will safely remove components that are no longer in use. In most conventional tooling you may accidentally remove a package which may be still in use. Packages which are still installed in a profile of a user or which are a dependency of another package, will not be removed by Nix. Moreover, running processes and open files are also garbage collector roots, which makes it safe to run the garbage collector at any time.

After reading this blog post, you may have become interested in trying the Nix package manager. The simplest way to try out Nix is by downloading and installing NixOS. Although NixOS is a Linux distribution built around Nix, you can also use Nix on conventional Linux distributions, if you find it more convenient to keep using the operating system you are used to. Moreover, the Nix package manager is quite portable and can also be used on different operating systems, such as FreeBSD, OpenSolaris, Mac OS X and Windows (through Cygwin).

Except for plain package management and the management of a Linux distribution, Nix is also used as foundation for some other tooling. Hydra is a continuous build and integration server built on top of Nix, which allows you to continuously checkout sources from repositories and to build components in several variations (e.g. using various compilers, libraries and platforms). I have developed an extension called Disnix, which extends the Nix package manager for the deployment of distributable components (or services) into a network of machines.

Another thing I want to clarify is that I don't want to claim that other package management solutions (and deployment tooling) are bad solutions. In fact, I think other package management solutions have done a great job in dealing with certain installation issues. Moreover, I find package management tools one the strong points of Linux distributions, although we support some distinct features which we care about, which other package managers don't.

We are not the only group doing research in package management. There is also a project, called Mancoosi, in which package and system management issues are investigated. They have a different view about certain principles, such as keeping maintainer scripts which imperatively modify the system configuration (whereas we generate everything as packages in the Nix store). Moreover, their solutions are based on package management tools widely used in Linux distributions, such as RPM and dpkg.

For more information about Nix, have a look at the Nix website: http://nixos.org/nix or have a look at Eelco's excellent PhD thesis titled: 'The Purely Functional Software Deployment Model', which can be found on his publications page.