The framework is built around two concepts. As its name implies, the Nix package manager is used to deploy all required packages and static artifacts, and a process manager of choice (e.g. sysvinit, systemd, supervisord and others) is used to manage the life-cycles of the processes.
Moreover, it is built around flexible concepts allowing integration with solutions that are not qualified as process managers (but can still be used as such), such as Docker -- each process instance can be deployed as a Docker container with a shared Nix store using the host system's network.
As explained in an earlier blog post, Docker has become such a popular solution that it has become a standard for deploying (micro)services (often as a utility in the Kubernetes solution stack).
When deploying a system that consists of multiple services with Docker, a typical strategy (and recommended practice) is to use multiple containers that have only one root application process. Advantages of this approach is that Docker can control the life-cycles of the applications, and that each process is (somewhat) isolated/protected from other processes and the host system.
By default, containers are isolated, but if they need to interact with other processes, then they can use all kinds of integration facilities -- for example, they can share namespaces, or use shared volumes.
In some situations, it may also be desirable to deviate from the one root process per container practice -- for some systems, processes may need to interact quite intensively (e.g. with IPC mechanisms, shared files or shared memory, or a combination these) in which the container boundaries introduce more inconveniences than benefits.
Moreover, when running multiple processes in a single container, common dependencies can also typically be more efficiently shared leading to lower disk and RAM consumption.
As explained in my previous blog post (that explores various Docker concepts), sharing dependencies between containers only works if containers are constructed from images that share the same layers with the same shared libraries. In practice, this form of sharing is not always as efficient as we want it to be.
Configuring a Docker image to run multiple application processes is somewhat cumbersome -- the official Docker documentation describes two solutions: one that relies on a wrapper script that starts multiple processes in the background and a loop that waits for the "main process" to terminate, and the other is to use a process manager, such as supervisord.
I realised that I could solve this problem much more conveniently by combining the dockerTools.buildImage {} function in Nixpkgs (that builds Docker images with the Nix package manager) with the Nix process management abstractions.
I have created my own abstraction function: createMultiProcessImage that builds multi-process Docker images, managed by any supported process manager that works in a Docker container.
In this blog post, I will describe how this function is implemented and how it can be used.
Creating images for single root process containers
As shown in earlier blog posts, creating a Docker image with Nix for a single root application process is very straight forward.
For example, we can build an image that launches a trivial web application service with an embedded HTTP server (as shown in many of my previous blog posts), as follows:
{dockerTools, webapp}:
dockerTools.buildImage {
name = "webapp";
tag = "test";
runAsRoot = ''
${dockerTools.shadowSetup}
groupadd webapp
useradd webapp -g webapp -d /dev/null
'';
config = {
Env = [ "PORT=5000" ];
Cmd = [ "${webapp}/bin/webapp" ];
Expose = {
"5000/tcp" = {};
};
};
}
The above Nix expression (default.nix) invokes the dockerTools.buildImage function to automatically construct an image with the following properties:
- The image has the following name: webapp and the following version tag: test.
- The web application service requires some state to be initialized before it can be used. To configure state, we can run instructions in a QEMU virual machine with root privileges (runAsRoot).
In the above deployment Nix expression, we create an unprivileged user and group named: webapp. For production deployments, it is typically recommended to drop root privileges, for security reasons. - The Env directive is used to configure environment variables. The PORT environment variable is used to configure the TCP port where the service should bind to.
- The Cmd directive starts the webapp process in foreground mode. The life-cycle of the container is bound to this application process.
- Expose exposes TCP port 5000 to the public so that the service can respond to requests made by clients.
We can build the Docker image as follows:
$ nix-build
load it into Docker with the following command:
$ docker load -i result
and launch a container instance using the image as a template:
$ docker run -it -p 5000:5000 webapp:test
If the deployment of the container succeeded, we should get a response from the webapp process, by running:
$ curl http://localhost:5000
<!DOCTYPE html>
<html>
<head>
<title>Simple test webapp</title>
</head>
<body>
Simple test webapp listening on port: 5000
</body>
</html>
Creating multi-process images
As shown in previous blog posts, the webapp process is part of a bigger system, namely: a web application system with an Nginx reverse proxy forwarding requests to multiple webapp instances:
{ pkgs ? import <nixpkgs> { inherit system; }
, system ? builtins.currentSystem
, stateDir ? "/var"
, runtimeDir ? "${stateDir}/run"
, logDir ? "${stateDir}/log"
, cacheDir ? "${stateDir}/cache"
, tmpDir ? (if stateDir == "/var" then "/tmp" else "${stateDir}/tmp")
, forceDisableUserChange ? false
, processManager
}:
let
sharedConstructors = import ../services-agnostic/constructors.nix {
inherit pkgs stateDir runtimeDir logDir cacheDir tmpDir forceDisableUserChange processManager;
};
constructors = import ./constructors.nix {
inherit pkgs stateDir runtimeDir logDir tmpDir forceDisableUserChange processManager;
};
in
rec {
webapp = rec {
port = 5000;
dnsName = "webapp.local";
pkg = constructors.webapp {
inherit port;
};
};
nginx = rec {
port = 8080;
pkg = sharedConstructors.nginxReverseProxyHostBased {
webapps = [ webapp ];
inherit port;
} {};
};
}
The Nix expression above shows a simple processes model variant of that system, that consists of only two process instances:
- The webapp process is (as shown earlier) an application that returns a static HTML page.
- nginx is configured as a reverse proxy to forward incoming connections to multiple webapp instances using the virtual host header property (dnsName).
If somebody connects to the nginx server with the following host name: webapp.local then the request is forwarded to the webapp service.
Configuration steps
To allow all processes in the process model shown to be deployed to a single container, we need to execute the following steps in the construction of an image:
- Instead of deploying a single package, such as webapp, we need to refer to a collection of packages and/or configuration files that can be managed with a process manager, such as sysvinit, systemd or supervisord.
The Nix process management framework provides all kinds of Nix function abstractions to accomplish this.
For example, the following function invocation builds a configuration profile for the sysvinit process manager, containing a collection of sysvinit scripts (also known as LSB Init compliant scripts):
profile = import ../create-managed-process/sysvinit/build-sysvinit-env.nix { exprFile = ./processes.nix; stateDir = "/var"; };
- Similar to single root process containers, we may also need to initialize state. For example, we need to create common FHS state directories (e.g. /tmp, /var etc.) in which services can store their relevant state files (e.g. log files, temp files).
This can be done by running the following command:
nixproc-init-state --state-dir /var - Another property that multiple process containers have in common is that they may also require the presence of unprivileged users and groups, for security reasons.
With the following commands, we can automatically generate all required users and groups specified in a deployment profile:
${dysnomia}/bin/dysnomia-addgroups ${profile} ${dysnomia}/bin/dysnomia-addusers ${profile} - Instead of starting a (single root) application process, we need to start a process manager that manages the processes that we want to deploy. As already explained, the framework allows you to pick multiple options.
Starting a process manager as a root process
From all process managers that the framework currently supports, the most straight forward option to use in a Docker container is: supervisord.
To use it, we can create a symlink to the supervisord configuration in the deployment profile:
ln -s ${profile} /etc/supervisor
and then start supervisord as a root process with the following command directive:
Cmd = [
"${pkgs.pythonPackages.supervisor}/bin/supervisord"
"--nodaemon"
"--configuration" "/etc/supervisor/supervisord.conf"
"--logfile" "/var/log/supervisord.log"
"--pidfile" "/var/run/supervisord.pid"
];
(As a sidenote: creating a symlink is not strictly required, but makes it possible to control running services with the supervisorctl command-line tool).
Supervisord is not the only option. We can also use sysvinit scripts, but doing so is a bit tricky. As explained earlier, the life-cycle of container is bound to a running root process (in foreground mode).
sysvinit scripts do not run in the foreground, but start processes that daemonize and terminate immediately, leaving daemon processes behind that remain running in the background.
As described in an earlier blog post about translating high-level process management concepts, it is also possible to run "daemons in the foreground" by creating a proxy script. We can also make a similar foreground proxy for a collection of daemons:
#!/bin/bash -e
_term()
{
nixproc-sysvinit-runactivity -r stop ${profile}
kill "$pid"
exit 0
}
nixproc-sysvinit-runactivity start ${profile}
# Keep process running, but allow it to respond to the TERM and INT
# signals so that all scripts are stopped properly
trap _term TERM
trap _term INT
tail -f /dev/null & pid=$!
wait "$pid"
The above proxy script does the following:
- It first starts all sysvinit scripts by invoking the nixproc-sysvinit-runactivity start command.
- Then it registers a signal handler for the TERM and INT signals. The corresponding callback triggers a shutdown procedure.
- We invoke a dummy command that keeps running in the foreground without consuming too many system resources (tail -f /dev/null) and we wait for it to terminate.
- The signal handler properly deactivates all processes in reverse order (with the nixproc-sysvinit-runactivity -r stop command), and finally terminates the dummy command causing the script (and the container) to stop.
In addition supervisord and sysvinit, we can also use Disnix as a process manager by using a similar strategy with a foreground proxy.
Other configuration properties
The above configuration properties suffice to get a multi-process container running. However, to make working with such containers more practical from a user perspective, we may also want to:
- Add basic shell utilities to the image, so that you can control the processes, investigate log files (in case of errors), and do other maintenance tasks.
- Add a .bashrc configuration file to make file coloring working for the ls command, and to provide a decent prompt in a shell session.
Usage
The configuration steps described in the previous section are wrapped into a function named: createMultiProcessImage, which itself is a thin wrapper around the dockerTools.buildImage function in Nixpkgs -- it accepts the same parameters with a number of additional parameters that are specific to multi-process configurations.
The following function invocation builds a multi-process container deploying our example system, using supervisord as a process manager:
let
pkgs = import <nixpkgs> {};
createMultiProcessImage = import ../../nixproc/create-multi-process-image/create-multi-process-image.nix {
inherit pkgs system;
inherit (pkgs) dockerTools stdenv;
};
in
createMultiProcessImage {
name = "multiprocess";
tag = "test";
exprFile = ./processes.nix;
stateDir = "/var";
processManager = "supervisord";
}
After building the image, and deploying a container, with the following commands:
$ nix-build $ docker load -i result $ docker run -it --network host multiprocessimage:test
we should be able to connect to the webapp instance via the nginx reverse proxy:
$ curl -H 'Host: webapp.local' http://localhost:8080
<!DOCTYPE html>
<html>
<head>
<title>Simple test webapp</title>
</head>
<body>
Simple test webapp listening on port: 5000
</body>
</html>
As explained earlier, the constructed image also provides extra command-line utilities to do maintenance tasks, and control the life-cycle of the individual processes.
For example, we can "connect" to the running container, and check which processes are running:
$ docker exec -it mycontainer /bin/bash # supervisorctl nginx RUNNING pid 11, uptime 0:00:38 webapp RUNNING pid 10, uptime 0:00:38 supervisor>
If we change the processManager parameter to sysvinit, we can deploy a multi-process image in which the foreground proxy script is used as a root process (that starts and stops sysvinit scripts).
We can control the life-cycle of each individual process by directly invoking the sysvinit scripts in the container:
$ docker exec -it mycontainer /bin/bash $ /etc/rc.d/init.d/webapp status webapp is running with Process ID(s) 33. $ /etc/rc.d/init.d/nginx status nginx is running with Process ID(s) 51.
Although having extra command-line utilities to do administration tasks is useful, a disadvantage is that they considerably increase the size of the image.
To save storage costs, it is also possible to disable interactive mode to exclude these packages:
let
pkgs = import <nixpkgs> {};
createMultiProcessImage = import ../../nixproc/create-multi-process-image/create-multi-process-image.nix {
inherit pkgs system;
inherit (pkgs) dockerTools stdenv;
};
in
createMultiProcessImage {
name = "multiprocess";
tag = "test";
exprFile = ./processes.nix;
stateDir = "/var";
processManager = "supervisord";
interactive = false; # Do not install any additional shell utilities
}
Discussion
In this blog post, I have described a new utility function in the Nix process management framework: createMultiProcessImage -- a thin wrapper around the dockerTools.buildImage function that can be used to convienently build multi-process Docker images, using any Docker-capable process manager that the Nix process management framework supports.
Besides the fact that we can convienently construct multi-process images, this function also has the advantage (similar to the dockerTools.buildImage function) that Nix is only required for the construction of the image. To deploy containers from a multi-process image, Nix is not a requirement.
There is also a drawback: similar to "ordinary" multi-process container deployments, when it is desired to upgrade a process, the entire container needs to be redeployed, also requiring a user to terminate all other running processes.
Availability
The createMultiProcessImage function is part of the current development version of the Nix process management framework that can be obtained from my GitHub page.
No comments:
Post a Comment