Both of these underlying solutions have a number of similarities -- as their names obviously suggest, they both strongly depend on the Nix package manager to deploy all their package dependencies and static configuration artifacts, such as configuration files.
Furthermore, they are both driven by models written in the Nix expression language to automate the deployment processes of entire systems.
These models are built on a number of simple conventions that are frequently used in the Nix packages repository:
- All units of which a system consists are defined as Nix expressions declaring a function. Each function parameter refers to a dependency or configuration property required to construct the unit from its sources.
- To compose a particular variant of a unit, we must invoke the function that builds and configures the unit with parameters providing the dependencies and configuration properties that the unit needs.
- To make all units conveniently accessible from a single location, the content of the configuration units is typically blended into a symlink tree called Nix profiles.
Besides these commonalities, their main difference is that the process management framework is specifically designed as a solution for systems that are composed out of running processes (i.e. daemons in UNIX terminology).
This framework makes it possible to construct multiple instances of running processes, isolate their resources (by avoiding conflicting resource configuration properties), and manage running process with a variety of process management solutions, such as sysvinit scripts, BSD rc scripts, systemd, launchd and supervisord.
The process management framework is quite useful for single machine deployments and local experimentation, but it does not do any distributed deployment and heterogeneous service deployment -- it cannot (at least not conveniently) deploy units that are not daemons, such as databases, Java web applications deployed to a Servlet container, PHP applications deployed to a PHP-enabled web server etc.
Disnix is a solution to automate the deployment processes of service-oriented systems -- distributed systems that are composed of components, using a variety of technologies, into a network of machines.
To accomplish full automation, Disnix integrates and combines a number of activities and tools, such as Nix for package management and Dysnomia for state management (Dysnomia takes care of the activation, deactivation steps for services, and can optionally manage snapshots and restores of state). Dysnomia provides a plugin system that makes it possible to manage a variety of component types, including processes and databases.
Disnix and Dysnomia can also include the features of the Nix process management framework for the deployment of services that are running processes, if desired.
The scope of Disnix is quite broad in comparison to the process management framework, but it can also be used to automate all kinds of sub problems. For example, it can also be used as a remote package deployment solution to build and deploy packages in a network of heterogeneous machines (e.g. Linux and macOS).
After comparing the properties of both deployment solutions, I have identified another interesting sub use case for Disnix -- deploying heterogeneous service-oriented systems (that are composed out of components using a variety of technologies) locally for experimentation purposes.
In this blog post, I will describe how Disnix can be used for local deployments.
Motivating example: deploying a Java-based web application and web service system
One of the examples I have shown in the previous blog post, is an over engineered Java-based web application and web service system which only purpose is to display the string: "Hello world!".
The "Hello" string is returned by the HelloService and consumed by another service called HelloWorldService that composes the sentence "Hello world!" from the first message. The HelloWorld web application is the front-end responsible for displaying the sentence to the end user.
When deploying the system to a single target machine, it could have the following deployment architecture:
In the architecture diagram shown above, ovals denote services, arrows inter-dependency relationships (requiring that a service gets activated before another), the dark grey colored boxes container environments, and the light grey colored box a machine (which is only one machine in the above example).
As you may notice, only one service in the diagram shown above is a daemon, namely Apache Tomcat (simpleAppservingTomcat) that can be managed by the experimental Nix process management framework.
The remainder of the services have a different kind of form -- the web application front-end (HelloWorld) is a Java web application that is embedded in Catalina, the Servlet container that comes with Apache Tomcat. The web services are Axis2 archives that are deployed to the Axis2 container (that in turn is a web application managed by Apache Tomcat).
In the previous blog post, I have shown that we can deploy and distribute these services over a small network of machines.
It is also possible to completely deploy this system locally, without any external physical or virtual machines, and network connectivity.
Configuring the client interface for local deployment
To execute deployment tasks remotely, Disnix invokes an external process that is called a client interface. By default, Disnix uses the disnix-ssh-client that remotely executes commands via SSH and transfers data via SCP.
It is also possible to use alternative client interfaces so that different communication protocols and methods can be used. For example, there is also an external package that provides a SOAP client disnix-soap-client and a NixOps client (disnix-nixops-client).
Communication with a local Disnix service instance can also be done with a client interface. For example, configuring the following environment variable:
$ export DISNIX_CLIENT_INTERFACE=disnix-client
instructs the Disnix tools to use the D-Bus client to communicate with a local Disnix service instance.
It is also possible to bypass the local Disnix service and directly execute all deployment activities with the following interface:
$ export DISNIX_CLIENT_INTERFACE=disnix-runactivity
The disnix-runactivity client interface is particularly useful for single-user/unprivileged user deployments. In the former case, you need a Disnix D-Bus daemon running in the background that authorizes the user to execute deployments. For the latter, nothing is required beyond a single user Nix installation.
Deploying the example system locally
As explained in earlier blog posts about Disnix, deployments are driven by three kinds of deployment specifications: a services model capturing all the services of which a system consists and how they depend on each other, an infrastructure model captures all available target machines and their relevant configuration properties (including so-called container services that can host application services) and the distribution model maps services in the services model to target machines in the infrastructure model (and container services that a machine may provide).
Normally, Disnix deploys services to remote machines defined in the infrastructure model. For local deployments, we simply need to provide an infrastructure model with only one entry:
{
localhost.properties.hostname = "localhost";
}
In the distribution model, we must map all services to the localhost target:
{infrastructure}:
{
simpleAppservingTomcat = [ infrastructure.localhost ];
axis2 = [ infrastructure.localhost ];
HelloService = [ infrastructure.localhost ];
HelloWorldService = [ infrastructure.localhost ];
HelloWorld = [ infrastructure.localhost ];
}
With the above infrastructure and distribution model that facilitates local deployment, and the services model of the example system shown above, we can deploy the entire system on our local machine:
$ disnix-env -s services.nix -i infrastructure-local.nix -d distribution-local.nix
Deploying the example system locally as an unprivileged user
The deployment scenario shown earlier supports local deployment, but still requires super-user privileges. For example, to deploy Apache Tomcat, we must have write access to the state directory: /var to configure Apache Tomcat's state and deploy the Java web application archives. An unprivileged user typically lacks the permissions to perform modifications in the /var directory.
One of they key features of the Nix process management framework is that it makes all state directories are configurable. State directories can be changed in such a way that also unprivileged users can deploy services (e.g. by changing the state directory to a sub folder in the user's home directory).
Disnix service models can also define these process management configuration parameters:
{ pkgs, system, distribution, invDistribution
, stateDir ? "/var"
, runtimeDir ? "${stateDir}/run"
, logDir ? "${stateDir}/log"
, cacheDir ? "${stateDir}/cache"
, tmpDir ? (if stateDir == "/var" then "/tmp" else "${stateDir}/tmp")
, forceDisableUserChange ? false
, processManager ? "systemd"
}:
let
processType =
if processManager == null then "managed-process"
else if processManager == "sysvinit" then "sysvinit-script"
else if processManager == "systemd" then "systemd-unit"
else if processManager == "supervisord" then "supervisord-program"
else if processManager == "bsdrc" then "bsdrc-script"
else if processManager == "cygrunsrv" then "cygrunsrv-service"
else if processManager == "launchd" then "launchd-daemon"
else throw "Unknown process manager: ${processManager}";
constructors = import ../../../nix-processmgmt/examples/service-containers-agnostic/constructors.nix {
inherit pkgs stateDir runtimeDir logDir cacheDir tmpDir forceDisableUserChange processManager;
};
customPkgs = import ../top-level/all-packages.nix {
inherit system pkgs stateDir;
};
in
rec {
simpleAppservingTomcat = constructors.simpleAppservingTomcat {
httpPort = 8080;
type = processType;
};
...
}
The above Nix expression shows a partial Nix services model for the Java example system. The first four function parameters: pkgs, system, distribution, and invDistribution are standard Disnix service model parameters.
The remainder of the parameters are specific to the process management framework -- they allow you to change the state directories, force disable user changing (this is useful for unprivileged user deployments) and the process manager it should use for daemons.
I have added a new command-line parameter (--extra-params) to the Disnix tools that can be used to propagate values for these additional parameters.
With the following command-line instruction, we change the base directory of the state directories to the user's home directory, force disable user changing (only a privileged user can do this), and change the process manager to sysvinit scripts:
$ disnix-env -s services.nix -i infrastructure-local.nix -d distribution-local.nix \
--extra-params '{
stateDir = "/home/sander/var";
processManager = "sysvinit";
forceDisableUserChange = true;
}'
With the above command, we can deploy the example system completely as an unprivileged user, without requiring any process/service manager to manage Apache Tomcat.
Working with predeployed container services
In our examples so far, we have deployed systems that are entirely self contained. However, it is also possible to deploy services to container services that have already been deployed by other means. For example, it is also possible to install Apache Tomcat with your host system's distribution and use Dysnomia to integrate with that.
To allow Disnix to deploy services to these containers, we need an infrastructure model that knows its properties. We can automatically generate an infrastructure model from the Dysnomia container configuration files, by running:
$ disnix-capture-infra infrastructure.nix > \ infrastructure-captured.nix
and using the captured infrastructure model to locally deploy the system:
$ disnix-env -s services.nix -i infrastructure-captured.nix -d distribution-local.nix
Undeploying a system
For local experimentation, it is probably quite common that you want to completely undeploy the system as soon as you no longer need it. Normally, this should be done by writing an empty distribution model and redeploying the system with that empty distribution model, but that is still a bit of a hassle.
In the latest development version of Disnix, an undeploy can be done with the following command-line instruction:
$ disnix-env --undeploy -i infrastructure.nix
Availability
The --extra-params and --undeploy Disnix command-line options are part of the current development version of Disnix and will become available in the next release.
No comments:
Post a Comment