Although low-code development is conceptually different from a development perspective compared to more "traditional" development approaches (that require you to write code), there is one particular aspect a Mendix application lifecycle has in common. Eventually, you will have to deploy your app to an environment that makes your application available to end users.
For users of the Mendix cloud portal, deploying an application is quite convenient: with just a few simple mouse clicks your application gets deployed to a test, acceptance or production environment.
However, managing on-premise application deployments or actually managing applications in the cloud is all but a simple job. There all all kinds of challenges you need to cope with, such as:
- Making sure that all dependencies of an app are present, such as a database for storage.
- Executing all relevant deployment activities to make an app available for use.
- Upgrading is risky and difficult -- it may break the application and introduce downtime.
There are a variety of deployment solutions available to manage deployment processes. However, no solution is perfect -- every tool has its strengths and weaknesses and no tool is a perfect fit. As a result, we still have to develop custom solutions that automate missing parts in a deployment process and we have many kinds of additional complexities that we need to cope with.
Recently, I investigated whether it would be possible to deploy Mendix applications, with my favourite class of deployment utilities from the Nix project, and I gave an introduction to the Nix project to the R&D department at Mendix.
Using tools from the Nix project
For readers not familiar with Nix: the tools in the Nix project solve many configuration management problems in their own unique way. The basis of all the tools is the Nix package manager that borrows concepts from purely functional programming languages, such as Haskell.
To summarize Nix in just a few sentences: deploying a package with Nix is the same thing as invoking a pure function that constructs a package from source code and its build-time dependencies (that are provided as function parameters). To accomplish purity, Nix composes so-called "pure build environments", in which various restrictions are imposed on the build script to ensure that the outcome will be (almost) identical if a package is built with the same build inputs.
The purely functional deployment model has all kinds of benefits -- for example, it provides very strong dependency completeness and reproducibility guarantees, and all kinds of optimizations (e.g. a package that has been deployed before does not have to be built again, packages that have no dependency on each other can be built in parallel, builds can be downloaded from a remote location or delegated to another machine).
Another important property that all tools in the Nix project have in common is declarative deployment -- instead of describing the deployment activities that need to be carried out, you describe the structure of your system that want to deploy, e.g. the packages, a system configuration, or a network of machines/services. The deployment tools infer the activities that need to be carried out to get the system deployed.
Automating Mendix application deployments with Nix
As an experiment, I investigated how Mendix application deployments could fit in Nix's vision of declarative deployment -- the objective is to take a Mendix project created by the modeler (essentially the "source code" form of an application), write a declarative deployment specification for it, and use the tools from the Nix project to get a machine running with all required components to make the app run.
To bring a Mendix application in a running state, we require the following ingredients:
- We must obtain the Mendix runtime that interprets the Mendix models. Packaging the Mendix runtime in Nix is fairly straight forward -- simply unzipping the distribution, and moving the package contents into the Nix store, and adding a wrapper script launches the runtime suffices.
- We must produce a Mendix Deployment Archive (MDA file) that creates a Zip container with all artifacts required to run a Mendix app by the runtime. An MDA file can be produced from a Mendix project by invoking the MxBuild tool. Since MxBuild is required for this, I had to package it as well. Packaging mxbuild is a bit trickier, as it requires mono and Node.js.
Building an MDA file with Nix
The most interesting part is writing a new function abstraction for building MDA files with Nix -- in a Nix builder environment, (almost) any build tool can be used albeit with restrictions that are imposed on them to make builds more pure.
We can also create a function abstraction that invokes mxbuild in a Nix builder environment:
{stdenv, mxbuild, jdk, nodejs}:
{name, mendixVersion, looseVersionCheck ? false, buildInputs ? [], ...}@args:
let
mxbuildPkg = mxbuild."${mendixVersion}";
extraArgs = removeAttrs args [ "buildInputs" ];
in
stdenv.mkDerivation ({
buildInputs = [ mxbuildPkg nodejs ] ++ buildInputs;
installPhase = ''
mkdir -p $out
mxbuild --target=package \
--output=$out/${name}.mda \
--java-home ${jdk} \
--java-exe-path ${jdk}/bin/java \
${stdenv.lib.optionalString looseVersionCheck "--loose-version-check"} \
"$(echo *.mpr)"
mkdir -p $out/nix-support
echo "file binary-dist \"$(echo $out/*.mda)\"" > $out/nix-support/hydra-build-products
'';
} // extraArgs)
The above expression is a function that composes another function that takes common Mendix parameters -- the application name, the version of MxBuild that we want, and whether we want to use a strict or loose version check (it is possible to compile a project developed for a different version of Mendix, if desired).
In the body, we create an output directory in the Nix store, we invoke mxbuild to compile to MDA app and put it in the Nix store, and we generate a configuration file that makes it possible to expose the MDA file as a build product, when Hydra: the Nix-based continuous integration service is being used.
With the build function shown in the code fragment above, we can write a Nix expression for a Mendix project:
{ pkgs ? import { inherit system; }
, system ? builtins.currentSystem
}:
let
mendixPkgs = import ./nixpkgs-mendix/top-level/all-packages.nix {
inherit pkgs system;
};
in
mendixPkgs.packageMendixApp {
name = "conferenceschedule";
src = /home/sander/SharedWindowsFolder/ConferenceSchedule-main;
mendixVersion = "7.13.1";
}
The above expression (conferenceschedule.nix) can be used to build an MDA file for a project named: conferenceschedule, residing in the /home/sander/SharedWindowsFolder/ConferenceSchedule-main directory using Mendix version 7.13.1.
By running the following command-line instruction, we can use Nix to build our MDA:
$ nix-build conferenceschedule.nix /nix/store/nbaa7fnzi0xw9nkf27mixyr9awnbj16i-conferenceschedule $ ls /nix/store/nbaa7fnzi0xw9nkf27mixyr9awnbj16i-conferenceschedule conferenceschedule.mda nix-support
In addition to building an MDA, Nix will also download the dependencies: the Mendix runtime and MxBuild, if they have not been installed yet.
Running a Mendix application
Producing an MDA file is an important ingredient in the deployment lifecycle of a Mendix application, but it is not entirely what we want -- what we really want is a running system. To get a running system, additional steps are required beyond producing an MDA:
- We must unzip the MDA file into a directory with write permissions.
- We must create writable state sub directories, e.g. data/tmp, data/files.
- After starting the runtime, we must configure the admin interface, to send instructions to the runtime to initialize the database and start the app:
$ export M2EE_ADMIN_PORT=9000 $ export M2EE_ADMIN_PASS=secret
- Finally, we must communicate over the admin interface to configure, initialize the database and start the app:
curlCmd="curl -X POST http://localhost:$M2EE_ADMIN_PORT \ -H 'Content-Type: application/json' \ -H 'X-M2EE-Authentication: $(echo -n "$M2EE_ADMIN_PASS" | base64)' \ -H 'Connection: close'" $curlCmd -d '{ "action": "update_appcontainer_configuration", "params": { "runtime_port": 8080 } }' $curlCmd -d '{ "action": "update_configuration", "params": { "DatabaseType": "HSQLDB", "DatabaseName": "myappdb", "DTAPMode": "D" } }' $curlCmd -d '{ "action": "execute_ddl_commands" }' $curlCmd -d '{ "action": "start" }'
These deployment steps cannot be executed by Nix, because Nix's purpose is to manage packages, but not the state of a running process. To automate these remaining parts, we generate scripts that execute the above listed steps.
NixOS integration
NixOS is a Linux distribution that extends Nix's deployment facilities to complete systems. Aside from using the Nix package manage to deploy all packages including the Linux kernel, NixOS' main objective is to deploy an entire system from a single declarative specification capturing the structure of an entire system.
NixOS uses systemd for managing system services. The systemd configuration files are generated by the Nix package manager. We can integrate our Mendix activation scripts with a generated systemd job to fully automate the deployment of a Mendix application.
{pkgs, ...}:
{
...
systemd.services.mendixappcontainer =
let
runScripts = ...
appContainerConfigJSON = ...
configJSON = ...
in {
enable = true;
description = "My Mendix App";
wantedBy = [ "multi-user.target" ];
environment = {
M2EE_ADMIN_PASS = "secret";
M2EE_ADMIN_PORT = "9000";
MENDIX_STATE_DIR = "/home/mendix";
};
serviceConfig = {
ExecStartPre = "${runScripts}/bin/undeploy-app";
ExecStart = "${runScripts}/bin/start-appcontainer";
ExecStartPost = "${runScripts}/bin/configure-appcontainer ${appContainerConfigJSON} ${configJSON}";
};
};
The partial NixOS configuration shown above defines a systemd job that runs three scripts (as shown in the last three lines):
- The undeploy-app script removes all non-state artefacts from the working directory.
- The start-appcontainer script starts the Mendix runtime.
- The configure-appcontainer script configures the runtime, such as the embedded Jetty server and the database, and starts the application.
Writing a systemd job (as shown above) is a bit cumbersome. To make it more convenient to use, I captured all Mendix runtime functionality in a NixOS module, with an interface exposing all relevant configuration properties.
By importing the Mendix NixOS module into a NixOS configuration, we can conveniently define a machine configuration that runs our Mendix application:
{pkgs, ...}:
{
require = [ ../nixpkgs-mendix/nixos/modules/mendixappcontainer.nix ];
services = {
openssh.enable = true;
mendixAppContainer = {
enable = true;
adminPassword = "secret";
databaseType = "HSQLDB";
databaseName = "myappdb";
DTAPMode = "D";
app = import ../../conferenceschedule.nix {
inherit pkgs;
inherit (pkgs.stdenv) system;
};
};
};
networking.firewall.allowedTCPPorts = [ 8080 ];
}
In the above configuration, the mendixAppContainer captures all the properties of the Mendix application that we want to run:
- The password for communicating over the admin interface.
- The type of database we want to use (in this particular case an in memory HSQLDB instance) and the name of the database.
- Whether we want to use the application in development (D), test (T), acceptance (A) or production (P) mode.
- A reference to the MDA that we want to deploy (deployed by a Nix expression that invokes the Mendix build function abstraction shown earlier).
By writing a NixOS configuration file, storing it in /etc/nixos/configuration.nix and running the following command-line instruction:
$ nixos-rebuild switch
A complete system gets deployed with the Nix package manager that runs our Mendix application.
For production use, HSQLDB and directly exposing the embedded Jetty HTTP server is not recommended -- instead a more sophisticated database, such as PostgreSQL should be used. For serving HTTP requests, it is recommended to use nginx as a reverse proxy and use it to serve static data and provide caching.
It is also possible to extend the above configuration with a PostgreSQL and nginx system service. The NixOS module system can be used to retrieve the properties from the Mendix app container to make the configuration process more convenient.
Conclusion
In this blog post, I have investigated how Mendix applications can be deployed by using tools from the Nix project. This resulted in the following deployment functionality:
- A Nix function that can be used to compile an MDA file from a Mendix project.
- Generated scripts that configure and launch the runtime and the application.
- A NixOS module that can be used to deploy a running Mendix app as part of a NixOS machine configuration.
Future work
Currently, only single machine deployments are possible. It may also be desirable to connect a Mendix application to a database that is stored on a remote machine. Furthermore, we may also want to deploy multiple Mendix applications to multiple machines in a network. With Disnix, it is possible to automate such scenarios.
Availability
The Nix function abstractions and NixOS module can be obtained from the Mendix GitHub page and used under the terms and conditions of the Apache Software License version 2.0.
Acknowledgements
The work described in this blog post is the result of the so-called "crafting days", in which Mendix supports its employees to experiment completely freely two full days a month.
Furthermore, I have given a presentation about the functionality described in this blog post and an introduction to the Nix project:
and I have also written an introduction-oriented article about it on the Mendix blog.
No comments:
Post a Comment