However, when you are developing a product or service, you typically do not only want to use configuration management tools, such as Nix -- you may also want to build a platform that is tailored towards your needs, so that common operations can be executed structurally and conveniently.
When it is desired to integrate custom solutions with Nix-related tools, you basically have one recurring challenge -- you must generate deployment specifications in the Nix expression language.
The most obvious solution is to use string manipulation to generate the expressions we want, but this has a number of disadvantages. Foremost, composing strings is not a very intuitive activity -- it is not always obvious to see what the end result would be by looking at the code.
Furthermore, it is difficult to ensure that a generated expression is correct and safe. For example, if a string value is not properly escaped, it may be possible to inject arbitrary deployment code putting the security of the deployed system at risk.
For these reasons, I have developed NiJS: an internal DSL for JavaScript, a couple of years ago to make integration with JavaScript-based applications more convenient. Most notably, NiJS is used by node2nix to generate Nix expressions from NPM package deployment specifications.
I have been doing PHP development in the last couple of weeks and realized that I needed a similar solution for this language. In this blog post, I will describe PNDP, an internal DSL for Nix in PHP, and show how it can be used.
Composing Nix packages in PHP
The Nix packages repository follows a specific convention for organizing packages -- every package is a Nix expression file containing a function definition describing how to build a package from source code and its build-time dependencies.
A top-level composition expression file provides all the function invocations that build variants of packages (typically only one per package) by providing the desired versions of the build-time dependencies as function parameters.
Every package definition typically invokes stdenv.mkDerivation {} (or abstractions built around it) that composes a dedicated build environment in which only the specified dependencies can be found and other kinds of precautions are taken to improve build reproducibility. In this builder environment, we can execute many kinds of build steps, such as running GNU Make, CMake, or Apache Ant.
In our internal DSL in PHP we can replicate these conventions using PHP language constructs. We can compose a proxy to the stdenv.mkDerivation {} invocation in PHP by writing the following class:
namespace Pkgs;
use PNDP\AST\NixFunInvocation;
use PNDP\AST\NixExpression;
class Stdenv
{
public function mkDerivation($args)
{
return new NixFunInvocation(new NixExpression("pkgs.stdenv.mkDerivation"), $args);
}
}
In the above code fragment, we define a class named: Stdenv exposing a method named mkDerivation. The method composes an abstract syntax tree for a function invocation to stdenv.mkDerivation {} using an arbitrary PHP object of any type as a parameter.
With the proxy shown above, we can create our own in packages in PHP by providing a function definition that specifies how a package can be built from source code and its build-time dependencies:
namespace Pkgs;
use PNDP\AST\NixURL;
class Hello
{
public static function composePackage($args)
{
return $args->stdenv->mkDerivation(array(
"name" => "hello-2.10",
"src" => $args->fetchurl(array(
"url" => new NixURL("mirror://gnu/hello/hello-2.10.tar.gz"),
"sha256" => "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"
)),
"doCheck" => true,
"meta" => array(
"description" => "A program that produces a familiar, friendly greeting",
"homepage" => new NixURL("http://www.gnu.org/software/hello/manual"),
"license" => "GPLv3+"
)
));
}
}
The above code fragment defines a class named 'Hello' exposing one static method named: composePackage(). The composePackage method invokes the stdenv.mkDerivation {} proxy (shown earlier) to build GNU Hello from source code.
In addition to constructing a package, the above code fragment also follows the PHP conventions for modularization -- in PHP it is a common practice to modularize code chunks into classes that reside in their own namespace. For example, by following these conventions, we can also automatically load our package classes by using an autoloading implementation that follows the PSR-4 recommendation.
We can create compositions of packages as follows:
class Pkgs
{
public $stdenv;
public function __construct()
{
$this->stdenv = new Pkgs\Stdenv();
}
public function fetchurl($args)
{
return Pkgs\Fetchurl::composePackage($this, $args);
}
public function hello()
{
return Pkgs\Hello::composePackage($this);
}
}
As with the previous example, the composition example is a class. In this case, it exposes variants of packages by calling the functions with their required function arguments. In the above example, there is only one variant of the GNU Hello package. As a result, it suffices to just propagate the object itself as build parameters.
Contrary to the Nix expression language, we must expose each package composition as a method -- the Nix expression language is a lazy language that only invokes functions when their results are needed, PHP is an eager language that will evaluate them at construction time.
An implication of using eager evaluation is that opening the composition module, triggers all packages to be built. By wrapping the compositions into methods, we can make sure that only the requested packages are evaluated when needed.
Another practical implication of creating methods for each package composition is that it can become quite tedious if we have many of them. PHP offers a magic method named: __call() that gets invoked when we invoke a method that does not exists. We can use this magic method to automatically compose a package based on the method name:
public function __call($name, $arguments)
{
// Compose the classname from the function name
$className = ucfirst($name);
// Compose the name of the method to compose the package
$methodName = 'Pkgs\\'.$className.'::composePackage';
// Prepend $this so that it becomes the first function parameter
array_unshift($arguments, $this);
// Dynamically the invoke the class' composition method with $this as first parameter and the remaining parameters
return call_user_func_array($methodName, $arguments);
}
The above method takes the (non-existent) method name, converts it into the corresponding class name (by using the camel case naming convention), invokes the package's composition method using the composition object itself as a first parameter, and any other method parameters as successive parameters.
Converting PHP language constructs into Nix language constructs
Everything that PNDP does boils down to the phpToNix() function that automatically converts most PHP language constructs into semantically equivalent or similar Nix language constructs. For example, the following PHP language constructs are converted to Nix as follows:
- A variable of type boolean, integer or double are converted verbatim.
- A string will be converted into a string in the Nix expression language, and conflicting characters, such as the backslash and double quote, will be escaped.
- In PHP, arrays can be sequential (when all elements have numeric keys that appear in numeric order) or associative in the remainder of the cases. The generator tries to detect what kind of array we have. It recursively converts sequential arrays into Nix lists of Nix language elements, and associative arrays into Nix attribute sets.
- An object that is an instance of a class, will be converted into a Nix attribute set exposing its public properties.
- A NULL reference gets converted into a Nix null value.
- Variables that have an unknown type or are a resource will throw an exception.
As with NiJS (and JavaScript), the PHP host language does not provide equivalents for all Nix language constructs, such as values of the URL type, or encoding Nix function definitions.
You can still generate these objects by composing an abstract syntax from objects that are instances of the NixObject class. For example, when composing a NixURL object, we can generate a value of the URL type in the Nix expression language.
Arrays are a bit confusing in PHP, because you do not always know in advance whether it would yield a list or attribute set. To make these conversions explicit and prevent generation errors, they can be wrapped inside a NixList or NixAttrSet object.
Building packages programmatically
The PNDPBuild::callNixBuild() function can be used to build a generated Nix expression, such as the GNU Hello example shown earlier:
/* Evaluate the package */
$expr = PNDPBuild::evaluatePackage("Pkgs.php", "hello", false);
/* Call nix-build */
PNDPBuild::callNixBuild($expr, array());
In the code fragment above, we open the composition class file, named: Pkgs.php and we evaluate the hello() method to generate the Nix expression. Finally, we call the callNixBuild() function, in which we evaluate the generated expression by the Nix package manager. When the build succeeds, the resulting Nix store path is printed on the standard output.
Building packages from the command-line
As the previous code example is so common, there is also a command-line utility that can execute the same task. The following instruction builds the GNU Hello package from the composition class (Pkgs.php):
$ pndp-build -f Pkgs.php -A hello
It may also be useful to see what kind of Nix expression is generated for debugging or testing purposes. The --eval-only option prints the generated Nix expression on the standard output:
$ pndp-build -f Pkgs.js -A hello --eval-only
We can also nicely format the generated expression to improve readability:
$ pndp-build -f Pkgs.js -A hello --eval-only --format
Discussion
In this blog post, I have described PNDP: an internal DSL for Nix in PHP.
PNDP is not the first internal DSL I have developed for Nix. A couple of years ago, I also wrote NiJS: an internal DSL in JavaScript. PNDP shares a lot of concepts and implementation details with NiJS.
Contrary to NiJS, the functionality of PNDP is much more limited -- I have developed PNDP mainly for code generation purposes. In NiJS, I have also been exploring the abilities of the JavaScript language, such as exposing JavaScript functions in the Nix expression language, and the possibilities of an internal DSL, such as creating an interpreter that makes it a primitive standalone package manager. In PNDP, all this extra functionality is missing, since I have no practical need for them.
In a future blog post, I will describe an application that uses PNDP as a generator.
Availability
PNDP can obtained from Packagist as well as my GitHub page. It can be used under the terms and conditions of the MIT license.
