Defining non-native types in JavaScript
As explained in my previous blog post about NiJS, we can "translate" most JavaScript language objects to equivalent/similar Nix expression language constructs. However, the Nix expression language has certain value types, such as files and a URLs, that JavaScript does not have. In the old NiJS implementation, objects of these types can be artificially created by adding a _type object member that refers to a string containing url or file.
For various reasons, I have never really liked that approach very much. Therefore, I have adapted NiJS to use prototypes to achieve the same goal, as I now have discovered how to properly use them in JavaScript (which is not logical if you'd ask me). To create a file, we must create an object that is an instance of the NixFile prototype. URLs can be created by instantiating NixURL and recursive attribute sets by instantiating NixRecursiveAttrSet.
By adapting the GNU Hello NiJS package from the previous NiJS blog post using prototypes, we end up with the following CommonJS module:
var nijs = require('nijs');
exports.pkg = function(args) {
return args.stdenv().mkDerivation ({
name : "hello-2.8",
src : args.fetchurl({
url : new nijs.NixURL("mirror://gnu/hello/hello-2.8.tar.gz"),
sha256 : "0wqd8sjmxfskrflaxywc7gqw7sfawrfvdxd9skxawzfgyy0pzdz6"
}),
doCheck : true,
meta : {
description : "A program that produces a familiar, friendly greeting",
homepage : new nijs.NixURL("http://www.gnu.org/software/hello/manual"),
license : "GPLv3+"
}
});
};
In my opinion, this looks a bit better and more logical.
Using CommonJS modules in embedded JavaScript functions
In the previous NiJS blog post, I have also shown that we can wrap JavaScript functions in a function proxy and call them from Nix expressions as if they were Nix functions.
One of the things that is a bit inconvenient is that these functions must be self-contained -- they cannot refer to anything outside the function's scope and we have to provide everything the function needs ourselves.
For me it's also useful to use third party libraries, such as the Underscore library. I have adapted the nijsFunProxy with two extra optional parameters. The modules parameter can be used to refer to external Node.JS packages (provided by Nixpkgs), that are added to the buildInputs of the proxy. The requires parameter can be used to generate require() invocations that import the CommonJS modules that we want to use.
The following example expression invokes a JavaScript function that utilises and imports the Underscore library to convert an array of integers to an array of strings:
{stdenv, nodejs, underscore, nijsFunProxy}:
let
underscoreTestFun = numbers: nijsFunProxy {
function = ''
function underscoreTestFun(numbers) {
var words = [ "one", "two", "three", "four", "five" ];
var result = [];
_.each(numbers, function(elem) {
result.push(words[elem - 1]);
});
return result;
}
'';
args = [ numbers ];
modules = [ underscore ];
requires = [
{ var = "_"; module = "underscore"; }
];
};
in
stdenv.mkDerivation {
name = "underscoreTest";
buildCommand = ''
echo ${toString (underscoreTestFun [ 5 4 3 2 1 ])} > $out
'';
}
In the above expression, the requires parameter to the proxy generates the following line before the function definition, allowing us to properly use the functions provided by the Underscore library:
var _ = require('underscore');
Calling asynchronous JavaScript functions from Nix expressions
This nijsFunProxy is interesting, but most of the functions in the NodeJS API and external NodeJS libraries are executed asynchronously, meaning that they will return immediately and invoke a callback function when the work is done. We cannot use these functions properly from a proxy that is designed to support synchronous functions.
I have extended the nijsFunProxy to take the async parameter, which defaults to false. When the async parameter is set to true, it does not take the return value of the function, but waits until a callback function is called (nijsCallbacks.onSuccess) with a JavaScript object as parameter serving the equivalent of a return value. This can be used to invoke asynchronous JavaScript functions from a Nix function.
{stdenv, nijsFunProxy}:
let
timerTest = message: nijsFunProxy {
function = ''
function timerTest(message) {
setTimeout(function() {
nijsCallbacks.onSuccess(message);
}, 3000);
}
'';
args = [ message ];
async = true;
};
in
stdenv.mkDerivation {
name = "timerTest";
buildCommand = ''
echo ${timerTest "Hello world! The timer test works!"} > $out
'';
}
The above Nix expression shows a simple example, in which we create a timeout event after three seconds. The timer callback calls the nijsCallbacks.onSuccess() callback function to provide a return value containing the message that the user has given as a parameter to the Nix function invocation.
Writing inline JavaScript code in Nix expressions
NiJS makes it possible to use JavaScript instead of the Nix expression language to describe package definitions and their compositions, although I have no reasons to recommend the former over the latter.
However, the examples that I have shown so far in the blog posts use generic build procedures that basically execute the standard GNU Autotools build procedure: ./configure; make; make install. We may also have to implement custom build steps for packages. Usually this is done by specifying custom build steps in Bash shell code embedded in strings.
Not everyone likes to write shell scripts and to embed them in strings. Instead, it may also be desirable to use JavaScript for the same purpose. I have made this possible by creating a nijsInlineProxy function, that generates a string with shell code executing NodeJS to execute a piece of JavaScript code within the same build process.
The following Nix expression uses the nijsInlineProxy to implement the buildCommand in JavaScript instead of shell code:
{stdenv, nijsInlineProxy}:
stdenv.mkDerivation {
name = "createFileWithMessage";
buildCommand = nijsInlineProxy {
requires = [
{ var = "fs"; module = "fs"; }
{ var = "path"; module = "path"; }
];
code = ''
fs.mkdirSync(process.env['out']);
var message = "Hello world written through inline JavaScript!";
fs.writeFileSync(path.join(process.env['out'], "message.txt"), message);
'';
};
}
The above Nix expression creates a Nix store output directory and writes a message.txt file into the corresponding output directory.
As with ordinary Nix expressions, we can refer to the parameters passed to stdenv.mkDerivation as well as the output folder by using environment variables. Inline JavaScript code has the same limitations as embedded JavaScript functions, such as the fact that we can't refer to global variables.
Writing inline JavaScript code in NiJS package modules
If we create a NiJS package module, we also have to use shell code embedded in strings to implement custom build steps. We can also use inline JavaScript code by creating an object that is an instance of the NixInlineJS prototype. The following code fragment is the NiJS equivalent of the previous Nix expression:
var nijs = require('nijs');
exports.pkg = function(args) {
return args.stdenv().mkDerivation ({
name : "createFileWithMessageTest",
buildCommand : new nijs.NixInlineJS({
requires : [
{ "var" : "fs", "module" : "fs" },
{ "var" : "path", "module" : "path" }
],
code : function() {
fs.mkdirSync(process.env['out']);
var message = "Hello world written through inline JavaScript!";
fs.writeFileSync(path.join(process.env['out'], "message.txt"), message);
}
})
});
};
The constructor of the NixInlineJS prototype can take two types of parameters. It can take a string containing JavaScript code or a JavaScript function (that takes no parameters). The latter case has the advantage that its syntax can be checked by an interpreter/compiler and that we can use syntax highlighting in editors.
By using inline JavaScript code in NiJS, we can create Nix packages by only using JavaScript. Isn't that awesome? :P
Conclusion
In this blog post, I have described some interesting improvements to NiJS, such as the fact that we can create Nix packages by only using JavaScript. NiJS seems to be fairly complete in terms of features now.
If you want to try it, the source can be obtained from the NiJS GitHub page, or installed from Nixpkgs or by NPM.
Another thing that's in my mind now is whether I can do the same stuff for a different programming language. Maybe when I'm bored or I have a use case for it, I'll give it a try.
No comments:
Post a Comment