9th annual blog reflection

Today, it is my blog's 9th anniversary. Similar to previous years, this is a good opportunity for reflection.

A summary of 2019

There are a variety of things that I did in 2019 that can be categorized as follows:

Documenting the architecture of service-oriented systems

In the beginning of the year, most of my efforts were concentrated on software architecture related aspects. At Mendix, I have spent quite a bit of time on documenting the architecture of the platform that I and my team work on.

I wrote a blog post about my architecture documentation approach, that is based on a small number of simple configuration management principles that I came up several years ago. I gave a company wide presentation about this subject, that was well-received.

Later, I have extended Dynamic Disnix, a self-adaptive deployment framework that I have developed as part of my past research, with new tools to automate the architecture documentation approach that I have presented previously -- with these new tools, it has become possible to automatically generate architecture diagrams and descriptions from Disnix service models.

Furthermore, the documentation tools can also group components (making it possible to decompose diagrams into layers) and generate architecture documentation catalogs for complete systems.

Improving Disnix's data handling and data integration capabilities

While implementing the architecture documentation tools in Dynamic Disnix, I ran into several limitations of Disnix's internal architecture.

To alleviate the burden of developing new potential future integrations, I have "invented" NixXML, an XML file format that is moderately readable, that can be easily converted from and to Nix expressions, and is still useful to be used without Nix. I wrote a library called libnixxml that can be used to produce and consume NixXML data.

In addition to writing libnixxml. I also rewrote significant portions of Disnix and Dynamic Disnix to work with libnixxml (instead of hand written parsing and Nix expression generation code). The result is that we need far less boilerplate code, we have more robust data parsing, and we have significantly better error reporting.

In the process of rewriting the Disnix data handling portions, I have been following a number of conventions that I have documented in this blog post.

Implementing a new input model transformation pipeline in Disnix

Besides data integration and data handling, I have also re-engineered the input model transformation pipeline in Disnix, that translates the Disnix input models (a services, infrastructure and distribution model) capturing aspects of a service-oriented systems to an executable model (a deployment manifest/model) from which the deployment activities that it needs to run can be derived.

The old implementation evolved organically and has grown so much that it became too complicated to adjust and maintain. In contrast, the new transformation pipeline consists of well-defined intermediate models.

As a result of having a better, more well-defined transformation pipeline, more properties can be configured.

Moreover, it is now also possible to use a fourth and new input model: a packages model that can be used to directly deploy a set of packages to a target machine in the network.

The new data handling features and transformation pipeline were integrated into Disnix 0.9 that was released in September 2019.

An experimental process management framework

In addition to the release of Disnix 0.9 that contains a number of very big internal improvements, I have started working on a new experimental Nix-based process management framework.

I wrote a blog post about the first milestone: a functional organization that makes it possible to construct multiple instances of managed processes, that can be deployed by privileged and unprivileged users.

Miscellaneous

Earlier this year, I was also involved in several discussions about developer motivation. Since this is a common and recurring subject in the developer community, I have decided to write about my own experiences.

Some thoughts

Compared to previous years, this year is my least productive from a blogging perspective -- this can be attributed to several reasons. First and foremost, there are a number of major personal and work-related events that contributed to a drop in productivity for a while.

Second, some of the blog posts that I wrote are results of very big undertakings -- for example, the internal improvements to Disnix took me several months to complete.

As an improvement for next year, I will try to more carefully decompose my tasks, prioritize and order them in a better way. For example, implementing the architecture documentation tools was unnecessarily difficult before NixXML and the revised data models were implemented. I should have probably done these tasks in the opposite order.

Overall top 10 of my blog posts

As with previous years, I am going to publish the top 10 of my most frequently read blog posts. Surprisingly, not much has changed compared to last year:

1. Managing private Nix packages outside the Nixpkgs tree. As with last year, this blog post remains to be the most popular. I believe this can be attributed to the fact that there is still no official, simple hands-on tutorial avaialble that allows users to easily experiment with building packages.
2. On Nix and GNU Guix. This blog post used to be my most popular blog since 2012, but has dropped to the second place last year. I think this shows that the distinction between Nix and GNU Guix has become more clear these days.
3. An evaluation and comparison of Snappy Ubuntu. Still remains very popular, but I have not heard much from Snappy lately so it still remains a bit of a mystery why it remains third.
4. Setting up a multi-user Nix installation on non-NixOS systems. Some substantial improvements have been made to support multi-user installations on other operating systems than NixOS, but I believe we can still do better. There are some ongoing efforts to improve the Nix installer even further.
5. Yet another blog post about Object Oriented Programming and JavaScript. This blog post is still at the same spot compared to last year, and still seems to be read quite frequently. It seems that documenting my previous misunderstandings are quite helpful.
6. An alternative explanation of the Nix package manager. This blog post is a somewhat unconventional Nix explanation recipe, that typically has my preference. It seems to catch quite a bit of attention.
7. On NixOps, Disnix, service deployment and infrastructure deployment. This blog post is still as popular as last year. It seems that it is quite frequently used as an introduction to NixOps, although I did not write this blog post for that purpose.
8. Asynchronous programming with JavaScript. A blog post I wrote several years ago, when I was learning about Node.js and asynchronous programming concepts. It still seems to remain popular even though the blog post is over six years old.
9. Composing FHS-compatible chroot environments with Nix (or deploying Steam in NixOS). This blog post is still read quite frequently, probably because of Steam, which is a very popular application for gamers.
10. Auto patching prebuilt binary software packages for deployment with the Nix package manager. This is the only new blog post in the overall top 10 compared to last year. Patching binaries to deploy software on NixOS is a complicated subject. The popularity of this blog post proves that developing solutions to make this process more convenient is valuable.

Conclusion

As with previous years, I am still not out of ideas so stay tuned!

There is one more think I'd like to say and that is:

HAPPY NEW YEAR!!!!!!!!!!!