A Reference Architecture for Distributed Software Deployment

Today, I'm happy to announce the title and contents of my PhD dissertation on this blog!

Background

I have written many software deployment related topics on my blog. Earlier, I have explained that certain developments in software engineering, such as component-based software engineering (CBSE), have increased the quality of software systems, but also have significantly increased the complexity of deployment processes of software systems. Moreover, I have written about solutions, typically involving tools from the Nix project.

Before I started my research as PhD (and master's) student, we already had the following deployment tools in the Nix project:

• The Nix package manager for deployment of individual packages from declarative specifications.
• NixOS, a Linux distribution completely built around the Nix package manager. NixOS deploys entire system configurations from a single declarative specification.
• Hydra, formerly known as just the "Nix build farm". Hydra builds jobsets from declarative specifications.

Distributed software deployment

As you may notice, these tools all have something in common: to fully automate software deployment processes from declarative specifications, in a reliable, reproducible, generic and efficient way. Their difference is the level in which they operate.

However, to support modern generation systems, typically exposed as services through the Internet, these tools are not sufficient:

• We have to deploy multiple machines in a network, instead of a single machine.
• Components may have dependencies on other components residing on different machines in the network, as well as system configurations (called inter-dependencies in this thesis).
• Components need underlying infrastructure components, e.g. an application server or a DBMS.
• Upgrading these systems may give a large time window in which a system is inconsistent.
• We have to meet important non-functional requirements, e.g. performance, security, privacy and the license under which components are governed.

To support these new generation of systems, we have developed a collection of new tools in the Nix project:

• Disnix, to deploy services (such as web services, web applications, UNIX processes etc) inside networks of machines from declarative specifications.
• Dynamic Disnix, to automatically redeploy services through Disnix based on technical and non-functional constraints if an event occurs in the network, e.g. machine crash.
• NixOps (formerly known as Charon and nixos-deploy-network) and the NixOS test driver, to deploy and test networks of NixOS configurations from declarative specifications.
• Dysnomia, a prototype-tool to deploy mutable software components.
• grok-trace, a prototype-tool to dynamically analyze build processes to discover license constraints.

These tools solve various distributed deployment aspects having similar quality attributes as the older Nix tools and make it possible to support some important non-functional requirements.

A Reference Architecture for Distributed Software Deployment

However, to fully automate deployments of service-oriented systems, we must combine these tools implementing generic deployment aspects with components supporting important non-functional requirements of a domain, such as performance, security and privacy. However, there is no silver bullet that solves all these problems for any domain, as some non-functional requirements are more important in one domain than another, and some may conflict with each other.


Instead, we have designed a reference architecture that can be used to construct a concrete architecture for a domain-specific deployment tool, capable of fully automating a deployment process with its desired quality attributes.

My thesis gives some background information about software deployment in general and Nix, it proposes a reference architecture for distributed software deployment, it describes most of the newer Nix components, and shows how these can be applied. Finally, we reflect on how well we have achieved our desired quality attributes.

Availability

My PhD thesis can be obtained from the PhD thesis page of my homepage. Moreover, the defence will be at June 3, 2013 14:30 in the Aula Congresscentre of Delft University of Technology. If you're interested in attending my defence, please let me know!