Furthermore, I raised a couple of questions and intentionally left one unmentioned question unanswered. The unmentioned question that I was referring to is the following: "Is Disnix still alive or is it dead?".
The answer is that Disnix's development was progressing at a very low pace for some time after I left academia -- I made minor changes once in a while, but nothing really interesting happened.
However, for the last few months, I am using it on a daily basis and made many big improvements. Moreover, I have reached a stable point and decided that this is a good moment to announce the next release!
New features
So what is new in this release?
Visualization tool
I have added a new tool to the Disnix toolset named: disnix-visualize that generates Graphviz images to visualize a particular deployment scenario. An example image is shown below:
The above picture shows a deployment scenario of the StaffTracker Java example in which services are divided over two machines in a network and have all kinds of complex dependency relationships as denoted by the arrows.
The tool was already included in the development versions for a while, but has never been part of any release.
Dysnomia
I also made a major change in Disnix's architecture. As explained in my old blog post about Disnix, activating and deactivating services cannot be done generically and I have developed a plugin system to take care of that.
This plugin system package (formerly known as Disnix activation scripts) has now become an independent tool named Dysnomia and can also be applied in different contexts. Furthermore, it can also be used as a standalone tool.
For example, a running MySQL DBMS instance (called a container in Dysnomia terminology) could be specified in a configuration file (such as ~/mysql-production) as follows:
type=mysql-database mysqlUsername=root mysqlPassword=verysecret
A database can be encoded as an SQL file (~/test-database/createdb.sql) creating the schema:
create table author
( AUTHOR_ID INTEGER NOT NULL,
FirstName VARCHAR(255) NOT NULL,
LastName VARCHAR(255) NOT NULL,
PRIMARY KEY(AUTHOR_ID)
);
create table books
( ISBN VARCHAR(255) NOT NULL,
Title VARCHAR(255) NOT NULL,
AUTHOR_ID VARCHAR(255) NOT NULL,
PRIMARY KEY(ISBN),
FOREIGN KEY(AUTHOR_ID) references author(AUTHOR_ID)
on update cascade on delete cascade
);
We can use the following command-line instruction to let Dysnomia deploy the database to the MySQL DBMS container we have just specified earlier:
$ dysnomia --operation activate --component ~/test-database \ --container ~/mysql-production
When Disnix has to execute deployment operations, two external tools are consulted -- Nix takes care of all deployment operations of the static parts of a system, and Dysnomia takes care of performing the dynamic activation and deactivation steps.
Concurrent closure transfers
In the previous versions of Disnix, only one closure (of a collection of services and its intra-dependencies) is transferred to a target machine at the time. If a target machine has more network bandwidth than the coordinator, this is usually fine, but in all other cases, it slows the deployment process down.
In the new version, two closures are transferred concurrently by default. The amount of concurrent closure transfers can be adjusted as follows:
$ disnix-env -s services.nix -i infrastructure.nix \ -d distribution.nix --max-concurrent-transfers 4
The last command-line argument states that 4 closures should be transferred concurrently.
Concurrent service activation and deactivation
In the old Disnix versions, the activation and deactivation steps of a service on a target machine were executed sequentially, i.e. one service on a machine at the time. In all my old testcases these steps were quite cheap/quick, but now that I have encountered systems that are much bigger, I noticed that there is a lot of deployment time that we can save.
In the new implementation, Disnix tries to concurrently activate or deactivate one service per machine. The amount of services that can be concurrently activated or deactivated per machine can be raised in the infrastructure model:
{
test1 = {
hostname = "test1";
numOfCores = 2;
};
}
In the above infrastructure model, the numOfCores attribute states that two services can be concurrently activated/deactivated on machine test1. If this attribute has been omitted, it defaults to 1.
Multi connection protocol support
By default, Disnix uses an SSH protocol wrapper to connect to the target machines. There is also an extension available, called DisnixWebService, that uses SOAP + MTOM instead.
In the old version, changing the connection protocol means that every target machine should be reached with it. In the new version, you can also specify the target property and client interface in the infrastructure model to support multi connection protocol deployments:
{
test1 = {
hostname = "test1";
targetProperty = "hostname";
clientInterface = "disnix-ssh-client";
};
test2 = {
hostname = "test2";
targetEPR = http://test2:8080/DisnixWebService/services/DisnixWebService;
targetProperty = "targetEPR";
clientInterface = "disnix-soap-client";
};
}
The above infrastructure model states the following:
- To connect to machine: test1, the hostname attribute contains the address and the disnix-ssh-client tool should be invoked to connect it.
- To connect to machine: test2, the targetEPR attribute contains the address and the disnix-soap-client tool should be invoked to connect it.
NixOps integration
As described in my previous blog post, Disnix does service deployment and can integrate NixOS' infrastructure deployment features with an extension called DisnixOS.
DisnixOS can now also be used in conjunction with NixOps -- NixOps can be used to instantiate and deploy a network of virtual machines:
$ nixops create ./network.nix ./network-ec2.nix -d ec2 $ nixops deploy -d ec2
and DisnixOS can be used to deploy services to them:
$ export NIXOPS_DEPLOYMENT=ec2
$ disnixos-env -s services.nix -n network.nix -d distribution.nix \
--use-nixops
Omitted features
There are also a couple of features described in some older blog posts, papers, and my PhD thesis, which have not become part of the new Disnix release.
Dynamic Disnix
This is an extended framework built on top of Disnix supporting self-adaptive redeployment. Although I promised to make it part of the new release a long time ago, it did not happen. However, I did update the prototype to work with the current Disnix implementation, but it still needs refinements, documentation and other small things to make it usable.
Brave people who are eager to try it can pull the Dynamic Disnix repository from my GitHub page.
Snapshot/restore features of Dysnomia
In a paper I wrote about Dysnomia I also proposed state snapshotting/restoring facilities. These have not become part of the released versions of Dysnomia and Disnix yet.
The approach I have described is useful in some scenarios but also has a couple of very big drawbacks. Moreover, it also significantly alters the behavior of Disnix. I need to find a way to properly integrate these features in such a way that they do not break the standard approach. Moreover, these techniques must be selectively applied as well.
Conclusion
In this blog post, I have announced the availability of the next release of Disnix. Perhaps I should give it the codename: "Disnix Forever!" or something :-). Also, the release date (Friday the 13th) seems to be appropriate.
Moreover, the previous release was considered an advanced prototype. Although I am using Disnix on a daily basis now to eat my own dogfood, and the toolset has become much more usable, I would not yet classify this release as something that is very mature yet.
Disnix can be obtained by installing NixOS, through Nixpkgs or from the Disnix release page.
I have also updated the Disnix homepage a bit, which should provide you more information.
No comments:
Post a Comment