Deploying .NET services with Disnix

In two earlier blog posts, I have explained how the the Nix package manager can be used to deploy .NET software. I'm happy to report that I have extended these possibilities to Disnix.

With these new features it is possible to develop databases for Microsoft SQL server, implement WCF web services and ASP.NET web applications (which may be inter-connected to each other) and to automatically and reliably deploy them in a network of machines using Disnix.

Modifications to Disnix

The modifications I had to make to Disnix were relatively minor. Disnix can already be compiled on Cygwin, just as the Nix package manager. Since Disnix is built on top of Nix, it reuses the Nix functions I have developed in earlier blog posts to build Visual Studio projects.

The only missing piece in the deployment process, is the activation and deactivation of Microsoft SQL server databases and ASP.NET web applications on Internet Information Services (IIS) for which activation scripts must be developed. Luckily, Microsoft SQL server and IIS have command-line tools which can be scripted, to do this job.

To support these new types of services, I have developed the following activation scripts:

• mssql-database. This activation script loads a schema on initial startup if the database does not exists. It uses the OSQL.EXE tool included with Microsoft SQL server, to automatically execute the SQL instructions from shell scripts to check whether the database exists and to create the tables if needed.
• iis-webapplication. This activation script activates or deactivates a web application on Microsoft Internet Information Services. It uses the MSDeploy.exe tool to automatically activate a web application or deactivate a web application.

These activation scripts are automatically used by assigning a mssql-database or iis-webapplication type to a service in the Disnix services model.

Installing Disnix on Windows

Important is to know how to get Disnix working on Windows and how to enable support for .NET applications. Most of the installation steps on Windows/Cygwin are the same as UNIX systems. Details of the Disnix installation process can be found in the Disnix documentation.

However, there are a number of details that must be taken care of, which are not described in the manual (yet). Furthermore, the best way to get Disnix working is by compiling it from source so that all the required optional features are enabled.

In order to enable the mssql-database and iis-webapplication activation types, you must first manually install SQL server and IIS on your Windows system:

• I have used the following instructions to install Microsoft SQL server: http://visualcsharptutorials.com/2011/05/installing-sql-server-2008-express. Important is that you need to configure a SQL server user account with administration rights, a.k.a. the 'sa' user, which must use SQL server authentication. More information about this can be found here: http://www.eukhost.com/forums/f15/login-failed-user-sa-microsoft-sql-server-error-18456-a-12544/
• Microsoft IIS was already installed on my machine, but perhaps this page can act as a reference: http://learn.iis.net/page.aspx/28/installing-iis-on-windows-vista-and-windows-7. Important is that you include the Web Deployment tool, which is not enabled by default.

Moreover, the configure script of the disnix-activation-scripts package must be able to find OSQL.EXE and MSDeploy command-line tools, which must be in your PATH. Otherwise the activation types that we need are disabled and we cannot deploy .NET applications.

On my Windows 7 machine, OSQL can be found in: C:\Program Files\Microsoft SQL Server\100\Tools\Binn and MSDeploy in: C:\Program Files\IIS\Microsoft Web Deploy. I have included a screenshot above, which shows you what the output should of the configure script should look like. As you can see, the configure script was able to detect the locations of the command line tools, because I have adapted the PATH environment variable.

Running the Disnix daemon

We also need to run the Disnix daemon on every machine in the network, so that we can remotely deploy the services we want. Probably the best way to get Cygwin services running is by using the cygrunsrv command, which runs Cygwin programs as Windows services.

Since the core Disnix daemon is a D-Bus service, we need to run the D-Bus system daemon, which can be configured by typing:

$ cygrunsrv -I dbus -p /usr/sbin/dbus-daemon.exe -a \
    '--system --nofork'

The Disnix service can be configured by typing:

$ cygrunsrv -I disnix -p /usr/local/bin/disnix-service.exe -a \
  '--activation-modules-dir /usr/local/libexec/disnix/activation-scripts' \
  -e 'PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin' \
  -y dbus

Disnix also needs to be remotely connectible. Disnix can use various interfaces, but the recommended interface is SSH. In order to connect through SSH, you also need to configure a SSH server. This can be done by executing the following script on Cygwin:

$ ssh-host-config

And you probably need to configure some SSH keys as well, to prevent Disnix asking for passwords for each operation. Check the OpenSSH documentation for more information.

After configuring the services, you probably need to activate them for the fist time, which can be done by the Windows service manager (Control Panel -> System and Security -> Administrative Tools -> Services). You need to pick the Disnix service and select the start option. If you want to use the SSH server, you need to pick and start the 'CYGWIN sshd' service as well. A screenshot is included above.

Example case

Now that I have explained how Disnix can be installed and configured on Windows, we probably also like to see what it's capabilities are. As an example case, I have ported the StaffTracker, a motivating example in our WASDeTT paper, from Java to .NET technology, using C# as an implementation language, ADO.NET as database manager, WCF to implement web services, and ASP.NET to create the web application front-end.

It was a nice opportunity to learn some of these technologies. I have to admit that Visual Studio 2010 was a very convenient development environment and it didn't take much time for me to port the example case. Although I was impressed by this, I currently have no plans to write any software using .NET technology except for this example case. (Perhaps I will port it to Mono as an experiment some day).

Distributed deployment

In order to make our example deployable through Disnix, I had to write Disnix expressions for each service component and I had to write a services, infrastructure and distribution model. A Disnix expression for a WCF web service looks like this:

{dotnetenv}:
{zipcodes}:

dotnetenv.buildSolution {
  name = "ZipcodeService";
  src = ../../../../services/webservices/ZipcodeService;
  baseDir = "ZipcodeService";
  slnFile = "ZipcodeService.csproj";
  targets = "Package";
  preBuild = ''
    sed -e 's|.\SQLEXPRESS|${zipcodes.target.hostname}\SQLEXPRESS|' \
        -e 's|Initial Catalog=zipcodes|Initial catalog=${zipcodes.name}|' \
        -e 's|User ID=sa|User ID=${zipcodes.target.msSqlUsername}|' \
        -e 's|Password=admin123$|Password=${zipcodes.target.msSqlPassword}|' \
        Web.config
  '';
}

As you may notice, the expression above looks similar to an ordinary Nix expression building a Visual Studio project, except that it uses the inter-dependency parameter (zipcodes) to configure a database connection string defined in the Web.config file, so that the web service can connect to its database back-end. More information about setting a connection string through this configuration file, can be found here: http://weblogs.asp.net/owscott/archive/2005/08/26/Using-connection-strings-from-web.config-in-ASP.NET-v2.0.aspx.

By using the Disnix expressions in conjunction with the services, infrastructure and distribution models, the .NET example can be deployed in a network of machines with a single command line instruction:

$ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix

Below, I have a included a screenshot showing a Disnix deployment scenario with our .NET Staff Tracker example. In this screenshot, you can see a console showing Disnix output, a web browser displaying the entry page of the web application front-end, the IIS manager showing various deployed WCF web services, and the ASP.NET web front-end and the SQL server management studio showing a number of deployed databases. The complete system is deployed using a single command-line instruction.

Limitations

In this blog post I have shown how service-oriented .NET applications can be automatically deployed with Disnix. There are several slight inconveniences however:

• Disnix is used to manage the service components of a system. Disnix does not deploy infrastructure components, such as web server or database server. DisnixOS is a NixOS based extension that takes care of this. However, DisnixOS cannot be used on Windows, because SQL server and IIS are tightly integrated into the Windows operating system and registry. We cannot use the Nix store to safely isolate them. You need to either install these infrastructure components manually or use other deployment solutions.
• As mentioned in our previous blog post about .NET deployment, the .NET framework needs to be installed manually in order to be able to build Visual Studio projects. On most Windows installations, however, the .NET framework is already included.
• The .NET build functions and activation scripts are quite new and not very well tested. Moreover, they could also break, because we currently have no way to automatically test them like we do with Linux software.
• Also not all desired deployment features may be supported. I'd like to have feedback on this :-)
  

References

• To use the .NET deployment features described in this blog post, you need to obtain the latest pre-release of Disnix, which can be downloaded from the Disnix Hydra project page
• The StaffTracker .NET example case, can also be downloaded from the Disnix Hydra project page and can be freely used under the MIT license.
• A description of the StaffTracker example case can be found in our WASDeTT paper, titled: 'Disnix: A toolset for distributed deployment' which can be obtained from my publications page.
• I have used the following web page as a reference for writing WCF web services: http://www.codeproject.com/KB/WCF/first_WCF_Service.aspx
• The following page is used as a reference for ADO.NET: http://www.csharp-station.com/Tutorials/AdoDotNet/Lesson02.aspx
• The W3Schools tutorial about ASP.NET was sufficient for me, to understand it.
• I have recently presented this subject at Philips. The slides can be found on my talks page.

Deploying .NET applications with the Nix package manager (part 2)

In my previous blog post, I have explained how the Nix package manager can be used to deploy .NET applications. One of the open issues was that run-time dependencies can't be resolved in a convenient way. I have explained three possible solutions, each having its pros and cons and none of them was ideal.

After writing that blog post, I have received a number of suggestions and reactions from people from the #nixos freenode channel. Moreover, during the dinner of the SEAMS symposium in Hawaii, I have heard similar suggestions. It seems that blogging about certain issues pays off after all!

The last two days I have decided to look at these suggestions and to do some experiments at Philips. I'm happy to report that I have a follow up story now, in which I have a new solution for resolving run-time dependencies of .NET executables. This solution is also the best option (in my opinion).

Implementing a wrapper for .NET applications

Apparently .NET has a reflection API. With this reflection API you can dynamically load classes and dynamically invoke methods. You can also load assemblies dynamically from any location whether they have a strong name or not.

The .NET runtime also seems to fire an AssemblyResolve event, in case a library assembly can't be found. Apparently you can create your own event handler, dealing with such an event and use it to load a missing assembly through the reflection API.

So by taking these features into account, it is possible to create a wrapper executable capable of resolving the run-time dependencies that we need. This is what the wrapper I have developed for Nix looks like (I actually had to write some C# code for this):

using System;
using System.Reflection;
using System.IO;

namespace HelloWorldWrapper
{
    class HelloWorldWrapper
    {
        private String[] AssemblySearchPaths = {
          @"C:\cygwin\nix\store\23ga...-ALibrary",
          @"C:\cygwin\nix\store\833p...-BLibrary"
        };

        private String ExePath =
          @"C:\cygwin\nix\store\27f2...-Executable\Executable.exe";

        private String MainClassName =
          "SomeExecutable.Executable";

        public HelloWorldWrapper(string[] args)
        {
            // Attach the resolve event handler to the AppDomain
            // so that missing library assemblies will be searched
            AppDomain currentDomain = AppDomain.CurrentDomain;
            currentDomain.AssemblyResolve +=
              new ResolveEventHandler(MyResolveEventHandler);

            // Dynamically load the executable assembly
            Assembly exeAssembly = Assembly.LoadFrom(ExePath);

            // Lookup the main class
            Type mainClass = exeAssembly.GetType(MainClassName);

            // Lookup the main method
            MethodInfo mainMethod = mainClass.GetMethod("Main");

            // Invoke the main method
            mainMethod.Invoke(this, new Object[] {args});
        }

        static void Main(string[] args)
        {
            new HelloWorldWrapper(args);
        }

        private Assembly MyResolveEventHandler(object sender,
          ResolveEventArgs args)
        {
            // This handler is called only when the common language
            // runtime tries to bind to the assembly and fails.

            Assembly MyAssembly;
            String assemblyPath = "";
            String requestedAssemblyName =
              args.Name.Substring(0, args.Name.IndexOf(","));

            // Search for the right path of the library assembly
            foreach (String curAssemblyPath in AssemblySearchPaths)
            {
                assemblyPath = curAssemblyPath + "/" +
                  requestedAssemblyName + ".dll";

                if (File.Exists(assemblyPath))
                    break;
            }

            // Load the assembly from the specified path. 
            MyAssembly = Assembly.LoadFrom(assemblyPath);

            // Return the loaded assembly.
            return MyAssembly;
        }

    }
}

The wrapper class defined above has a number of fields. The AssemblySearchPaths field defines a String array containing all the Nix store paths of the runtime dependencies. The exePath field defines a String referring to the path of the executable in the Nix store which we want to run. The MainClassName field defines the full name of the class containing the Main method we want to run.

In the Main method of this class, we create an instance of the wrapper. In the constructor, we attach our own custom resolve event handler to the app domain controller. Then we use the reflection API to dynamically load the actual executable and to invoke the main method in the specified main class.

When we load the executable assembly, the resolve event handler is triggered a number of times. Our custom MyResolveEvent handler, tries to load to given assembly in all the search paths defined in the AssemblySearchPaths string array, which should succeed if all runtime dependencies are present.

There is a small caveat, however, with dynamically invoking the Main method of another executable. By default, the Main method in C# programs is defined like this:

namespace SomeNamespace
{
    class SomeClass
    {
       static void Main(string[] args)
       {
       }
    }
}

Apparently, it has no access modifier, which means the internal access modifier is used by default. The internal access modifier restricts access to this method to all members of the same assembly. This means that we cannot invoke an external Main method from a different assembly like the wrapper. To counter this, we need to make the access modifier of the actual executable public (or make the wrapper class a friend, but nonetheless we need to make a small modification anyway).

Usage

I have implemented a convenience function in Nixpkgs: dotnetenv.buildWrapper that automatically builds a .NET executable and generates a wrapper for the given executable. The function can be invoked like this:

{dotnetenv, MyAssembly1, MyAssembly2}:

dotnetenv.buildWrapper {
  name = "My.Test.Assembly";
  src = /path/to/source/code;
  slnFile = "Assembly.sln";
  assemblyInputs = [
    dotnetenv.assembly20Path
    MyAssembly1
    MyAssembly2
  ];
  namespace = "TestNamespace";
  mainClassName = "MyTestApplication";
  mainClassFile = "MyTestApplication.cs";
}

As you may see, the structure of the dotnetenv.buildWrapper is similar to the dotnetenv.buildSolution function, except that it requires several additional parameters for the wrapper, such as the namespace, class name and file location of the class containing the Main method of the actual executable. The function automatically makes the given Main method in the given main class file public and it creates a wrapper class containing the right properties to run the actual executable, such as the location of the actual executable and the paths of the run-time dependencies.

By using this wrapper function, it is possible to run a .NET executable assembly from the Nix store without much trouble.

Conclusion

In this blog post, I have implemented a wrapper executable that deals with resolving run-time dependencies of a .NET application. The wrapper uses a resolve event handler which loads all the required library assemblies through the .NET reflection API. This wrapper can be automatically generated from a convenience function, which makes it possible to run .NET applications from the Nix store, without much trouble.

References

• This page explains how the reflection API can be used to dynamically load assemblies: http://www.codeproject.com/KB/cs/DynLoadClassInvokeMethod.aspx
• The following page explains various ways to load library assemblies beyond the base directory of the executable. It also includes an explanation of the AssemblyResolve event: http://support.microsoft.com/kb/837908

Deploying .NET applications with the Nix package manager

This probably sounds like a very strange topic to some (or perphaps, most) readers, but I have done some experiments in the past with deploying .NET applications by using the Nix package manager. The Nix package manager is mostly used on Unix-like systems (Linux, FreeBSD, etc.) and designed with Unix-principles in mind. Furthermore, a lot of people know me as a Microsoft-critic. So you probably wonder why I want to do this?

Motivation

Being able to use Nix for deploying .NET applications has the following benefits:

• For installing or upgrading .NET applications you have the same deployment benefits that Nix has: Being able to store multiple versions/variants next to each other, dependency completeness, atomic upgrades and rollbacks and a garbage collector which safely removes components no longer in use.
• You can use Hydra, our continuous build and integration server, for building and testing .NET applications in various environments including environmental dependencies
• You can use Disnix, to manage the deployment of a service-oriented applications developed using .NET technology in a network machines. This also works for web applications. For example, you can deploy your ASP.NET / Microsoft SQL server database environment from a declarative specification. Because Disnix is built on top of Nix, it also provides features such as dependency completeness, (almost) atomic upgrades and a garbage collector in a distributed environment.
• The Nix deployment technology and related tooling are designed as generic tools (i.e. not developed for a particular component technology). Being able to support .NET applications is a useful addition.
• And finally, we have an industry partner in our research project, who's interested in this.

Global Assembly Cache (GAC)

When I talk about Nix (and especially about the principle of the Nix store) to .NET people, I often hear that the Global Assembly Cache (GAC) already solves the DLL-hell, so you have no worries. Although the GAC solves several common deployment issues, it has a number of drawbacks compared to the Nix store:

• It only provides isolation for library assemblies. Other components such as executables, compilers, configuration files, or native libraries are not supported.
• A library assembly must have a strong name, which gives a library an unique name. A strong name is composed of several attributes, such as a name, version number and culture. Furthermore, the library assembly is signed with a public/private key pair.
• Creating a strong-named assembly is in many cases painful. A developer must take care that the combination of attributes is always unique. For example, for a new release the version number must be increased. Because developers have to take care of this, people typically don't use strong names for internal release cycles, because it's too much work.
• Creating a strong-named assembly could go wrong. It may be possible that a developer forgets to update any of these strong name attributes, which makes it possible to create a different assembly with the same strong name. Then isolation in the GAC can't be provided.

In contrast to the GAC, you can store any type of component in the Nix store, such as executables, configuration files, compilers etc. Furthermore, the Nix store uses hash codes derived from all build-time dependencies of a component, which always provides unique component file names.

Building Visual Studio projects in Nix

So how can Visual Studio projects be supported in Nix to compile .NET applications? We have implemented a Nix function to support this:

{stdenv, dotnetfx}:
{ name, src, slnFile, targets ? "ReBuild"
, options ? "/p:Configuration=Debug;Platform=Win32"
, assemblyInputs ? []
}:
stdenv.mkDerivation {
  inherit name src;
  buildInputs = [ dotnetfx ];
  installPhase = ''
    for i in ${toString assemblyInputs}; do
      windowsPath=$(cygpath --windows $i)
      AssemblySearchPaths="$AssemblySearchPaths;$windowsPath"
    done
    export AssemblySearchPaths
    ensureDir $out
    outPath=$(cygpath --windows $out)\\
    MSBuild.exe ${slnFile} /nologo /t:${targets} \
      /p:OutputPath=$outPath ${options} ...
  '';
}

The Nix expression code fragment above shows you the definition of the dotnetenv.buildSolution function, which builds Visual Studio projects and stores the output in the Nix store.

The idea of this function is easy: The function takes several parameters, such as the name of the component, a build time options string, the filename of the Visual Studio solution file (SLN) and a list of libraries (Assembly Inputs). It uses the dotnetfx (.NET framework) as a buildtime dependency, which provides access to the MSBuild executable, used to build Visual Studio solution files.

In order to let MSBuild find its library dependencies, we set the AssemblySearchPaths environment variable to contain the paths to the Nix store components containing the library assemblies. After setting the environment variable, the MSBuild command is invoked to build the given solution file and to produce the output in a unique path in the Nix store. The cygpath command is used to convert UNIX path names to Windows path names (and vice versa).

{dotnetenv, MyAssembly1, MyAssembly2}:

dotnetenv.buildSolution {
  name = "My.Test.Assembly";
  src = /path/to/source/code;
  slnFile = "Assembly.sln";
  assemblyInputs = [
    dotnetenv.assembly20Path
    MyAssembly1
    MyAssembly2
  ];
}

The above Nix expression shows you how this function can be used to build a Visual Studio project. Like ordinary Nix expressions, this expression is also a function taking several input arguments, such as dotnetenv which provides the Visual Studio build function (shown in the previous code fragment) and the library assemblies which are required to build the project. In the body we call the buildSolution function with the right parameters, such as the Solution file and the library assemblies which this project requires. The dotnetenv.assembly20Path refers to the .NET 2.0 system assemblies directory.

rec {
  dotnetfx = ...
  stdenv = ...
  dotnetenv = import ../dotnetenv {
    inherit stdenv dotnetfx;
  };

  MyAssembly1 = import ../MyAssembly1 {
    inherit dotnetenv;
  };

  MyAssembly2 = import ../MyAssembly1 {
    inherit dotnetenv;
  };

  MyTestAssembly = import ../MyTestAssembly {
    inherit dotnetenv MyAssembly1 MyAssembly2;
  };
}

Like ordinary Nix expressions, we also have to compose Visual Studio components by calling the build function in the previous code fragment with the right parameters. This is done in the Nix expression shown above. The last attribute: MyTestAssembly imports the expression shown in the previous code fragement with the required function arguments. As you may see, also all dependencies of MyTestAssembly are defined in this file. By running the following command-line instruction (pkgs.nix is the filename of the code fragement above):

nix-env -f pkgs.nix -iA MyTestAssembly

The assembly in our example is build from source, including all library dependencies and the output is produced in:

/nix/store/ri0zzm2hmwg01w2wi0g4a3rnp0z24r8p-My.Test.Assembly

Running .NET applications from the Nix store

We have explained how can build .NET applications and how MSBuild is able to find the required build time dependencies. Except for building a .NET application with Nix, we also have to be able to run them from the Nix store. To make this possible, an executable assembly needs to find its runtime dependencies, which is more complicated than I thought.

The .NET runtime locates assemblies as follows:

• First, it tries to determine the correct version of the assembly (only for strong named assemblies)
• If the strong named assembly has been bound before in memory, that version will be used.
• If the assembly is not already in memory, it checks the Global Assembly Cache (GAC).
• And otherwise it probes the assembly, by looking in a config file or by using some probing heuristics.

Because Nix stores all components, including library assemblies, in unique folders in Nix store, this gives some challenges. If an executable is started from the Nix store, the required libraries can't be found, because probing heuristics look for libraries in the same basedir as the executable.

Currently, I have implemented three methods to resolve runtime dependencies (each approach has its pros and cons and none of them is ideal):

• Copying DLLs into the same folder as the executable. This is the most expensive and inefficient method, because libraries are not shared on the hard drive. However, it does work with both private and strong named assemblies and it also works on older versions of Windows, such as Windows XP.
• Creating a config file, which specifies where the libraries can be found. A disadvantage of this approach is that .NET does not allow private assemblies to be looked up in other locations beyond the basedir of the executable. Therefore assemblies need a strong name, which is not very practical because these have to be generated by hand.
• The third option is creating NTFS symlinks in the same folder as the executable. This works also for private libraries. A disadvantage of this approach is that NTFS symlinks are only supported from Windows Vista and upwards. Furthermore, you need special user privileges to create them and their semantics are not exactly the same as UNIX symlinks.

Usage

If you want to experiment with the Visual Studio build functions in Nix, you need to install Nix on Cygwin and you need a checkout of Nixpkgs. Check the Nix documentation for more instructions on this.

The dotnetenv component can be found in the pkgs/buildsupport/ directory of Nixpkgs. You need to install the .NET framework yourself and you have to edit some of the attributes of dotnetenv so that the .NET framework utilities can be found.

Unfortunately, the .NET framework can't be deployed through Nix (yet), because of dependencies on the Windows register. I also haven't looked into the scripting possibilities of the installer. So the .NET framework deployment isn't entirely pure. (Perhaps someone is able to make some tweaks to do this, but I don't know if this is legal to do).

Conclusion

In this blog post, I have described how .NET applications can be deployed with Nix. However, there are some minor issues, such as the fact that runtime dependencies can't be resolved in a convenient way. Furthermore, the deployment isn't entirely pure as the .NET framework must be installed manually. I know that Mono is able to use the MONO_PATH environment variable to look for libraries in arbitrary locations. Unfortunately, it seems that the .NET framework does not have something like this.

I have been told that it's also possible to resolve run time dependencies programmatically. This way you can load any library assembly you want from any location. I'm curious if somebody has more information on this. Any feedback would be welcome, since I'm not a .NET expert.

References

• I have given a presentation about this subject some time ago. Slides can be found on my presentations page.
• The following page explains how .NET locates assemblies: http://msdn.microsoft.com/en-us/library/yx7xezcf%28VS.71%29.aspx