Deploying .NET services with Disnix

In two earlier blog posts, I have explained how the the Nix package manager can be used to deploy .NET software. I'm happy to report that I have extended these possibilities to Disnix.

With these new features it is possible to develop databases for Microsoft SQL server, implement WCF web services and ASP.NET web applications (which may be inter-connected to each other) and to automatically and reliably deploy them in a network of machines using Disnix.

Modifications to Disnix

The modifications I had to make to Disnix were relatively minor. Disnix can already be compiled on Cygwin, just as the Nix package manager. Since Disnix is built on top of Nix, it reuses the Nix functions I have developed in earlier blog posts to build Visual Studio projects.

The only missing piece in the deployment process, is the activation and deactivation of Microsoft SQL server databases and ASP.NET web applications on Internet Information Services (IIS) for which activation scripts must be developed. Luckily, Microsoft SQL server and IIS have command-line tools which can be scripted, to do this job.

To support these new types of services, I have developed the following activation scripts:

• mssql-database. This activation script loads a schema on initial startup if the database does not exists. It uses the OSQL.EXE tool included with Microsoft SQL server, to automatically execute the SQL instructions from shell scripts to check whether the database exists and to create the tables if needed.
• iis-webapplication. This activation script activates or deactivates a web application on Microsoft Internet Information Services. It uses the MSDeploy.exe tool to automatically activate a web application or deactivate a web application.

These activation scripts are automatically used by assigning a mssql-database or iis-webapplication type to a service in the Disnix services model.

Installing Disnix on Windows

Important is to know how to get Disnix working on Windows and how to enable support for .NET applications. Most of the installation steps on Windows/Cygwin are the same as UNIX systems. Details of the Disnix installation process can be found in the Disnix documentation.

However, there are a number of details that must be taken care of, which are not described in the manual (yet). Furthermore, the best way to get Disnix working is by compiling it from source so that all the required optional features are enabled.

In order to enable the mssql-database and iis-webapplication activation types, you must first manually install SQL server and IIS on your Windows system:

• I have used the following instructions to install Microsoft SQL server: http://visualcsharptutorials.com/2011/05/installing-sql-server-2008-express. Important is that you need to configure a SQL server user account with administration rights, a.k.a. the 'sa' user, which must use SQL server authentication. More information about this can be found here: http://www.eukhost.com/forums/f15/login-failed-user-sa-microsoft-sql-server-error-18456-a-12544/
• Microsoft IIS was already installed on my machine, but perhaps this page can act as a reference: http://learn.iis.net/page.aspx/28/installing-iis-on-windows-vista-and-windows-7. Important is that you include the Web Deployment tool, which is not enabled by default.

Moreover, the configure script of the disnix-activation-scripts package must be able to find OSQL.EXE and MSDeploy command-line tools, which must be in your PATH. Otherwise the activation types that we need are disabled and we cannot deploy .NET applications.

On my Windows 7 machine, OSQL can be found in: C:\Program Files\Microsoft SQL Server\100\Tools\Binn and MSDeploy in: C:\Program Files\IIS\Microsoft Web Deploy. I have included a screenshot above, which shows you what the output should of the configure script should look like. As you can see, the configure script was able to detect the locations of the command line tools, because I have adapted the PATH environment variable.

Running the Disnix daemon

We also need to run the Disnix daemon on every machine in the network, so that we can remotely deploy the services we want. Probably the best way to get Cygwin services running is by using the cygrunsrv command, which runs Cygwin programs as Windows services.

Since the core Disnix daemon is a D-Bus service, we need to run the D-Bus system daemon, which can be configured by typing:

$ cygrunsrv -I dbus -p /usr/sbin/dbus-daemon.exe -a \
    '--system --nofork'

The Disnix service can be configured by typing:

$ cygrunsrv -I disnix -p /usr/local/bin/disnix-service.exe -a \
  '--activation-modules-dir /usr/local/libexec/disnix/activation-scripts' \
  -e 'PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin' \
  -y dbus

Disnix also needs to be remotely connectible. Disnix can use various interfaces, but the recommended interface is SSH. In order to connect through SSH, you also need to configure a SSH server. This can be done by executing the following script on Cygwin:

$ ssh-host-config

And you probably need to configure some SSH keys as well, to prevent Disnix asking for passwords for each operation. Check the OpenSSH documentation for more information.

After configuring the services, you probably need to activate them for the fist time, which can be done by the Windows service manager (Control Panel -> System and Security -> Administrative Tools -> Services). You need to pick the Disnix service and select the start option. If you want to use the SSH server, you need to pick and start the 'CYGWIN sshd' service as well. A screenshot is included above.

Example case

Now that I have explained how Disnix can be installed and configured on Windows, we probably also like to see what it's capabilities are. As an example case, I have ported the StaffTracker, a motivating example in our WASDeTT paper, from Java to .NET technology, using C# as an implementation language, ADO.NET as database manager, WCF to implement web services, and ASP.NET to create the web application front-end.

It was a nice opportunity to learn some of these technologies. I have to admit that Visual Studio 2010 was a very convenient development environment and it didn't take much time for me to port the example case. Although I was impressed by this, I currently have no plans to write any software using .NET technology except for this example case. (Perhaps I will port it to Mono as an experiment some day).

Distributed deployment

In order to make our example deployable through Disnix, I had to write Disnix expressions for each service component and I had to write a services, infrastructure and distribution model. A Disnix expression for a WCF web service looks like this:

{dotnetenv}:
{zipcodes}:

dotnetenv.buildSolution {
  name = "ZipcodeService";
  src = ../../../../services/webservices/ZipcodeService;
  baseDir = "ZipcodeService";
  slnFile = "ZipcodeService.csproj";
  targets = "Package";
  preBuild = ''
    sed -e 's|.\SQLEXPRESS|${zipcodes.target.hostname}\SQLEXPRESS|' \
        -e 's|Initial Catalog=zipcodes|Initial catalog=${zipcodes.name}|' \
        -e 's|User ID=sa|User ID=${zipcodes.target.msSqlUsername}|' \
        -e 's|Password=admin123$|Password=${zipcodes.target.msSqlPassword}|' \
        Web.config
  '';
}

As you may notice, the expression above looks similar to an ordinary Nix expression building a Visual Studio project, except that it uses the inter-dependency parameter (zipcodes) to configure a database connection string defined in the Web.config file, so that the web service can connect to its database back-end. More information about setting a connection string through this configuration file, can be found here: http://weblogs.asp.net/owscott/archive/2005/08/26/Using-connection-strings-from-web.config-in-ASP.NET-v2.0.aspx.

By using the Disnix expressions in conjunction with the services, infrastructure and distribution models, the .NET example can be deployed in a network of machines with a single command line instruction:

$ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix

Below, I have a included a screenshot showing a Disnix deployment scenario with our .NET Staff Tracker example. In this screenshot, you can see a console showing Disnix output, a web browser displaying the entry page of the web application front-end, the IIS manager showing various deployed WCF web services, and the ASP.NET web front-end and the SQL server management studio showing a number of deployed databases. The complete system is deployed using a single command-line instruction.

Limitations

In this blog post I have shown how service-oriented .NET applications can be automatically deployed with Disnix. There are several slight inconveniences however:

• Disnix is used to manage the service components of a system. Disnix does not deploy infrastructure components, such as web server or database server. DisnixOS is a NixOS based extension that takes care of this. However, DisnixOS cannot be used on Windows, because SQL server and IIS are tightly integrated into the Windows operating system and registry. We cannot use the Nix store to safely isolate them. You need to either install these infrastructure components manually or use other deployment solutions.
• As mentioned in our previous blog post about .NET deployment, the .NET framework needs to be installed manually in order to be able to build Visual Studio projects. On most Windows installations, however, the .NET framework is already included.
• The .NET build functions and activation scripts are quite new and not very well tested. Moreover, they could also break, because we currently have no way to automatically test them like we do with Linux software.
• Also not all desired deployment features may be supported. I'd like to have feedback on this :-)
  

References

• To use the .NET deployment features described in this blog post, you need to obtain the latest pre-release of Disnix, which can be downloaded from the Disnix Hydra project page
• The StaffTracker .NET example case, can also be downloaded from the Disnix Hydra project page and can be freely used under the MIT license.
• A description of the StaffTracker example case can be found in our WASDeTT paper, titled: 'Disnix: A toolset for distributed deployment' which can be obtained from my publications page.
• I have used the following web page as a reference for writing WCF web services: http://www.codeproject.com/KB/WCF/first_WCF_Service.aspx
• The following page is used as a reference for ADO.NET: http://www.csharp-station.com/Tutorials/AdoDotNet/Lesson02.aspx
• The W3Schools tutorial about ASP.NET was sufficient for me, to understand it.
• I have recently presented this subject at Philips. The slides can be found on my talks page.

Deploying .NET applications with the Nix package manager (part 2)

In my previous blog post, I have explained how the Nix package manager can be used to deploy .NET applications. One of the open issues was that run-time dependencies can't be resolved in a convenient way. I have explained three possible solutions, each having its pros and cons and none of them was ideal.

After writing that blog post, I have received a number of suggestions and reactions from people from the #nixos freenode channel. Moreover, during the dinner of the SEAMS symposium in Hawaii, I have heard similar suggestions. It seems that blogging about certain issues pays off after all!

The last two days I have decided to look at these suggestions and to do some experiments at Philips. I'm happy to report that I have a follow up story now, in which I have a new solution for resolving run-time dependencies of .NET executables. This solution is also the best option (in my opinion).

Implementing a wrapper for .NET applications

Apparently .NET has a reflection API. With this reflection API you can dynamically load classes and dynamically invoke methods. You can also load assemblies dynamically from any location whether they have a strong name or not.

The .NET runtime also seems to fire an AssemblyResolve event, in case a library assembly can't be found. Apparently you can create your own event handler, dealing with such an event and use it to load a missing assembly through the reflection API.

So by taking these features into account, it is possible to create a wrapper executable capable of resolving the run-time dependencies that we need. This is what the wrapper I have developed for Nix looks like (I actually had to write some C# code for this):

using System;
using System.Reflection;
using System.IO;

namespace HelloWorldWrapper
{
    class HelloWorldWrapper
    {
        private String[] AssemblySearchPaths = {
          @"C:\cygwin\nix\store\23ga...-ALibrary",
          @"C:\cygwin\nix\store\833p...-BLibrary"
        };

        private String ExePath =
          @"C:\cygwin\nix\store\27f2...-Executable\Executable.exe";

        private String MainClassName =
          "SomeExecutable.Executable";

        public HelloWorldWrapper(string[] args)
        {
            // Attach the resolve event handler to the AppDomain
            // so that missing library assemblies will be searched
            AppDomain currentDomain = AppDomain.CurrentDomain;
            currentDomain.AssemblyResolve +=
              new ResolveEventHandler(MyResolveEventHandler);

            // Dynamically load the executable assembly
            Assembly exeAssembly = Assembly.LoadFrom(ExePath);

            // Lookup the main class
            Type mainClass = exeAssembly.GetType(MainClassName);

            // Lookup the main method
            MethodInfo mainMethod = mainClass.GetMethod("Main");

            // Invoke the main method
            mainMethod.Invoke(this, new Object[] {args});
        }

        static void Main(string[] args)
        {
            new HelloWorldWrapper(args);
        }

        private Assembly MyResolveEventHandler(object sender,
          ResolveEventArgs args)
        {
            // This handler is called only when the common language
            // runtime tries to bind to the assembly and fails.

            Assembly MyAssembly;
            String assemblyPath = "";
            String requestedAssemblyName =
              args.Name.Substring(0, args.Name.IndexOf(","));

            // Search for the right path of the library assembly
            foreach (String curAssemblyPath in AssemblySearchPaths)
            {
                assemblyPath = curAssemblyPath + "/" +
                  requestedAssemblyName + ".dll";

                if (File.Exists(assemblyPath))
                    break;
            }

            // Load the assembly from the specified path. 
            MyAssembly = Assembly.LoadFrom(assemblyPath);

            // Return the loaded assembly.
            return MyAssembly;
        }

    }
}

The wrapper class defined above has a number of fields. The AssemblySearchPaths field defines a String array containing all the Nix store paths of the runtime dependencies. The exePath field defines a String referring to the path of the executable in the Nix store which we want to run. The MainClassName field defines the full name of the class containing the Main method we want to run.

In the Main method of this class, we create an instance of the wrapper. In the constructor, we attach our own custom resolve event handler to the app domain controller. Then we use the reflection API to dynamically load the actual executable and to invoke the main method in the specified main class.

When we load the executable assembly, the resolve event handler is triggered a number of times. Our custom MyResolveEvent handler, tries to load to given assembly in all the search paths defined in the AssemblySearchPaths string array, which should succeed if all runtime dependencies are present.

There is a small caveat, however, with dynamically invoking the Main method of another executable. By default, the Main method in C# programs is defined like this:

namespace SomeNamespace
{
    class SomeClass
    {
       static void Main(string[] args)
       {
       }
    }
}

Apparently, it has no access modifier, which means the internal access modifier is used by default. The internal access modifier restricts access to this method to all members of the same assembly. This means that we cannot invoke an external Main method from a different assembly like the wrapper. To counter this, we need to make the access modifier of the actual executable public (or make the wrapper class a friend, but nonetheless we need to make a small modification anyway).

Usage

I have implemented a convenience function in Nixpkgs: dotnetenv.buildWrapper that automatically builds a .NET executable and generates a wrapper for the given executable. The function can be invoked like this:

{dotnetenv, MyAssembly1, MyAssembly2}:

dotnetenv.buildWrapper {
  name = "My.Test.Assembly";
  src = /path/to/source/code;
  slnFile = "Assembly.sln";
  assemblyInputs = [
    dotnetenv.assembly20Path
    MyAssembly1
    MyAssembly2
  ];
  namespace = "TestNamespace";
  mainClassName = "MyTestApplication";
  mainClassFile = "MyTestApplication.cs";
}

As you may see, the structure of the dotnetenv.buildWrapper is similar to the dotnetenv.buildSolution function, except that it requires several additional parameters for the wrapper, such as the namespace, class name and file location of the class containing the Main method of the actual executable. The function automatically makes the given Main method in the given main class file public and it creates a wrapper class containing the right properties to run the actual executable, such as the location of the actual executable and the paths of the run-time dependencies.

By using this wrapper function, it is possible to run a .NET executable assembly from the Nix store without much trouble.

Conclusion

In this blog post, I have implemented a wrapper executable that deals with resolving run-time dependencies of a .NET application. The wrapper uses a resolve event handler which loads all the required library assemblies through the .NET reflection API. This wrapper can be automatically generated from a convenience function, which makes it possible to run .NET applications from the Nix store, without much trouble.

References

• This page explains how the reflection API can be used to dynamically load assemblies: http://www.codeproject.com/KB/cs/DynLoadClassInvokeMethod.aspx
• The following page explains various ways to load library assemblies beyond the base directory of the executable. It also includes an explanation of the AssemblyResolve event: http://support.microsoft.com/kb/837908