Porting GNU Autotools projects to Visual C++ projects (unconventional style)

Not so long ago, I have ported my IFF experiments' sub projects to SDL 2.0. Besides moving to SDL 2.0, I did some additional development work. One of the interesting things that I did was porting these sub projects to Visual C++ to make them work under Windows natively.

The main reason why I did this, is because I have received numerous feature requests for Visual C++ support in the past. Finally, I found a bit of time to actually do it. :-)

In general, the porting process was straight forward, but nonetheless I encountered a few annoyances. Moreover, since I have a research background in software deployment, I also want to make this deployment process manageable using my favourite deployment tools.

In this blog post, I will describe the porting steps I have done using my own "unconventional" style.

Generating projects

The first thing I did was turning relevant units of each package into Visual C++ projects so that they can be actually built with Visual C++. Typically, a Visual Studio project produces one single binary artifact, such as a library or executable.

Visual Studio 2013 has a very useful feature that composes solutions out of existing source directories automatically. It can be invoking by opening Visual Studio, and selecting 'File' -> 'New' -> 'Project From Existing Code...' from the menu bar.

For example, for the libiff package, I created three projects. One library project that builds libiff and two projects that produce executables: iffpp and iffjoin. To compose a project out of the src/libiff sub directory, I provided the following settings to the import wizard:

• What type of project would you like to create? Visual C++
• Project file location: D:\cygwin64\home\Sander\Development\libiff\src\libiff
• Project name: libiff
• How do you want to build the project? Use Visual Studio
• Project type: Dynamically linked library (DLL) project

For the remaining projects (the command-line utilities), I used Console application project as a project type.

Configuring project dependencies

After generating solutions for all relevant units of each package, I have configured their dependencies so that they can be built correctly.

The first aspect is to get the dependencies right between projects in a package. For example, the libiff package builds two executables: iffpp and iffjoin which both have a common dependency on the libiff shared library. The solutions I just generated from the sub directories, have no knowledge about any of its dependencies yet.

To set the inter-project dependencies, I composed a new empty solution, through 'File' -> 'New' -> 'Project' and then by picking 'Blank Solution' under 'Other project types'. In this blank solution, I have added all the projects of the package that I have generated previously.

After adding the projects to the solution, I can configure their dependency relationships. This is done by right clicking on each project and selecting the 'Build dependencies' -> 'Project dependencies...'. I used this dialog to make libiff a project dependency of iffpp and iffjoin.

After setting up a common solution for all the package's sub projects, I can delete the individual solution files for each project, since they are no longer needed.

Configuring library dependencies

Besides getting the dependencies among projects right, executables must also be linked against dynamic libraries that are either produced by other projects in the solution or by other means (e.g. in a different solution or prebuilt). In order to configure these dependencies, we have to change the project settings:

• To link an executable to a dynamic library, we must right click on the corresponding project, select 'Properties', and pick the option 'Configuration Properties' -> 'Linker' -> 'Input'. In the configuration screen, we must add the name of the export library (a *.LIB file) to the 'Additional Dependencies' field.
• We must also specify where the export libraries can be found. The library search directories can be configure by selecting 'Configuration Properties' -> 'Linker' -> 'General' in the properties screen and adapting the 'Additional Library Directories' field by adding its corresponding path.
• Projects using shared libraries typically also have to find their required header files. The paths to these files can be configured by picking the option 'Configuration Properties' -> 'C/C++' -> 'General'. The required paths must be added to the 'Additional Include Directories' property.

It is a bit tricky to specify some of these paths in a "portable" way. Fortunately, there were a couple very useful macros that I was able to use, such as: $(OutDir) to refer to the output directory of the solution.

To refer to external libraries that do not reside in the solution (such as SDL 2.0), I defined my own custom properties and manually added them to the Visual C++ project files. For example, to allow a project to find SDL 2.0, I added the following lines to SDL_ILBM's project file:

<PropertyGroup>
  <SDL2IncludePath>..\..\..\SDL2-2.0.3\include</SDL2IncludePath>
  <SDL2LibPath>..\..\..\SDL2-2.0.3\lib\x86</SDL2LibPath>
</PropertyGroup>

I can refer to the properties with the following macros: $(SDL2IncludePath), $(SDL2LibPath) from the 'Additional Includes' and 'Additional Library Directories' fields. Moreover, these properties can also be overridden by the build infrastructure, which is very useful as we will see later.

Building export libraries

Another thing I observed is that in Visual C++, you need export libraries (*.LIB files) in order to be able to link a dynamic library to something else. However, if no exports are defined in a library project, then this file is not generated.

The most common way to export functions is probably by annotating headers and class definitions, but I don't find this very elegant for my project, since it requires me to adapt the source code and add non-standard pieces to it.

Another way is creating a Module-Definition File (*.DEF file) and adding it to the project that builds a library. A module definition file can be added to a project by right clicking on it, picking 'Add new item...', and selecting 'Visual C++' -> 'Code' -> 'Module-Defintion File (.def)'.

Creating this module definition file is straight forward. I investigated all headers files of the library to see what functions need to be accessible. Then I created a module definition file that looks as follows:

LIBRARY    libiff
EXPORTS
    IFF_readFd  @1
    IFF_read    @2
    IFF_writeFd @3
    IFF_write   @4
    IFF_free    @5
    IFF_check   @6
    IFF_print   @7
    IFF_compare @8

The above file basically lists the names of all publicly accessible functions with a unique numeric id. These steps were enough for me to get an export library built.

Porting the command-line interfaces

Another porting issue was getting the command-line interfaces to work. This is actually the only "non-standard" piece of code in the IFF sub projects and depends on getopt() or getopt_long(), which is not part of Visual C++'s standard runtime. Furthermore, getopt-style parameters are a bit weird for Windows command line utilities.

I have decided to create a replacement command-line interface for Windows, that follows Windows console application conventions for command line parameters. For example on Unix-like platforms we can request the help text of iffpp as follows:

$ iffpp -h

On windows the same option can be requested as follows:

$ iffpp /?

As can be observed, we use Windows-style command-line parameters.

Automating build processes

The last aspect I had to take care of is getting the entire build process of all the sub projects automated. People who know me, know that I have a preference for Nix-related tools, for various reasons (check the link for the exact reasons).

Like .NET software, Visual C++ projects can be built from the command-line with MSBuild. Fortunately, I have already created a Nix function that invokes MSBuild to compile C# projects some time ago.

I have not used Nix on Cygwin for a while, and Nix's Cygwin support seemed to be broken, so I had to revive it. Fortunately, the changes were relatively minor. Moreover, the .NET build function still seemed to work after reviving Cygwin support.

To support building Visual C++ projects, I basically had to make two changes to the function that I have used for Visual C# projects. First, I had to set the following environment variable:

$ export SYSTEMDRIVE="C:"

Without this environment variable set, the compiler complains that paths are not well formed.

The second thing is to make the parameters to MSBuild configurable through an environment variable named msBuildOpts:

$ MSBuild.exe ... $msBuildOpts

The reason why I have added this feature, is because I want to make the properties that refer to external libraries (such as SDL 2.0 through the $(SDL2IncludePath) and $(SDL2LibPath) macros) configurable so that they can refer to dependencies that reside in the Nix store.

With these changes, I can write a Nix expression for any IFF file format experiment project. I used the following partial expression to build SDL_ILBM with Visual C++:

with import <nixpkgs> {};

let
  SDL2devel = stdenv.mkDerivation {
    name = "SDL2-devel-2.0.3";
    src = fetchurl {
      url = http://www.libsdl.org/release/SDL2-devel-2.0.3-VC.zip;
      sha256 = "0q6fs678i59xycjlw7blp949dl0p2f1y914prpbs1cspz98x3pld";
    };
    buildInputs = [ unzip ];
    installPhase = ''
      mkdir -p $out
      mv * $out
    '';
    dontStrip = true;
  }; 
in
dotnetenv.buildSolution {
  name = "SDL_ILBM";
  src = ./.;
  baseDir = "src";
  slnFile = "SDL_ILBM.sln";
  preBuild = ''
    ...
    export msBuildOpts="$msBuildOpts /p:SDL2IncludePath=\"$(cygpath --windows ${SDL2devel}/include)\""
    export msBuildOpts="$msBuildOpts /p:SDL2LibPath=\"$(cygpath --windows ${SDL2devel}/lib/x86)\""
  '';
}

The above expression builds the SDL_ILBM solution that resides in the src/ folder of the package. It uses the msBuildOpts variable to override the properties that we have defined earlier to pass the paths of the external projects to the build, such as SDL 2.0. It uses the cygpath command to translate UNIX paths to Windows paths so that they can be used with MSBuild.

By running the following command-line instruction:

$ nix-build sdlilbm.nix

SDL_ILBM including all its dependencies are automatically downloaded and built by Nix and stored in the Nix store.

Conclusion

By performing all the steps described in the blog post, I was able to port all my IFF file format experiment sub projects to Visual C++, which I can also automatically build with the Nix package manager to make the deployment process convenient and repeatable.

The following screenshots may show some interesting results to you:

Availability

The updated IFF projects can be obtained from my GitHub page. Moreover, if you want to use Nix to build Visual C++ projects on Cygwin, then you need to use my personal forks of Nix and Nixpkgs, which contain Cygwin specific fixes. I may push these changes upstream if others are interested in it and I consider them stable enough.