As I have mentioned in my previous blog post (in which I reflected over the last year), the Amiga is still not quite dead yet and (for some reason) my blog post about
my good ol' Amiga has attracted much more visitors than I expected. In fact, for a long time this was the most interesting story I have written, while most of my other blog posts are mostly about the research that I'm currently doing.
Nowadays, there is still an active community developing Amiga software. For that reason, I have decided to pay some attention to this subject once more. This time I'm going to make it a bit more related to the research I'm currently doing. If you are curious to see how I achieve that goal, please read on...
Geek Gadgets
One of the interesting efforts done in the past is a project called
Geek Gadgets, in which the GNU toolchain and many other UNIX utilities are ported to AmigaOS. You could see this effort as something similar to
Cygwin, providing a UNIX environment and GNU toolchain on Microsoft Windows.
Interesting packages included in this project are:
- ixemul.library. A BSD 4.3 kernel running under AmigaOS, which internally translates a number system API calls to AmigaOS kernel calls. The API is specifically modelled after NetBSD.
- libnix. An ANSI C API library directly using the AmigaOS libraries, instead of the Unix compatibility layer. This library is better suited for developing native AmigaOS applications not requiring any UNIX functionality.
- GNU Compiler Collection. The free/open-source/ubiquitous compiler suite used on many platforms, such as Linux, Cygwin, FreeBSD, OpenBSD, NetBSD and Mac OS X.
- Korn Shell. Geek Gadgets provides the pdksh shell as a Bourne-compatible shell instead of bash, which is commonly used on other GNU systems. Probably because it's smaller and faster.
These tools make it possible to port Unix-like applications to AmigaOS as well as writing native AmigaOS applications, using development tools commonly found on free software systems, such as Linux.
Quite a number of applications are ported using this toolset such as TeX, Emacs, the X Window System and
SDL, a library which is commonly used on Linux to develop games and other multimedia applications.
Using Geek Gadgets
Below, I have included a screenshot showing the Amiga Workbench and an Amiga Shell. In this Amiga shell, I have opened a Korn Shell session to show the contents of the Geek Gadgets toolset and the version of the GCC compiler.
As you may notice in the screenshot above, the
/gg directory is the Geek Gadgets root directory in which a typical UNIX-like filesystem hierarchy is defined with its associated files and utilities.
The Amiga filesystem is organized quite differently as a UNIX filesystem. Whereas a UNIX filesystem is a tree with only one root directory, an Amiga filesystem has many roots. For example, on the Amiga a path can be identified by
Assign:RootDirectory/Subdirectory/File, in which an assignment can represent a disk drive identifier, disk label or a random assignment which you can attach to any folder in the filesystem.
Like Cygwin, the ixemul kernel also provides a UNIX to native filesystem mapping, by defining assignments as root directories. For example the
/gg directory represents the
GG: assignment in which the Geek Gadgets files are stored.
Deploying Geek Gadgets
Deploying a Geek Gadgets environment was a bit trickier than I expected:
- The latest stable version is much too old, dating from 1998. The Geek Gadgets documentation is quite outdated and does not accurately describe what packages you need from the latest snapshot.
- In order to use any Geek Gadgets tools, I need to manually set up the Geek Gadgets environment each time, by executing the GG:Sys/S/GG-Startup script and by raising the stack size, because otherwise I can get mysterious lock-ups/crashes. Of course, I could also add these instructions to S:User-Startup, but I don't want the Geek Gadgets to spoil up my regular system when I use native AmigaOS applications.
- Geek Gadgets uses a traditional UNIX filesystem organisation, with all its disadvantages. For example, packages are deployed in an impure manner because their contents are scattered across the filesystem. I don't like it very much to let development packages pollute my filesystem.
Most importantly, my research as a PhD student is about software deployment in which I try to deal with such inconveniences. For Amiga readers not knowing anything about it, one of our key aspects is the
Nix package manager, which we use to build software components from declarative build specifications. Furthermore, Nix has several distinct features such as the ability to store components in isolation, atomic upgrades and rollbacks and garbage collector which safely removes components no longer in use.
Nowadays, I use the Nix package manager for virtually everything that I develop and I have to compile, because it stores all builds isolated and it never pollutes my system. Furthermore, I can also easily use the Nix package manager to build several variants of the same package (e.g. for
i686-linux and
x86_64-linux), just by modifying several build parameters in the Nix expression for that particular package.
Building AmigaOS applications with Nix
So an interesting question that I have raised to myself is: "Can I use my research about software deployment to deal with these deployment inconveniences of Geek Gadgets on AmigaOS?".
An answer that you probably would expect me to give is that I have ported the Nix package manager to AmigaOS, but unfortunately that isn't possible because Nix requires much more memory than an Amiga has. Furthermore, the ixemul library does not completely support the POSIX API and does not support symlinks, which makes it impossible to run the Nix package manager properly.
Instead, I have developed a Nix function, which should be invoked on a Linux machine. This function launches
UAE, the Ultimate Amiga Emulator, to perform a build and stores the build result in the Nix store of the host system. To develop this Nix function I had to create a basic disk image and a function that starts UAE with this disk image in which the actual build is performed.
Creating a disk image
Creating a basic disk image is a straightforward process:
Installing the Geek Gadgets environment is a little trickier though:
Writing a Nix function
Now that we have a basic disk image containing the Amiga Workbench and the Geek Gadgets utilities, we need to write a Nix function that automatically performs the build steps. I will list pieces of the code of this Nix function here, with some explanation.
The AmigaOS build function, takes 4 arguments where 1 argument is optional:
{name, src, buildCommand, buildInputs ? []}:
The
name parameter specifies the name of the component, which also appears in the resulting Nix store path. The
src parameter is used to specify the source code that must be compiled. This parameter could refer to a tarball, or a directory containing source code. You could also bind this parameter to a function, such as
fetchurl to remotely obtain a source tarball. The
buildCommand parameter is a string which specifies how the source code can be built. The
buildInputs parameter can be used to install additional packages into the Geek Gadgets environment, such as libraries or other build tools which aren't included in the basic disk image.
The next line imports all the Nix package build functions into the scope of our AmigaOS build function, so that we can conveniently access UAE and other utilities:
with import <nixpkgs> {};
We also need to specify where we can find the disk image containing the Workbench and Geek Gadgets. The UAE emulator also needs a kickstart ROM image. The following code fragment specifies where to find them (you need to replace these paths to match your situation):
let
amigaBase = ./amigabase;
kickRom = ./kickrom/kick.rom;
in
The following lines invoke the
stdenv.mkDerivation function, which we need to specify how we can actually perform a build. The following code fragment inherits the name from our function header. Furthermore, it adds the UAE emulator in our environment so that we can boot our disk image and perform the compilation. We need
procps to kill the emulator, once the build has finished:
stdenv.mkDerivation {
inherit name;
buildInputs = [ uae procps ];
buildCommand = ''
The remainder of the function specifies how the build should be performed. As we're not allowed to mess up the original base image (it's read-only anyway during a Nix build), we need to create a new one in our temp directory of our build. Fortunately, most parts of the AmigaOS filesystem can be used in read-only mode, so we only need to symlink these system directories. The following code fragment creates symlinks to all static AmigaOS directories:
mkdir hd
cd hd
for i in ${amigaBase}/{C,Classes,Expansion,Fonts,L,Libs,Locale} \
${amigaBase}/{Prefs,Rexxc,S,Storage,System,Tools,Utilities}
do
ln -s "$i"
# Symlink icons
if [ -f "$i.info" ]
then
ln -sf "$i.info"
fi
done
The Geek Gadgets environment however, must be copied to our new temporary hard drive, because we may need to install other packages in this environment. Additionally, we need to fix the permissions of the copied GG directory, because in the Nix store it has been made read-only and we require a writeable GG directory:
cp -av ${amigaBase}/GG .
chmod 755 GG
cd GG
for i in `find . -type d` `find . -type f`
do
chmod 755 $i
done
If we have any build inputs (such as library packages), we need to unpack them into the Geek Gadgets environment:
for i in ${toString buildInputs}
do
cp -av $i/* .
done
The next step is copying the source code into a location in which a build can be performed. For this purpose we create a temp
T directory in our AmigaOS filesystem. If the source code is a directory, we need to strip the hash off it and we need to make it writable again:
cd ..
mkdir T
cd T
stripHash ${src}
cp -av ${src} $strippedName
if [ -d "$strippedName" ]
then
chmod -R 755 "$strippedName"
fi
Then we have to execute the instructions defined in the
buildCommand somehow. We generate a shell script called
buildinstructions.sh containing the contents of the
buildCommand string. This script is executed at boot time by the Korn shell.
echo "src=$strippedName" > buildinstructions.sh
cat >> buildinstructions.sh << "EOF"
${buildCommand}
EOF
We can't directly invoke the build instructions, because we also have to determine whether a build has succeeded or failed. The following code fragments creates an additional script called:
build.sh, which invokes the
buildinstructions.sh script. After the build instructions script has finished, it determines whether it has succeeded or failed and writes the status into a file called
done which is stored in the resulting Nix store path:
cat > build.sh << "EOF"
( sh -e buildinstructions.sh
if [ $? = 0 ]
then
echo "success" > /OUT/done
else
echo "failure" > /OUT/done
fi
) 2>&1 | tee /OUT/log.txt
EOF
We now configured a bootable disk image, which automatically performs a build. The next step is to provide a number of settings to UAE so that our temporary disk image can be used and that a build is performed properly:
cd ../..
# Create UAE config file
export HOME=$(pwd)
cat > .uaerc <<EOF
config_description=UAE default configuration
use_gui=no
kickstart_rom_file=${kickRom}
sound_output=none
fastmem_size=8
chipmem_size=4
cpu_speed=max
cpu_type=68ec020
gfx_linemode_windowed=double
filesystem2=rw,:HD0:$(pwd)/hd,1
filesystem=rw,HD0:$(pwd)/hd
filesystem2=rw,:OUT:$out,0
filesystem=rw,OUT:$out
EOF
The code fragment above defines our temporary build directory as HOME directory (In a Nix build environment this variable is set to a non-existent path:
/homeless-shelter in order to remove side effects). Furthermore, we generate a UAE configuration file with the following properties:
- We disable the GUI, because this prevents us running UAE non-interactively.
- We need to specify where the Kickstart ROM image can be found. This property is defined earlier in the kickRom variable.
- We disable sound support, because we don't need it and it may cause lock ups in some cases.
- The Geek Gadgets tools require quite an amount of RAM to run properly, so we've configured the UAE instance to use 2 MiB of Chip RAM and 8 MiB of Fast RAM.
- Normally the UAE emulator tries to accurately match the speed of a 7 MHz Motorla 68000 chip, which makes building software very slow compared to nowadays' standards. Fortunately, UAE can also use the host CPU as efficiently as possible, which we do in our build function to make builds significantly faster.
- We also configured two hard drive instances. The HD0: drive corresponds to the disk image we have created earlier containing the Workbench and Geek Gadgets. The OUT: drive corresponds to the Nix store output path, in which we have to store our build result.
After configuring UAE, we have to run it. Since UAE is a GUI application requiring a X11 display server, we have to specify the
DISPLAY environment variable:
mkdir -p $out
export DISPLAY=:0
# Start UAE
uae &
There is no way to shut UAE down from an Amiga session, so we wait until we see the
done file in our output directory. Once this file exists we kill the UAE process:
while [ ! -f $out/done ]
do
sleep 1
done
# Kill the emulator
pkill uae
sleep 1
# Check the build status
[ "$(cat $out/done)" = "success" ]
After executing all the previous steps, the build has either succeeded or failed. If a build has succeeded, we can remove the
done file, as well as some temp files created by UAE:
cd $out
rm done
rm -f `find . -name _UAEFSDB.___`
'';
}
Using the Nix function
The following code fragment shows how this Nix function can be used to build GNU Hello for AmigaOS:
import ./amigaosbuild.nix {
name = "hello";
src = fetchurl {
url = mirror://gnu/hello/hello-2.1.1.tar.gz;
sha256 = "1md7jsfd8pa45z73bz1kszpp01yw6x5ljkjk2hx7wl800any6465";
};
buildCommand = ''
tar xfvz $src
cd hello-2.1.1
./configure --prefix=/OUT
make || true
make install
'';
}
The expression above looks almost identical to the regular Nix expression building GNU Hello. Important to point out is that we store the output of the GNU Hello world in the
/OUT directory, which corresponds to the
OUT: drive. The GNU Hello build apparently requires
help2man to generate a manual page. This tool is not present in our environment, and to make the build succeed I've added
|| true to the make instruction, so that it always succeeds. By running the following command line instruction:
$ nix-build hello.nix
The GNU Hello component gets build for AmigaOS, which looks like this:
I have also tried compiling a native AmigaOS application, which creates a very simple Intuition GUI window. I wrote the following test program for this purpose:
#include <exec/types.h>
#include <intuition/intuition.h>
#include <intuition/intuitionbase.h>
#include <intuition/screens.h>
#include <clib/exec_protos.h>
#include <clib/dos_protos.h>
#include <clib/intuition_protos.h>
#include <stdio.h>
#define INTUITION_VERSION 37
struct Library *IntuitionBase = NULL;
struct TagItem win_tags[] = {
{WA_Left, 20},
{WA_Top, 20},
{WA_Width, 300},
{WA_Height, 120},
{WA_CloseGadget, TRUE},
{WA_Title, "Hello world!"},
{WA_IDCMP, IDCMP_CLOSEWINDOW},
{TAG_DONE, NULL}
};
void handle_window_events(struct Window *window)
{
WaitPort(window->UserPort);
}
int main(int argc, char *argv[])
{
IntuitionBase = OpenLibrary("intuition.library",
INTUITION_VERSION);
if(!IntuitionBase)
{
fprintf(stderr, "Error opening intuition library!\n");
return 1;
}
else
{
struct Window *window = OpenWindowTagList(NULL, win_tags);
if(window == NULL)
fprintf(stderr, "Window failed to open!\n");
else
{
handle_window_events(window);
CloseWindow(window);
}
CloseLibrary(IntuitionBase);
return 0;
}
}
And this is what the result looks like:
So apparently it works fine!
Conclusion
This blog post covers one of the craziest experiments I have done so far. I have developed a Nix function, which builds software for AmigaOS using the Geek Gadgets toolset by creating a disk image and by using UAE.
After reading this blog post (in case you're still interested :-) ), you may wonder if I actually have a
real use case for this. The answer is: YES, although I'm not going to reveal what it is yet. Probably it will take a while, because I currently don't have that much spare time for fun projects like these.
I have decided not to commit this function to Nixpkgs, because it's quite hacky and it depends on stuff (like the Amiga Workbench) which I cannot distribute freely. It's probably not so hard to recreate this function yourself. You can freely use the code I have written under the
MIT license.
References
- In order to use this build function you need an Amiga Kickstart ROM image and the Amiga Workbench diskettes. If you don't have these, the legal way to obtain these is by ordering one of the Cloanto's Amiga Forever discs. Alternatively, you could try AROS, a free/open-source operating system reimplementing the AmigaOS APIs, although I have no experience with it so far.
- Performing builds in virtual machines inside a Nix expression is nothing new. We have similar approaches in our Hydra build farm, in which we generate OpenSUSE, Ubuntu and Fedora machines to build RPM and Debian packages from Nix expressions. NixOS virtualization provides additional advantages, which I have described here.
Availability
UPDATE: I didn't expect this, but it seems that there are people around who actually want to use this. I have recently put some effort in it to make it a bit usable. The source code can be found on the
nix-amigaosenv GitHub page, which contains updated instructions. Furthermore, I have made some enhancements, so that builds can be performed much faster and cheaper.
