In my previous blog post, I have shown a custom built Linux distribution based on Linux From Scratch (LFS) for my late 90s, early 00s retro PC to run Linux software that I find interesting from that time period. To make the construction process manageable, I have salvaged my custom developed build tool (that is called CBT) that I used to automate my LFS build processes with between 2001 and 2009.
Although I am quite happy with the result, I realized that there are a number of limitations that my custom 2002 Linux deployment has:
- It uses Linux kernel version 2.4.23, which still had a number of fundamental drawbacks in supporting certain kinds of hardware. For example, support for USB storage devices was still quite primitive -- there was no way to easily detect USB flash drives and automatically mount a partition in a KDE or GNOME desktop session. I always had to rely on executing command-line instructions.
- Sound support in Linux still used to rely on the Open Sound System (OSS), which lacked a driver for my Sound Blaster Audigy 2 card. I had to install packages from the Advanced Linux Sound Architecture (ALSA) project and deploy its OSS emulation layer to still have compatibility with existing applications.
- In mid 2002, both the KDE and GNOME desktop projects made their next major releases, significantly updating their capabilities and aesthetics. A drawback is that their underlying developer platforms were not 100% backwards compatible, requiring application developers to make significant modifications.
- Many interesting applications were released in the years after 2002, that I consider worth running on the machine, such as Mozilla Firefox (that at the time of its 1.0 release in 2004, managed to gain quite a bit of market share from Internet Explorer, the then most dominant web browser), and the open-source release of Quake 3 arena in 2005.
To run these new applications, I need much newer versions of the build toolchain packages and various kinds of libraries. These applications are not compatible with the packages in my 2002 distribution.
To better use the Linux potential of my retro PC, I became interested in doing a more modern Linux from Scratch deployment.
Unfortunately, in 2001-2009, I did not make it a habit to archive earlier versions of my custom LFS configurations. As a result, I thought to only have my last LFS configuration from 2009 archived somewhere.
Fortunately, I discovered a configuration from 2007 on an old laptop that I used during my studies. I was able to salvage, extend and upgrade this configuration to suit the needs for my retro PC. As a result, I did not have to do all the work completely from scratch.
In this blog post, I will give an overview of the new features of UL-6.4, a newer version of my custom Linux distribution based on Linux from Scratch.
An overview of new features
UL-6.4 has a number of major upgrades over version 3.3, described in my previous blog post.
Based on Linux from Scratch 6.4
I have used a substantially newer version of Linux from Scratch: version 6.4. It has much newer versions of all kinds of packages including a new GNU build tool chain consisting of glibc 2.8, GCC 4.3.6 and Binutils 2.18.
Moreover, the book uses a more robust bootstrap strategy in which the GNU build toolchain is built in two passes first, before the other required tools of the base system are constructed, making the outcome more reliable and predictable.
In my previous blog post, I have developed a modified multi-pass bootstrap procedure myself, but for this version it was not necessary saving me a considerable amount of time and effort.
Linux kernel 2.6.27.57
In this distribution I am also using a substantially newer version of the Linux kernel: version 2.6.27.57. The Linux 2.6.x series contains a considerable amount of changes over the 2.4.23 kernel, that was used in my 3.3 distribution, including many new features and hardware support.
For example, in Linux 2.6.0 the ALSA subsystem got integrated into the kernel tree as a replacement for the Open Sound System (OSS). As a result, I no longer had to install any ALSA drivers separately to get sound support.
Beyond having all these new capabilities, the 2.6.26.57 kernel release has two other advantages:
- It is a long-term stable support (LTS) release. As a result, it had many iterations of improvements (57 to be precise), and is considered quite stable.
- It is a kernel that is still compatible with version 71.86.14 of NVIDIA's proprietary display driver. I need this specific version of the driver to still use the maximum potential of my Diamond Viper V770 video card (that has an NVIDIA RIVA TNT2 chipset).
In 2008, a free/open-source driver for NVIDIA cards was already in development (Nouveau), but it lacked a substantial amount of features and performance. After version 71.86.14, NVIDIA stopped providing drivers for the RIVA TNT2 and older chipsets.
Display systems
In my previous blog post, I have explained that my distribution supports various kinds of display backends, such as the X Window System, aalib, DirectFB and svgalib.
In this release, I have added libcaca as an additional backend, making it possible to render graphics as colored ASCII Art. The following picture demonstrates that MPlayer is capable of using the caca backend:
Another interesting change is that I have upgraded to X Window System to X.Org version 7.3. As seasoned Linux distribution developers may probably already know, in 2004 there were some internal struggles in the XFree86 project, causing it to be forked. X.Org became the new de-facto X Window System implementation on Linux and other free UNIX-like operating systems.
I learned that X.Org 7.3 is still a good fit for my retro PC -- it includes xorg-server version 1.4 which is still compatible with the 71.86.14 release of the proprietary NVIDIA driver still allowing me to use the full potential of my video card. Newer versions xorg-server no longer seem to be compatible.
I can play the ioquake 3 port quite decently:
Desktops
Similar to the previous release, my distribution supports multiple window managers and desktops. You can easily switch between them by using the KDM, GDM or WDM display managers, which is quite useful for experimentation purposes. This feature is also well supported in this newer version.
As explained in the introduction, both the KDE and GNOME projects released new major versions of their desktops in 2002, improving their features and aesthetics, but also breaking compatibility with existing applications.
In this newer version of my custom Linux distribution, I have upgraded both of these desktops to their next major releases:
- I have upgraded KDE to version 3.5.10, the latest version in the 3.x series. Fortunately, doing this was relatively straight forward, as I had already packaged a slightly older version in the 2007 configuration that I salvaged from my old laptop: 3.5.7.
Compared to previous releases, KDE 3.5 was mainstream for a relatively long time (from 2005 until 2008). The reason is that the KDE project decided to do a major rewrite of the entire software stack for KDE 4.0. It took them a very long time to get feature parity. (If I recall it correctly: I felt that KDE 4.2 was the first release that came close). - I have upgraded GNOME to version 2.32.1, the latest version in the 2.x series. In the 2007 configuration that I salvaged, I had already packaged version 2.18, saving me a bit of time.
Nonetheless, it still took me quite a bit of time to get everything properly packaged, because GNOME consists of a substantial amount of packages with many kinds of changing dependencies.
Furthermore, a few components: cheese, gnome-color-manager and the gnome-disk-manager, could not be packaged because they rely on libgudev that is only included with newer versions of udev (and newer versions of udev also rely on newer versions of the Linux kernel headers etc.). To support these, I need to make substantial changes to the base system.
KDE, GNOME and X-specific sub menus
Since KDE 3.0 and GNOME 2.0, organizing menu items has been standardized by the freedesktop.org group. Both desktop environments use the same file formats and the same search directories for desktop menu items.
An advantage of this standardization effort is that both desktops have access to exactly the same application set. Moreover, it is no longer required to maintain two different menu configurations. As a result, for application developers it has become easier to support any kind of freedesktop.org-compatible desktop environment.
Something that I consider a disadvantage is the usability issues that surface when multiple desktop environments are concurrently installed on the same system.
Some applications are designed for one kind of desktop specifically. As a consequence, the desktop environments' menus display multiple applications that perform exactly the same functions, such as:
- A file manager (GNOME has Nautilus, KDE has Konqueror, for ordinary window managers you may want to use XFM)
- An image viewer (GNOME has EOG, KDE has KView and Kuickshow, for ordinary window managers you may want to use ImageMagick's viewer)
- A terminal emulator (GNOME has the GNOME Terminal, KDE has Konsole, ordinary window managers typically refer to xterm)
Although it is nice to have the freedom to, for example, use the GNOME terminal from a KDE session, you most likely want to give the preference to the applications that are tailored towards the needs of the desktop environment that you have selected.
If a menu is polluted with similar applications from multiple desktops, I consider the user experience to be confusing, especially if you have to show the Linux desktop experience to less technical users.
In UL-6.4, I have categorized some applications as X, KDE and GNOME-specific and created three specialized sub menus:
- When starting a KDE session, all GNOME-specific applications are organized in a specialized "GNOME Menu" sub menu and all X-specific applications in an "X Applications" sub menu. The KDE sub menus only display KDE-specific and general applications.
- When starting a GNOME session, all KDE-specific applications are organized in a specialized "KDE Menu" sub menu and all X-specific applications in an "X Applications" sub menu. The GNOME sub menus only display GNOME-specific and general applications.
Some applications may not be desktop-specific (but may still be using technology preferred by a specific desktop). For example, both Mozilla Firefox and the GIMP use GTK+ (the GUI toolkit used by the GNOME desktop), but I have not considered them to be GNOME specific. General applications are available in the default menus of all desktop environments.
XDG menu integration in non-standardized window managers
As explained earlier in this blog post, KDE and GNOME use standardized file formats for organizing their menus. The other window managers in this distribution also have menu functionality, but they use their own file formats.
It is possible to easily integrate a freedesktop.org-compatible menu organization into these other window managers' configurations by using a tool called: xdg_menu. xdg_menu can automatically convert freedesktop.org menus to the file formats of all kinds of window managers.
All window managers in this distribution (TWM, MWM, fvwm95 and Window Maker) are supported. When XDG menu integration is enabled, you should see an "XDG Menu" sub menu, as shown in the screenshots below:
When using the XDG menu integration feature, the menu structure gives the preference to ordinary X applications and general applications. KDE applications are organized in a "KDE Menu" sub menu and GNOME applications in a "GNOME Menu" sub menu.
New applications
As already explained, due to all the package upgrades and newer versions of the GNOME and KDE desktops, I also got access to more modern applications, such as Audacious (the spiritual successor of the XMMS media player), the GIMP, Inkscape and a much newer version of OpenOffice.org (supporting the OpenDocument format):
Legacy application support
As described in my previous blog posts, one of my motivating reasons to create a legacy Linux distribution is to run interesting old Linux applications, such as games from Loki games.
In previous blog posts, I have explained that lasting binary compatibility is all but a strength of the Linux package eco-system. Although the Linux kernel adopted a policy to "not break the userspace", many other package maintainers do not have such a policy in place. For example, both the GCC compiler and glibc package have had ABI breakages in the past.
In my new distribution version, I also ran into compatibility issues as a consequence of using a much newer GNU build toolchain (consisting of GCC, glibc and Binutils). Unfortunately, it seems that some Loki games, most notably Unreal Tournament (if I use an upgraded OpenGL-renderer) and Sim City 3000 no longer seem to work.
For example, by inspecting the library dependencies of the updated OpenGL renderer in Unreal Tournament, I discovered that it requires a different libstdc++ library:
$ ldd ./OpenGLDrv.so linux-gate.so.1 => (0xffffe000) libdl.so.2 => /lib/libdl.so.2 (0xb7874000) libnsl.so.1 => /lib/libnsl.so.1 (0xb785d000) libpthread.so.0 => /lib/libpthread.so.0 (0xb7844000) libX11.so.6 => /usr/X11R7.3/lib/libX11.so.6 (0xb7727000) libXext.so.6 => /usr/X11R7.3/lib/libXext.so.6 (0xb7718000) libXmu.so.6 => /usr/X11R7.3/lib/libXmu.so.6 (0xb7702000) libXt.so.6 => /usr/X11R7.3/lib/libXt.so.6 (0xb76b1000) libXi.so.6 => /usr/X11R7.3/lib/libXi.so.6 (0xb76a9000) libSM.so.6 => /usr/X11R7.3/lib/libSM.so.6 (0xb76a1000) libICE.so.6 => /usr/X11R7.3/lib/libICE.so.6 (0xb7688000) libXpm.so.4 => /usr/X11R7.3/lib/libXpm.so.4 (0xb7677000) Core.so => ./Core.so (0xb7508000) libstdc++-libc6.2-2.so.3 => not found libm.so.6 => /lib/libm.so.6 (0xb74e2000) libc.so.6 => /lib/libc.so.6 (0xb739e000) /lib/ld-linux.so.2 (0xb78d8000) libxcb.so.1 => /usr/X11R7.3/lib/libxcb.so.1 (0xb7385000) libXau.so.6 => /usr/X11R7.3/lib/libXau.so.6 (0xb7382000) libXdmcp.so.6 => /usr/X11R7.3/lib/libXdmcp.so.6 (0xb737d000) libSDL-1.1.so.0 => ./libSDL-1.1.so.0 (0xb72de000)
As you may probably notice by looking at the above output, the library: libstdc++-libc6.2-2.so.3 cannot be found.
To fix the missing libstdc++ problem, I had to compile GCC 2.95.3 with its C++ runtime. Bootstrapping GCC 2.95.3 on my new system turned out to be much more challenging than I thought: there are some glibc 2.8 headers that cannot be compiled with this version of GCC. Fortunately, I discovered a blog post describing how to bootstrap this particular version of GCC (and other old GCC releases) on modern systems.
I also ran into another problem with my legacy libstdc++ library. After applying the fix from the blog post and successfully compiling the libstdc++ runtime, I still could not run any old C++ applications. I ran into these kinds of runtime errors:
symbol lookup error: /usr/lib/libstdc++-libc6.1-1.so.2: undefined symbol: _pthread_cleanup_push_defer__FP23_pthread_cleanup_bufferPFPv_vPv
At first, I thought these missing symbol errors were caused by a dangling library dependency or a linker error, but none of that seemed to be the case.
After doing a more extensive search and some thinking, I realized that the problem is caused by the name mangling property of the C++ compiler. The symbol: _pthread_cleanup_push_defer__FP23_pthread_cleanup_bufferPFPv_vPv (as you may probably already notice, the symbol follows some weird naming convention) is a function invocation to a function named: pthread_cleanup_buffer.
I discovered that pthread_cleanup_buffer is a C API function provided by glibc. The problem is that linker is unable to find it because the function name got mangled.
To fix the above problem, I have patched the affected C++ files in GCC 2.95.3. I have searched for all C++ files that require pthread functionality. I have modified these files to always declare C prototype functions at the beginning:
For example, I have added the following prototype declaration to the beginning of libio/osform.cc:
#include <pthread.h> extern "C" void _pthread_cleanup_push_defer(struct _pthread_cleanup_buffer *, void (*)(void *), void*); extern "C" void _pthread_cleanup_pop_restore(_pthread_cleanup_buffer *, int);
Adding these kinds of prototype declarations prevented these function names to get mangled.
With these modifications, I was able to play Unreal Tournament with the updated OpenGL renderer:
For Sim City 3000, I ran into a different runtime error resulting in an early segmentation fault:
$ s3cu Segmentation fault
After doing a search on the web, I learned that the threading implementation in glibc is the problem. In the previous version of my distribution, I was using glibc 2.2.5 with LinuxThreads. In my new distribution, I am using glibc 2.8 using the Native POSIX Thread Library (NPTL). There are some incompatibilities between these threading implementations.
Fortunately, I found a project: sc3u-nptl that solved the problem. The trick is to preload a proxy library that implements a workaround for the affected threading API function.
With this proxy library and running the following instructions, I can happily play Sim City 3000 on my new Linux configuration:
$ export LD_PRELOAD=/usr/local/lib/s3cu-nptl.so $ sc3u
Conclusion
In this blog post, I have given an overview of the new features of UL-6.4, a custom Linux distribution that contains somewhat newer versions of packages for my retro PC.
I have published the build recipes in the CBT configs repository, for those who are interested.
