Tuesday, July 4, 2023

Using a site map for generating dynamic menus in web applications

In the last few weeks, I have been playing around with a couple of old and obsolete web applications that I have developed in the past with my own web framework. Much of the functionality that these custom web applications offer are facilitated by my framework, but sometimes these web applications also contain significant chunks custom code.

One of the more interesting features provided by custom code is folding menus (also known as dropdown and dropright menus etc.), that provide a similar experience to the Windows start menu. My guess is that because the start menu experience is so familiar to many users, it remains a frequently used feature by many web applications as of today.

When I still used to actively develop my web framework and many custom web applications (over ten years ago), implementing such a feature heavily relied on JavaScript code. For example, I used the onmouseover attribute on a hyperlink to invoke a JavaScript function that unfolds a panel and the onmouseout attribute to fold a panel again. The onmouseover event handler injects a menu section into the DOM using CSS absolute positioning to put it in the right position on the screen.

I could not use the standard menu rendering functionality of my layout framework, because it deliberately does not rely on the usage of JavaScript. As a consequence, I had to write a custom menu renderer for a web application that requires dynamic menu functionality.

Despite the fact that folding menus are popular and I have implemented them as custom code, I never made it a feature of my layout framework for the following two reasons:

  • I want web applications built around my framework to be as declarative as possible -- this means that I want to concisely express as much as possible what I want to render (a paragraph, an image, a button etc. -- this is something HTML mostly does), rather than specifying in detail how to do it (in JavaScript code). As a result, the usage of JavaScript code in my framework is minimized and non-essential.

    All functionality of the web applications that I developed with my framework must be accessible without JavaScript as much possible.
  • Another property that I appreciate of web technology is the ability to degrade gracefully: the most basic and primary purpose of web applications is to provide information as text.

    Because this property is so important, many non-textual elements, such as an image (img element), provide fallbacks (such as an alt attribute) that simply renders alternative text when graphics capabilities are absent. As a result, it is possible to use more primitive browsers (such as text-oriented browsers) or alternative applications to consume information, such as a text-to-speech system.

    When essential functionality is only exposed as JavaScript code (which more primitive browsers cannot interpret), this property is lost.

Recently, I have discovered that there is a way to implement folding menus that does not rely on the usage of JavaScript.

Moreover, there is also another kind of dynamic menu that has become universally accepted -- the mobile navigation menu (or hamburger menu) making navigation convenient on smaller screens, such as mobile devices.

Because these two types of dynamic menus have become so common, I want to facilitate the implementation of such dynamic menus in my layout framework.

I have found an interesting way to make such use cases possible while retaining the ability to render text and degrade gracefully -- we can use an HTML representation of a site map consisting of a root hyperlink and a nested unordered list as a basis ingredient.

In this blog post, I will explain how implementing these use cases are possible.

The site map feature


As already explained, the basis for implementing these dynamic menus is a textual representation of a site map. Generating site maps is a feature that is already supported by the layout framework:


The above screenshot shows an example page that renders a site map of the entire example web application. In HTML, the site map portion has the following structure:

<a href="/examples/simple/index.php">Home</a>

<ul>
    <li>
        <a href="/examples/simple/index.php/home">Home</a>
    </li>
    <li>
        <a href="/examples/simple/index.php/page1">Page 1</a>
        <ul>
            <li>
                <a href="/examples/simple/index.php/page1/page11">Subpage 1.1</a>
            </li>
            <li>
                <a href="/examples/simple/index.php/page1/page12">Subpage 1.2</a>
            </li>
        </ul>
    </li>
    <li>
        <a href="/examples/simple/index.php/page2">Page 2</a>
        ...
    </li>
    ...
</ul>

The site map, shown in the screenshot and code fragment above, consists of three kinds of links:

  • On top, the root link is displayed that brings the user to the entry page of the web application.
  • The unordered list displays links to all the pages visible in the main menu section that are reachable from the entry page.
  • The nested unordered list displays links to all the pages visible in the sub menu section that are reachable from the selected sub page in the main menu.

With a few simple modifications to my layout framework, I can use a site map as an alternative type of menu section:

  • I have extended the site map generator with the ability to mark selected sub pages and as active, similar to links in menu sections. By adding the active CSS class as an attribute to a hyperlink, a link gets marked as active.
  • I have introduced a SiteMapSection to the layout framework that can be used as a replacement for a MenuSection. A MenuSection displays reachable pages as hyperlinks from a selected page on one level in the page hierarchy, whereas a SiteMap section renders the selected page as a root link and all its visible sub pages and transitive sub pages.

With the following model of a layout:

$application = new Application(
    /* Title */
    "Site map menu website",

    /* CSS stylesheets */
    array("default.css"),

    /* Sections */
    array(
        "header" => new StaticSection("header.php"),
        "menu" => new SiteMapSection(0),
        "contents" => new ContentsSection(true)
    ),

    ...
);

We may render an application with pages that have the following look:


As can be seen in the above screenshot and code fragment, the application layout defines three kinds of sections: a header (a static section displaying a logo), a menu (displaying links to sub pages) and a contents section that displays the content based on the sub page that was selected by the user (in the menu or by opening a URL).

The menu section is displayed as a site map. This site map will be used as the basis for the implementation of the dynamic menus that I have described earlier in this blog post.

Implementing a folding menu


Turning a site map into a folding menu, by using only HTML and CSS, is a relatively straight forward process. To explain the concepts, I can use the following trivial HTML page as a template:


The above page only contains a root link and nested unordered list representing a site map.

In CSS, we can hide the root link and the nested unordered lists by default with the following rules:

/* This rule hides the root link */
body > a
{
    display: none;
}

/* This rule hides nested unordered lists */
ul li ul
{
    display: none;
}

resulting in the following page:


With the following rule, we can make a nested unordered list visible when a user hovers over the surrounding list item:

ul li:hover ul
{
    display: block;
}

Resulting in a web page that behaves as follows:


As can be seen, the unordered list that is placed under the Page 2 link became visible because the user hovers over the surrounding list item.

I can make the menu a bit more fancy if I want to. For example, I can remove the bullet points with the following CSS rule:

ul
{
    list-style-type: none;
    margin: 0;
    padding: 0;
}

I can add borders around the list items to make them appear as buttons:

ul li
{
    border-style: solid;
    border-width: 1px;
    padding: 0.5em;
}

I can horizontally align the buttons by adopting a flexbox layout using the row direction property:

ul
{
    display: flex;
    flex-direction: row;
}

I can position the sub menus right under the buttons of the main menu by using a combination of relative and absolute positioning:

ul li
{
    position: relative;
}

ul li ul
{
    position: absolute;
    top: 2.5em;
    left: 0;
}

Resulting in a menu with the following behaviour:


As can be seen, the trivial example application provides a usable folding menu thanks to the CSS rules that I have described.

In my example application bundled with the layout framework, I have applied all the rules shown above and combined them with the already existing CSS rules, resulting in a web application that behaves as follows:


Displaying a mobile navigation menu


As explained in the introduction, another type of dynamic menu that has been universally accepted is the mobile navigation menu (also known as a hamburger menu). Implementing such a menu, despite its popularity, is challenging IMHO.

Although there seem to be ways to implement such a menu without JavaScript (such as this example using a checkbox) the only proper way to do it IMO is still to use JavaScript. Some browsers have trouble accepting such HTML+CSS-only implementations and it requires the use of an HTML element (an input element) that is not designed for that purpose.

In my example web application, I have implemented a custom JavaScript module, that dynamically transforms a site map (that may have already been displayed as a folding menu) into a mobile navigation menu by performing the following steps:

  • We query the root link of the site map and transform it into a mobile navigation menu button by replacing the text of the root link by an icon image. Clicking on the menu button makes the navigation menu visible or invisible.
  • The first level sub menu becomes visible by adding the CSS class: navmenu_active to the unordered list.
  • The menu button becomes active by adding the CSS class: navmenu_icon_active to the image of the root link.
  • Nested menus can be unfolded or folded. The JavaScript code adds fold icons to each list item of the unordered lists that embed a nested unordered list.
  • Clicking on the fold icon makes the nested unordered list visible or invisible.
  • A nested unordered list becomes visible by adding the CSS class: navsubmenu_active to the unordered list
  • A fold button becomes active by adding the CSS class: navmenu_unfold_active to the fold icon image

It was quite a challenge to implement this JavaScript module, but it does the trick. Moreover, the basis remains a simple HTML-rendered site map that can still be used in text-oriented browsers.

The result of using this JavaScript module is the following navigation menu that has unfoldable sub menus:


Concluding remarks


In this blog post, I have explained a new feature addition to my layout framework: the SiteMapSection that can be used to render menu sections as site maps. Site maps can be used as a basis to implement dynamic menus, such as folding menus and mobile navigation menus.

The benefit of using a site map as a basis ingredient is that a web page still remains useful in its most primitive form: text. As a result, I retain two important requirements of my web framework: declarativity (because a nested unordered list describes concisely what I want) and the ability to degrade gracefully (because it stays useful when it is rendered as text).

Developing folding/navigation menus in the way I described is not something new. There are plenty of examples on the web that show how such features can be developed, such as these W3Schools dropdown menu and mobile navigation menu examples.

Compared to many existing solutions, my approach is somewhat puristic -- I do not abuse HTML elements (such as a check box), I do not rely on using helper elements (such as divs and spans) or helper CSS classes/ids. The only exception is to support dynamic features that are not part of HTML, such as "active links" and the folding/unfolding buttons of the mobile navigation menu.

Although it has become possible to use my framework to implement mobile navigation menus, I still find it sad that I have to rely on JavaScript code to do it properly.

Folding menus, despite their popularity, are nice but the basic one-level menus (that only display a collection of links/buttons of sub pages) are in my opinion fine too and much simpler -- the same implementation is usable on desktops, mobile devices and text-oriented browsers.

With folding menus, I have to test multiple resolutions and devices to check whether they provide the right user experience. Folding menus are useless on mobile devices --- you cannot separately trigger a hover event without generating a click event, making it impossible to unfold a sub menu and peek what is inside.

When it is also desired to provide an optimal mobile device experience, you also need to implement an alternative menu. This requirement makes the implementation of a web application significantly more complex.

Availability


The SiteMapSection has become a new feature of the Java, PHP and JavaScript implementations of my layout framework and can be obtained from my GitHub page.

In addition, I have added a sitemapmenu example web application that displays a site map section in multiple ways:

  • In text mode, it is just displayed as a (textual) site map
  • In graphics mode, when the screen width is 1024 pixels or greater, it displays a horizontal folding menu.
  • In graphics mode, when the screen width is smaller than 1024 pixels and JavaScript is disabled, it displays a vertical folding menu.
  • In graphics mode, when the screen width is smaller than 1024 pixels and JavaScript is enabled, it displays a mobile navigation menu.

Friday, December 30, 2022

Blog reflection over 2022

Today, it is my blog's anniversary. As usual, this is a nice opportunity to reflect over the last year.

Eelco Visser


The most shocking event of this year is the unfortunate passing of my former PhD supervisor: Eelco Visser. I still find it hard to believe that he is gone.

Although I left the university for quite some time now, the things I learned while I was employed at the university (such as having all these nice technical discussions with him) still have a profound impact on me today. Moreover, without his suggestion this blog would probably not exist.

Because the original purpose of my blog was to augment my research with extra details and practical information, I wrote a blog post with some personal anecdotes about him.

COVID-19 pandemic


In my previous blog reflection, I have explained that we were in the third-wave of the COVID pandemic caused by the even more contagious Omicron variant of the COVID-19 virus. Fortunately, it turned out that, despite being more contagious, this variant is less hostile than the previous Delta variant.

Several weeks later, the situation got under control and things were opened up again. The situation remained pretty stable afterwards. This year, it was possible for me to travel again and to go to physical concerts, which feels a bit weird after staying home for two whole years.

The COVID-19 virus is not gone, but the situation is under control in Western Europe and the United States. There have not been any lockdowns or serious capacity problems in the hospitals.

When the COVID-19 pandemic started, my employer: Mendix adopted a work-from-home-first culture. By default, people work from home and if they need to go to the office (for example, to collaborate) they need to make a desk reservation.

As of today, I am still working from home most of my time. I typically visit the office only once a week, and I use that time to collaborate with people. In the remaining days, I focus myself on development work as much as possible.

I have to admit that I like the quietness at home -- not everything can be done at home, but for programming tasks I need to think, and for thinking I need silence. Before the COVID-19 pandemic started, the office was typically very noisy making it sometimes difficult for me to focus.

Learning modern JavaScript features


I used to intensively work with JavaScript at my previous employer: Conference Compass, but since I joined Mendix I am mostly using different kinds of technologies. During my CC days, I was still mostly writing old fashioned (ES5) JavaScript code, and I still wanted to familiarise myself with modern ES6 features.

One of the challenging aspects of using JavaScript is asynchronous programming -- making sure that the main thread of your JavaScript application never blocks too long (so that it can handle multiple connections or input events) and keeping your code structured.

With old fashioned ES5 JavaScript code, I had to rely on software abstractions to keep my code structured, but with the addition of Promises/A+ and the async/await concepts to the core of the JavaScript language, this can be done in a much cleaner way without using any custom software abstractions.

In 2014, I wrote a blog post about the problematic synchronous programming concepts in JavaScript and their equivalent asynchronous function abstractions. This year, I wrote a follow-up blog post about the ES6 concepts that I should use (rather than software abstractions).

To motivate myself learning about ES6 concepts, I needed a practical use case -- I have ported the layout component of my web framework (for which a Java and PHP version already exist) to JavaScript using modern ES6 features, such as async/await, classes and modules.

An interesting property of the JavaScript version is that it can be used both on the server-side (as a Node.js application) and client-side (directly in the browser by dynamically updating the DOM). The Java and PHP versions only work server-side.

Fun projects


In earlier blog reflections I have also decided to spend more time on useless fun projects.

In the summer of 2021, when I decided not to do any traveling, I had lots of time left to tinker with all kinds of weird things. One of my hobby projects was to play around with my custom maps for Duke3D and Shadow Warrior that I created while I was still a teenager.

While playing with these maps, I noticed a number of interesting commonalities and differences between Duke3D and Shadow Warrior.

Although both games use the same game engine: the BUILD-engine, their game mechanics are completely different. As an exercise, I have ported one of my Duke3D maps to Shadow Warrior and wrote a blog post about the process, including a description of some of their different game mechanics.

Although I did the majority of the work already back in 2021, I have found some remaining free time in 2022 to finally finish the project.

Web framework improvements


This year, I have also intensively worked on improving several aspects of my own web framework. My custom web framework is an old project that I started in 2004 and many parts of it have been rewritten several times.

I am not actively working on it anymore, but once in a while I still do some development work, because it is still in use by a couple of web sites, including the web site of my musical society.

One of my goals is to improve the user experience of the musical society web site on mobile devices, such as phones and tablets. This particular area was already problematic for years. Despite making the promise to all kinds of people to fix this, it took me several years to actually take that step. :-).

To improve the user experience for mobile devices, I wanted to convert the layout to a flexbox layout, for which I needed to extend my layout framework because it does not generate nested divs.

I have managed to improve my layout framework to support flexbox layouts. In addition, I have also made many additional improvements. I wrote a blog post with a summary of all my feature changes.

Nix-related work


In 2022, I also did Nix-related work, but I have not written any Nix-related blog posts this year. Moreover, 2022 is also the first time since the end of the pandemic that a physical NixCon was held -- unfortunately, I have decided not to attend it.

The fact that I did not write any Nix-related blog posts is quite exceptional. Since 2010, the majority of my blog posts are Nix-related and about software deployment challenges in general. So far, it has never happened that there has been an entire year without any Nix-related blog posts. I think I need to explain a thing or two about what has happened.

This year, it was very difficult for me to find the energy to undertake any major Nix developments. Several things have contributed to that, but the biggest take-away is that I have to find the right balance.

The reason why I got so extremely out of balance is that I do most of my Nix-related work in my spare time. Moreover, my primary motivation to do Nix-related work is because of idealistic reasons -- I still genuinely believe that we can automate the deployment of complex systems in a much better way than the conventional tools that people currently use.

Some of the work for Nix and NixOS is relatively straight forward -- sometimes, we need to package new software, sometimes a package or NixOS service needs to be updated, or sometimes broken features need to be fixed or improved. This process is often challenging, but still relatively straight forward.

There are also quite a few major challenges in the Nix project, for which no trivial solutions exist. These are problem areas that cannot be solved with quick fixes and require fundamental redesigns. Solving these fundamental problems is quite challenging and typically require me to dedicate a significant amount of my free time.

Unfortunately, due to the fact that most of my work is done in my spare time, and I cannot multi-task, I can only work on one major problem area at the time.

For example, I am quite happy with my last major development project: the Nix process management framework. It has all features implemented that I want/need to consistently eat my own dogfood. It is IMHO a pretty decent solution for use cases where most conventional developers would normally use Docker/docker-compose for.

Unfortunately, to reach all my objectives I had to pay a huge price -- I have published the first implementation of the process management framework already in 2019, and all my major objectives were reached in the middle of 2021. As a consequence, I have spend nearly two years of my spare time only working on the implementation of this framework, without having the option to switch to something else. For the first six months, I remained motivated, but slowly I ran into motivational problems.

In this two-year time period, there were lots of problems appearing in other projects I used to be involved in. I could not get these projects fixed, because these projects also ran into fundamental problems requiring major redesigns/revisions. This resulted in a number of problems with members in the Nix community.

As a result, I got the feeling the I lost control. Moreover, doing anything Nix-related work also gave (and in some extent still gives) me a lot of negative energy.

Next year, I intend to return and I will look into addressing my issues. I am thinking about the following steps:

  • Leaving the solution of some major problem areas to others. One of such areas is NPM package deployments with Nix. node2nix's was probably a great tool in combination with older versions of NPM, but its design has reached the boundaries of what is possible already years ago.

    As a result, node2nix does not support the new features of NPM and does not solve the package scalability issues in Nixpkgs. It is also not possible to properly support these use cases by implementing "quick fixes". To cope with these major challenges and keep the solution maintainable, a new design is needed.

    I have already explained my ideas on the Discourse mailing list and outlined what such a new design could look like. Fortunately, there are already some good initiatives started to address these challenges.
  • Building prototypes and integrate the ideas into Nixpkgs rather than starting an independent project/tool that attracts a sub community.

    I have implemented the Nix process management framework as a prototype with the idea to show how certain concepts work, rather than advertising the project as a new solution.

    My goal is to write an RFC to make sure that these ideas get integrated into the upstream Nixpkgs, so that it can be maintained by the community and everybody can benefit from it.

    The only thing I still need to do is write that RFC. This should probably be one of my top priorities next year.
  • Move certain things out of Nixpkgs. The Nixpkgs project is a huge project with several thousands of packages and services, making it quite a challenge to maintain and implement fundamental changes.

    One of the side effects of its scale is that the Nixpkgs issue tracker is a good as useless. There are thousands of open issues and it is impossible to properly track the status of individual aspects in the Nixpkgs repository.

    Thanks to Nix flakes, which unfortunately is still an experimental feature, we should be able to move certain non-essential things out of Nixpkgs and conveniently deploy them from external repositories. I have some things that I could move out of the Nixpkgs repository when flakes have become a mainstream feature.
  • Better communication about the context in which something is developed. When I was younger, I always used to advertise a new project as the next great thing that everybody should use -- these days, I am more conservative about the state of my projects and I typically try to warn people upfront that something is just a prototype and not yet ready for production use.

Blog posts


In my previous reflection blog posts, I always used to reflect over my overall top 10 of most popular blog posts. There are no serious changes compared to last year, so I will not elaborate about them. The fact that I have not been so active on my blog this year has probably contributed that.

Concluding remarks


Next year I will look into addressing my issues with Nix development. I hope to return to my software deployment/Nix-related work next year!

The final thing I would like to say is:


HAPPY NEW YEAR!!!

A summary of my layout framework improvements

It has been quiet for a while on my blog. In the last couple of months, I have been improving my personal web application framework, after several years of inactivity.

The reason why I became motivated to work on it again, is because I wanted to improve the website of the musical society that I am a member of. This website is still one of the few consumers of my personal web framework.

One of the areas for improvement is the user experience on mobile devices, such as phones and tablets.

To make these improvements possible, I wanted to get rid of complex legacy functionality, such as the "One True Layout" method, that heavily relies on all kinds of interesting hacks that are no longer required in modern browsers. Instead, I wanted to use a flexbox layout that is much more suitable for implementing the layout aspects that I need.

As I have already explained in previous blog posts, my web application framework is not monolithic -- it consists of multiple components each addressing a specific concern. These components can be used and deployed independently.

The most well-explored component is the layout framework that addresses the layout concern. It generates pages from a high-level application model that defines common layout aspects of an application and the pages of which an application consists including their unique content parts.

I have created multiple implementations of this framework in three different programming languages: Java, PHP, and JavaScript.

In this blog post, I will give a summary of all the recent improvements that I made to the layout framework.

Background


As I have already explained in previous blog posts, the layout framework is very straight forward to use. As a developer, you need to specify a high-level application model and invoke a view function to render a sub page belonging to the application. The layout framework uses the path components in a URL to determine which sub page has been selected.

The following code fragment shows an application model for a trivial test web application:

use SBLayout\Model\Application;
use SBLayout\Model\Page\StaticContentPage;
use SBLayout\Model\Page\Content\Contents;
use SBLayout\Model\Section\ContentsSection;
use SBLayout\Model\Section\MenuSection;
use SBLayout\Model\Section\StaticSection;

$application = new Application(
    /* Title */
    "Simple test website",

    /* CSS stylesheets */
    array("default.css"),

    /* Sections */
    array(
        "header" => new StaticSection("header.php"),
        "menu" => new MenuSection(0),
        "contents" => new ContentsSection(true),
    ),

    /* Pages */
    new StaticContentPage("Home", new Contents("home.php"), array(
        "page1" => new StaticContentPage("Page 1", new Contents("page1.php")),
        "page2" => new StaticContentPage("Page 2", new Contents("page2.php")),
        "page3" => new StaticContentPage("Page 3", new Contents("page3.php"))
    ))
);

The above application model captures the following application layout properties:

  • The title of the web application is: "Simple test website" and displayed as part of the title of any sub page.
  • Every page references the same external CSS stylesheet file: default.css that is responsible for styling all pages.
  • Every page in the web application consists of the same kinds of sections:
    • The header element refers to a static header section whose purpose is to display a logo. This section is the same for every sub page.
    • The menu element refers to a MenuSection whose purpose is to display menu links to sub pages that can be reached from the entry page.
    • The contents element refers to a ContentsSection whose purpose is to display contents (text, images, tables, itemized lists etc.). The content is different for each selected page.
  • The application consists of a number of pages:
    • The entry page is a page called: 'Home' and can be reached by opening the root URL of the web application: http://localhost
    • The entry page refers to three sub pages: page1, page2 and page3 that can be reached from the entry page.

      The array keys refer to the path component in the URL that can be used as a selector to open the sub page. For example, http://localhost/page1 will open the page1 sub page and http://localhost/page2 will open the page2 sub page.

The currently selected page can be rendered with the following function invocation:

\SBLayout\View\HTML\displayRequestedPage($application);

By default, the above function generates a simple HTML page in which each section gets translated to an HTML div element:


The above screenshot shows what a page in the application could look like. The grey panel on top is the header that displays the logo, the blue bar is menu section (that displays links to sub pages that are reachable from the entry page), and the black area is the content section that displays the selected content.

One link in the menu section is marked as active to show the user which page in the page hierarchy (page1) has been selected.

Compound sections


Although the framework's functionality works quite well for most of my old use cases, I learned that in order to support flexbox layouts, I need to nest divs, which is something the default HTML code generator: displayRequestedPage() cannot do (as a sidenote: it is possible to create nestings by developing a custom generator).

For example, I may want to introduce another level of pages and add a submenu section to the layout, that is displayed on the left side of the screen.

To make it possible to position the menu bar on the left, I need to horizontally position the submenu and contents sections, while the remaining sections: header and menu must be vertically positioned. To make this possible with flexbox layouts, I need to nest the submenu and contents in a container div.

Since flexbox layouts have become so common nowadays, I have introduced a CompoundSection object, that acts as a generic container element.

With a CompoundSection, I can nest divs:

/* Sections */
array(
    "header" => new StaticSection("header.php"),
    "menu" => new MenuSection(0),
    "container" => new CompoundSection(array(
        "submenu" => new MenuSection(1),
        "contents" => new ContentsSection(true)
    ))
),

In the above code fragment, the container section will be rendered as a container div element containing two sub div elements: submenu and contents. I can use the nested divs structure to vertically and horizontally position the sections in the way that I described earlier.


The above screenshot shows the result of introducing a secondary page hierarchy and a submenu section (that has a red background).

By introducing a container element (through a CompoundSection) it has become possible to horizontally position the submenu next to the contents section.

Easier error handling


Another recurring issue is that most of my applications have to validate user input. When user input is incorrect, a page needs to be shown that displays an error message.

Previously, error handling and error page redirection was entirely the responsibility of the programmer -- it had to be implemented in every controller, which is quite a repetitive process.

In one of my test applications of the layout framework, I have created a page with a form that asks for the user's first and last name:


I wanted to change the example application to return an error message when any of these mandatory attributes were not provided.

To ease that burden, I have made framework's error handling mechanism more generic. Previously, the layout manager only took care of two kinds of errors: when an invalid sub page is requested, a PageNotFoundException is thrown redirecting the user to the 404 error page. When the accessibility criteria have not been met (e.g. a user is not authenticated) a PageForbiddenException is thrown directing the user to the 403 error page.

In the revised version of the layout framework, the PageNotFoundException and PageForbiddenException classes have become sub classes of the generic PageException class. This generic error class makes it possible for the error handler to redirect users to error pages for any HTTP status code.

Error pages should be added as sub pages to the entry page. The numeric keys should match the corresponding HTTP status codes:

/* Pages */
new StaticContentPage("Home", new Contents("home.php"), array(
    "400" => new HiddenStaticContentPage("Bad request", new Contents("error/400.php")),
    "403" => new HiddenStaticContentPage("Forbidden", new Contents("error/403.php")),
    "404" => new HiddenStaticContentPage("Page not found", new Contents("error/404.php"))
    ...
))
I have also introduced a BadRequestException class (that is also a sub class of PageException) that can be used for handling input validation errors.

PageExceptions can be thrown from controllers with a custom error message as a parameter. I can use the following controller implementation to check whether the first and last names were provided:

use SBLayout\Model\BadRequestException;

if($_SERVER["REQUEST_METHOD"] == "POST") // This is a POST request
{
    if(array_key_exists("firstname", $_POST) && $_POST["firstname"] != ""
        && array_key_exists("lastname", $_POST) && $_POST["lastname"] != "")
        $GLOBALS["fullname"] = $_POST["firstname"]." ".$_POST["lastname"];
    else
        throw new BadRequestException("This page requires a firstname and lastname parameter!");
}

The side effect is that if the user forgets to specify any of these mandatory attributes, he gets automatically redirected to the bad request error page:


This improved error handling mechanism significantly reduces the amount of boilerplate code that I need to write in applications that use my layout framework.

Using the iterator protocol for sub pages


As can be seen in the application model examples, some pages in the example applications have sub pages, such as the entry page.

In the layout framework, there are three kinds of pages that may provide sub pages:

  • A StaticContentPage object is a page that may refer to a fixed/static number of sub pages (as an array object).
  • A PageAlias object, that redirects the user to another sub page in the application, also offers the ability to refer users to a fixed/static number of sub pages (as an array object).
  • There is also a DynamicContentPage object in which a sub page can interpret the path component as a dynamic value. That dynamic value can, for example, be used as a parameter for a query that retrieves a record from a database.

In the old implementation of my framework, the code that renders the menu sections always has to treat these objects in a special way to render links to their available sub pages. As a result, I had to use the instanceof operator a lot, which is in a bad code smell.

I have changed the framework to use a different mechanism for stepping over sub pages: iterators or iterables (depending on the implementation language).

The generic Page class (that is the parent class of all page objects) provides a method called: subPageIterator() that returns an iterator/iterable that yields no elements. The StaticContentPage and PageAlias classes override this method to return an interator/iterable that steps over the elements in the array of sub pages.

Using iterators/iterables has a number of nice consequences -- I have eliminated two special cases and a bad code smell (the intensive use of instanceof), significantly improving the quality and readability of my code.

Another nice property is that it is also possible to override this method with a custom iterator, that for example, fetches sub page configurations from a database.

The pagemanager framework (another component in my web framework) offers a content management system giving end-users the ability to change the page structure and page contents. The configuration of the pages is stored in a database.

Although the pagemanager framework uses the layout framework for the construction of pages, it used to rely on custom code to render the menu sections.

By using the iterator protocol, it has become possible to re-use the menu section functionality from the layout framework eliminating the need for custom code. Moreover, it has also become much easier to integrate the pagemanager framework into an application because no additional configuration work is required.

I have also created a gallery application that makes it possible to expose the albums as items in the menu sections. Rendering the menu sections also used to rely on custom code, but thanks to using the iterator protocol that custom code was completely eliminated.

Flexible presentation of menu items


As I have already explained, an application layout can be divided into three kinds of sections. A StaticSection remains the same for any requested sub page, and a ContentSection is filled with content that is unique for the selected page.

In most of my use-cases, it is only required to have a single dynamic content section.

However, the framework is flexible enough to support multiple content sections as well. For example, the following screenshot shows the advanced example application (included with the web framework) in which both the header and the content sections change for each sub page:


The presentation of the third kind of section: MenuSection still used to remain pretty static -- they are rendered as div elements containing hyperlinks. The page that is currently selected is marked as active by using the active class property.

For most of my use-cases, just rendering hyperlinks suffices -- with CSS you can still present them in all kinds of interesting ways, e.g. by changing their colors, adding borders, and changing some its aspects when the user hovers with the mouse cursor over it.

In some rare cases, it may also be desired to present links to sub pages in a completely different way. For example, you may want to display an icon or add extra styling properties to an individual button.

To allow custom presentations of hyperlinks, I have added a new parameter: menuItem to the constructors of page objects. The menuItem parameter refers to a code snippet that decides how to render the link in a menu section:

new StaticContentPage("Icon", new Contents("icon.php"), "icon.php")

In the above example, the last parameter to the constructor, refers to an external file: menuitem/icon.php:

<span>
	<?php
	if($active)
	{
		?>
		<a class="active" href="<?= $url ?>">
			<img src="<?= $GLOBALS["baseURL"] ?>/image/menu/go-home.png" alt="Home icon">
			<strong><?= $subPage->title ?></strong>
		</a>
		<?php
	}
	else
	{
		?>
		<a href="<?= $url ?>">
			<img src="<?= $GLOBALS["baseURL"] ?>/image/menu/go-home.png" alt="Home icon">
			<?= $subPage->title ?>
		</a>
		<?php
	}
	?>
</span>

The above code fragment specifies how a link in the menu section should be displayed when the page is active or not active. We use the custom rendering code to display a home icon before showing the hyperlink.

In the advanced test application, I have added an example page in which every sub menu item is rendered in a custom way:


In the above screenshot, we should see two custom presented menu items in the submenu section on the left. The first has the home icon added and the second uses a custom style that deviates from the normal page style.

If no menuItem parameter was provided, the framework just renders a menu item as a normal hyperlink.

Other functionality


In addition to the new functionality explained earlier, I also made a number of nice small feature additions:


  • A function that displays bread crumbs (the route from the entry page to the currently opened page). The route is derived automatically from the requested URL and application model.
  • A function that displays a site map that shows the hierarchy of pages.
  • A function that makes it possible to embed a menu section in arbitrary sections of a page.

Conclusion


I am quite happy with the recent feature changes that I made to the layout framework. Although I have not done any web front-end development for quite some time, I had quite a bit of fun doing it.

In addition to the fact that useful new features were added, I have also simplified the codebase and improved its quality.

Availability


The Java, PHP and JavaScript implementations of my layout framework can be obtained from my GitHub page. Use them at your own risk!

Saturday, August 20, 2022

Porting a Duke3D map to Shadow Warrior


Almost six years ago, I wrote a blog post about Duke Nukem 3D, the underlying BUILD engine and my own total conversion that consists of 22 maps and a variety of interesting customizations.

Between 1997 and 2000, while I was still in middle school, I have spent a considerable amount of time developing my own maps and customizations, such as modified monsters. In the process, I learned a great deal about the technical details of the BUILD engine.

In addition to Duke Nukem 3D, the BUILD engine is also used as a basis for many additional games, such as Tekwar, Witchaven, Blood, and Shadow Warrior.

In my earlier blog post, I also briefly mentioned that in addition to the 22 maps that I created for Duke Nukem 3D, I have also developed one map for Shadow Warrior.

Last year, in my summer holiday, that was still mostly about improvising my spare time because of the COVID-19 pandemic, I did many interesting retro-computing things, such as fixing my old computers. I also played a bit with some of my old BUILD engine game experiments, after many years of inactivity.

I discovered an interesting Shadow Warrior map that attempts to convert the E1L2 map from Duke Nukem 3D. Since both games use the BUILD engine with mostly the same features (Shadow Warrior uses a slightly more advanced version of the BUILD engine), this map inspired me to also port one of my own Duke Nukem 3D maps, as an interesting deep dive to compare both game's internal concepts.

Although most of the BUILD engine and editor concepts are the same in both games, their game mechanics are totally different. As a consequence, the porting process turned out to be very challenging.

Another reason that it took me a while to complete the project is because I had to put it on hold in several occasions due to all kinds obligations. Fortunately, I have managed to finally finish it.

In this blog post, I will describe some of the things that both games have in common and the differences that I had to overcome in the porting process.

BUILD engine concepts


As explained in my previous blog post, the BUILD-engine is considered a 2.5D engine, not a true 3D engine due to the fact that it had to cope with all kinds of technical limitations of home computers commonly used at that time.

In fact, most of the BUILD-engine concepts are two-dimensional -- maps are made out two-dimensional surfaces called sectors:


The above picture shows a 2-dimensional top-level view of my ported Shadow Warrior map. Sectors are two dimensional areas surrounded by walls -- the white lines denote solid walls and red lines the walls between adjacent sectors. Red walls are invisible in 3D mode.

The purple and cyan colored objects are sprites (objects that typically provide some form of interactivity with the player, such as monsters, weapons, items or switches). The "sticks" that are attached to the sprites indicate in which direction the sprite is facing. When a sprite is purple, it will block the player. Cyan colored sprites allow a player to move through it.

You can switch between 2D and 3D mode in the editor by pressing the Enter-key on the numeric key pad.

In 3D mode, each sector's ceiling and floor can be given its own height, and we can configure textures for the walls, floors and ceilings (by pointing to any of these objects and pressing the 'V' key) giving the player the illusion to walk around in a 3D world:


In the above screenshot, we can see the corresponding 3D view of the 2D grid shown earlier. It consists of an outdoor area, grass, a lane, and the interior of the building. Each of these areas are separate 2D sectors with their own custom floor and ceiling heights, and their own textures.

The BUILD engine has all kinds of limitations. Although a world may appear to be (somewhat) 3-dimensional, it is not possible to stack multiple sectors on top of each other and simultaneously see them in 3D mode, although there are some tricks to cope with that limitation.

(As a sidenote: Shadow Warrior has a hacky feature that makes it possible for a player to observe multiple rooms stacked on top of each other, by using specialized wall/ceiling textures, special purpose sprites and a certain positioning of the sectors themselves. Sectors in the map are still separated, but thanks to the hack they can be visualized in such a way that they appear to be stacked on top of each other).

Moreover, the BUILD engine can also not change the perspective when a player looks up or down, although there is the possibility to give a player that illusion by stretching the walls. (As a sidenote: modern source ports of the BUILD engine have been adjusted to use Polymost, an OpenGL rendering extension, which actually makes it possible to provide a true 3D look).

Monsters, weapons, items, and most breakable/movable objects are sprites. Sprites are not really "true 3D" objects. Normally, sprites will always face the player from the same side, regardless of the position or the perspective of the player:

As can be seen, the guardian sprite always faces the player from the front, regardless of the angle of the camera.

Sprites can also be flattened and rotated, if desired. Then they will appear as a flat surface to the player:


For example, the wall posters in the screenshot above are flattened and rotated sprites.

Shadow warrior uses a slightly upgraded BUILD engine that can provide a true 3D experience for certain objects (such as weapons, items, buttons and switches) by displaying them as voxels (3D pixels):


The BUILD engine that comes with Duke Nukem 3D lacks the ability to display voxels.

Porting my Duke Nukem 3D map to Shadow Warrior


The map format that Duke Nukem 3D and Shadow Warrior use are exactly the same. To be precise: they both use version 7 of the map format.

At first, it seemed to look relatively straight forward to port a map from one game to another.

The first step in my porting process was to simply make a copy of the Duke Nukem 3D map and open it in the Shadow Warrior BUILD editor. What I immediately noticed is that all the textures and sprites look weird. The textures still have the same indexes and refer to textures in the Shadow Warrior catalog that are completely different:


Quite a bit of my time was spent on fixing all textures and sprites by looking for suitable replacements. I ended up replacing textures for the rocks, sky, buildings, water, etc. I also had to replace the monsters, weapons, items and other dynamic objects, and overcome some limitations for the player in the map, such as the absence of a jet pack. The process was a labourious, but straight forward.

For example, this is how I have fixed the beach area:


I have changed the interior of the office building as follows:


And the back garden as follows:


The nice thing about the garden area is that Shadow Warrior has a more diverse set of vegetation sprites. Duke Nukem 3D only has palm trees.

Game engine differences


The biggest challenge for me was porting the interactive parts of the game. As explained earlier, game mechanics are not implemented by the engine or the editor. BUILD-engine games are separated into an engine and game part in which only the former component is generalized.

This diagram (that I borrowed from a Duke Nukem 3D code review article written by Fabien Sanglard) describes the high-level architecture of Duke Nukem 3D:


In the above diagram, the BUILD engine (on the right) is general purpose component developed by Ken Silverman (the author of the BUILD engine and editor) and shipped as a header and object code file to 3D Realms. 3D Realms combines the engine with the game artifacts on the left to construct a game executable (DUKE3D.EXE).

To configure game effects in the BUILD editor, you need to annotate objects (walls, sprites and sectors) with tags and add special purpose sprites to the map. To the editor these objects are just meta-data, but the game engine treats them as parameters to create special effects.

Every object in a map can be annotated with meta data properties called Lotags and Hitags storing a 16-bit numeric value (by using the Alt+T and Alt+H key combinations in 2D mode).

In Shadow Warrior, the tag system was extended even further -- in addition to Lotags and Hitags, objects can potentially have 15 numerical tags (TAG1 corresponds to the Hitag, and TAG2 to the Lotag) and 11 boolean tags (BOOL1-BOOL11). In 2D mode, these can be configured with the ' and ; keys in combination with a numeric key (0-9).

We can also use special purpose sprites that are visible in the editor, but hidden in the game:


In the above screenshot of my Shadow Warrior map, there are multiple special purpose sprites visible: the ST1 sprites (that can be used to control all kinds of effects, such as moving a door). ST1 sprites are visible in the editor, but not in the game.

Although both games use the same principles for configuring game effects, their game mechanics are completely different.

In the next sections, I will show all the relevant game effects in my Duke Nukem 3D map and explain how I translated them to Shadow Warrior.

Differences in conventions


As explained earlier, both games frequently use Lotags and Hitags to create effects.

In Duke Nukem 3D, a Lotag value typically determines the kind of effect, while an Hitag value is used as a match tag to group certain events together. For example, multiple doors can be triggered by the same switch by using the same match tag.

Shadow Warrior uses the opposite convention -- a Hitag value typically determines the effect, while a Lotag value is often used as a match tag.


Furthermore, in Duke Nukem 3D there are many kinds of special purpose sprites, as shown in the screenshot above. The S-symbol sprite is called a Sector Effector that determines the kind of effect that a sector has, the M-symbol is a MUSIC&SFX sprite used to configure a sound for a certain event, and a GPSPEED sprite determines the speed of an effect.

Shadow Warrior has fewer special purpose sprites. In almost all cases, we end up using the ST1 sprite (with index 2307) for the configuration of an effect.

ST1 sprites typically combine multiple interactivity properties. For example, to make a sector a door, that opens slowly, produces a sound effect and that closes automatically, we need to use three Sector Effector sprites and one GPSPEED sprite in Duke Nukem 3D. In Shadow Warrior, the same is accomplished by only using two ST1 sprites.

The fact that the upgraded BUILD engine in Shadow Warrior makes it possible to change more than two numerical tags (and boolean values), makes it possible to combine several kinds of functionality into one sprite.

Co-op respawn points



To make it possible to play a multiplayer cooperative game, you need to add co-op respawn points to your map. In Duke Nukem 3D, this can be done by adding seven sprites with texture 1405 and setting the Lotag value of the sprites to 1. Furthermore, the player's respawn point is also automatically a co-op respawn point.

In Shadow Warrior, co-op respawn points can be configured by adding ST1 sprites with Hitag 48. You need eight of them, because the player's starting point is not a co-op start point. Each respawn point requires a unique Lotag value (a value between 0 and 7).

Duke match/Wang Bang respawn points


For the other multiplayer game mode: Duke match/Wang Bang, we also need re-spawn points. In both games the process is similar to their co-op counterparts -- in Duke Nukem 3D, you need to add seven sprites with texture 1405, and set the Lotag value to 0. Moreover, the player's respawn point is also a Duke Match respawn point.

In Shadow Warrior, we need to use ST1 sprites with a Hitag value of 42. You need eight of them and give each of them a unique Lotag value between 0-7 -- the player's respawn point is not a Wang Bang respawn point.

Underwater areas


As explained earlier, the BUILD engine makes it possible to have overlapping sectors, but they cannot be observed simultaneously in 3D mode -- as such, it is not possible to natively provide a room over room experience, although there are some tricks to cope with that limitation.

In both games it is possible to dive into the water and swim in underwater areas, giving the player some form of a room over room experience. The trick is that the BUILD engine does not render both sectors. When you dive into the water or surface again, you get teleported from one sector to another sector in the map.


Although both games use a similar kind of teleportation concept for underwater areas, they are configured in a slightly different way.

In both games, you need the ability to sink into the water in the upper area. In Duke Nukem 3D, the player automatically sinks by giving the sector a Lotag value of 1. In Shadow Warrior, you need to add a ST1 sprite with a Hitag value of 0, and a Lotag value that determines how much the player will sink. 40 is typically a good value for water areas.

The underwater sector in Duke Nukem 3D needs a Lotag value of 2. In the game, the player will automatically swim when it enters the sector and the colors will be turned blue-ish.

We also need to determine from what position in a sector a player will teleport. Both the upper and lower sector should have the same 2 dimensional shape. In Duke Nukem 3D, teleportation can be specified by two Sector Effector sprites having a Lotag 7. These sprites need to be exactly in the same position in the upper and lower sectors. The Hitag value (match tag) needs to be the same:


In the screenshot above, we should see a 2D grid with two Sector Effector sprites having a Lotag of 7 (teleporter) and unique match tags (110 and 111). Both the upper and underwater sectors have exactly the same 2-dimensional shape.

In Shadow Warrior, teleportation is also controlled by sprites that should be in exactly the same position in the upper and lower sectors.

In the upper area, we need an ST1 sprite with a Hitag value of 7 and a unique Lotag value. In the underwater area, we need an ST1 sprite with an Hitag value of 8 and the same match Lotag. The latter ST1 sprite (with Hitag 8) automatically lets the player swim. If the player is an under water area where he can not surface, the match Lotag value should be 0.

In Duke Nukem 3D the landscape will automatically look blue-ish in an underwater area. To make the landscape look blue-ish in Shadow Warrior, we need to adjust the palette of the walls, floors and ceilings from 0 to 9.


Garage doors


In my map, I commonly use garage/DOOM-style doors that move up when you touch them.


In Duke Nukem 3D, we can turn a sector into a garage door by giving it a Lotag value of 20 and lowering the ceiling in such a way that it touches the floor. By default, opening a door does not produce any sound. Moreover, a door will not close automatically.

We can adjust that behaviour by placing two special purpose sprites in the door sector:

  • By adding a MUSIC&SFX sprite we can play a sound. The Lotag value indicates the sound number. 166 is typically a good sound.
  • To automatically close the door after a certain time interval, we need to add a Sector Effector sprite with Lotag 10. The Hitag indicates the time interval. For many doors, 100 is a good value.


In the above screenshot, we can see what the garage door looks like if I slightly move the ceiling up (normally the ceiling should touch the floor). There is both a MUSIC&SFX (to give it a sound effect) as well as a Sector Effector sprite (to ensure that the door gets closed automatically) in the door sector.

In Shadow Warrior, we can accomplish the same thing by adding an ST1 sprite to the door sector with Hitag 92 (Vator). A vator is a multifunctional concept that can be used to move sectors up and down in all kinds of interesting ways.

An auto closing garage door can be configured by giving the ST1 sprite the following tag and boolean values:

  • TAG2 (Lotag). Is a match that that should refer to a unique numeric value
  • TAG3 specifies the type of vator. 0 indicates that it is operated manually or by a switch/trigger
  • TAG4 (angle) specifies the speed of the vator. 350 is a reasonable value.
  • TAG9 specifies the auto return time. 35 is a reasonable value.
  • BOOL1 specifies whether the door should be opened by default. Setting it to 1 (true) allows us to keep the door open in the editor, rather than moving the ceiling down so that it touches the floor.
  • BOOL3 specifies whether the door could crush the player. We set it to 1 to prevent this from happening.

By default, a vator moves a sector down on first use. To make the door move up, we must rotate the ST1 sprite twice in 3D mode (by pressing the F key twice).

We can configure a sound effect by placing another ST1 sprite near the door sector with a Hitag value of 134. We can use TAG4 (angle) to specify the sound number. 473 is a good value for many doors.


In the above screenshot, we should see what a garage door looks like in Shadow Warrior. The rotated ST1 sprite defines the Vator whereas the regular ST1 provides the sound effect.

Lifts


Another prominent feature of my Duke Nukem 3D map are lifts that allow the player to reach the top or roofs of the buildings.


In Duke Nukem 3D, lift mechanics are a fairly concept -- we should give a sector a Lotag value of 17 and the sector will automatically move up or down when the player presses the use key while standing in the sector. The Hitag of a MUSIC&SFX sprite determines the stop sound and a Lotag value the start sound.

In Shadow Warrior, there is no direct equivalent of the same lift concept, but we can create a switch-operated lift by using the Vator concept (the same ST1 sprite with Hitag 92 used for garage doors) with the following properties:

  • TAG2 (Lotag) should refer to a unique match tag value. The switches should use the exact same value.
  • TAG3 determines the type of vator. 1 is used to indicate that it can only be operated by switches.
  • TAG4 (Angle) determines the speed of the vator. 325 is a reasonable value.

We have to move the ST1 sprite to the same height where the lift should arrive after it was moved up.

Since it is not possible to respond to the use key while the player is standing in the sector, we have to add switches to control the lift. A possible switch is sprite number 575. The Hitag should match the Lotag value of the ST1 sprite. The switch sprite should have a Lotag value of 206 to indicate that it controls a Vator.


The above screenshot shows the result of my porting effort -- switches have been added, the MUSIC&SFX sprite was replaced by an equivalent ST1 sprite. The ST1 sprite that controls the movement is not visible because it was moved up to the same height as the adjacent upper floor.

Swinging doors


In addition to garage doors, my level also contains a number of swinging doors.

In Duke Nukem 3D, a sector can be turned into a swinging door by giving it a Lotag of 23 and moving the floor up a bit. We also need to add a Sector Effector with Lotag 11 and a unique Hitag value that acts as the door's pivot.

As with garage doors, they will not produce any sound effects or close automatically by default, unless we add a MUSIC&SFX and a Sector Effector sprite (with Lotag 10) to the door sector.


In Shadow Warrior, the rotating door concept is almost the same. We need to add an ST1 sprite with Hitag 144 and a unique Lotag value to the sector that acts as the door's pivot.

In addition, we need to add an ST1 sprite to the sector that configures a rotator:

  • TAG2/Lotag determines a unique match tag value that should be identical to the door's pivot ST1 sprite.
  • TAG3 determines the type of rotator. 0 indicates that it can be manually triggered or by a switch.
  • TAG5 determines the angle move amount. 512 specifies that it should move 90 degrees to the right. -512 is moving the door 90 degrees to the left.
  • TAG7 specifies the angle increment. 50 is a good value.
  • TAG9 specifies the auto return time. 35 is a good value.

As with garage doors, we also need to add an ST1 sprite (with Hitag 134) to produce a sound. TAG4 (the angle) can be used to specify the sound number. 170 is a good value for rotating doors.


Secret places



My map also has a number of secret places (please do not tell anyone :-) ). In Duke Nukem 3D, any sector that has a Lotag value of 32767 is considered a secret place. In Shadow Warrior the idea is the same -- any sector with a Lotag of 217 is considered a secret place.

Puzzle switches


Some Duke Nukem 3D maps also have so-called puzzle switches requiring the player to find the correct on-and-off combination to unlock something. In my map they are scattered all over the level to unlock the final key. The E2L1 map in Duke Nukem 3D shows a better example:


We can use the Hitag value to determine whether the switch needs to be switched off (0) or on (1). We can use the Lotag as a match tag to group multiple switches.

In Shadow Warrior, each switch uses a Hitag as a match tag and a Lotag value to configure the switch type. Giving a switch a Lotag value of 213 makes it a combo switch. TAG3 can be used set to 0 to indicate that it needs to be turned off and 1 that it needs to be turned on.

Skill settings


Both games have four skill levels. The idea is that the higher the skill level is, the more monsters you will have to face.

In Duke Nukem 3D you can specify the minimum skill level of a monster by giving the sprite a Lotag value that corresponds to the minimum skill level. For example, giving a monster a Lotag value of 2 means that it will only show up when the skill level is two or higher (Skill level 2 corresponds to: Let's rock). 0 (the default value) means that it will show up in any skill level:


In Shadow Warrior, each sprite has its own dedicated skill attribute that can be set by using the key combination ' + K. The skill level is displayed as one of the sprite's attributes.


In the above screenshot, the sprite on the left has a S:0 prefix meaning that it will be visible in skill level 0 or higher. The sprite on the right (with a prefix: S:2) appears from skill level 2 or higher.

End switch


In both games, you typically complete a level by touching a so-called end switch. In Duke Nukem 3D an ending switch can be created by using sprite 142 and giving it a Lotag of 65535. In Shadow Warrior the idea is the same -- we can create an end switch by using sprite 2470 and giving it a Lotag of 116.


Conclusion


In this blog post, I have described the porting process of a Duke Nukem 3D map to Shadow Warrior and explained some of the properties that are common and different in both games.

Although this project was a pretty useless project (the game is quite old, from the late 90s), I had a lot of fun doing it after not having touched this kind of technology for over a decade. I am quite happy with the result:


Despite the fact that this technology is old, I am still quite surprised to see how many maps and customizations are still being developed for these ancient games. I think this can be attributed to the fact that these engines and game mechanics are highly customizable and still relatively simple to use due to the technical limitations at the time they were developed.

Since I did most of my mapping/customization work many years before I started this blog, I thought that sharing my current experiences can be useful for others who intend to look at these games and creating their own customizations.