A layout framework experiment in JavaScript

It has been a while since I wrote a blog post about front-end web technology. The main reason is that I am not extensively doing front-end development anymore, but once in a while I still tinker with it.

In my Christmas break, I wanted to expand my knowledge about modern JavaScript programming practices. To make the learning process more motivating, I have been digging up my old web layout framework project and ported it to JavaScript.

In this blog post, I will explain the rationale of the framework and describe the features of the JavaScript version.

Background

Several years ago, I have elaborated about some of the challenges that I faced while creating layouts for web applications. Although front-end web technology (HTML and CSS) were originally created for pages (not graphical user interfaces), most web applications nowadays are complex information systems that typically have to present collections of data to end-users in a consistent manner.

Although some concepts of web technology are powerful and straight forward, a native way to isolate layout from a page's content and style is still virtually non-existent (with the exception of frames that have been deprecated a long time ago). As a consequence, it has become quite common to rely on custom abstractions and frameworks to organize layouts.

Many years ago, I also found myself repeating the same patterns to implement consistent layouts. To make my life easier, I have developed my own layout framework that allows you to define a model of your application layout, that captures common layout properties and all available sub pages and their dynamic content.

A view function can render a requested sub page, using the path in the provided URL as a selector.

I have created two implementations of the framework: one in Java and another in PHP. The Java version was the original implementation but I ended up using the PHP version the most, because nearly all of the web applications I developed were hosted at shared web hosting providers only offering PHP as a scripting language.

Something that I consider both an advantage and disadvantage of my framework is that it has to generate pages on the server-side. The advantage of this approach is that pages rendered by the framework will work in many browsers, even primitive text-oriented browsers that lack JavaScript support.

A disadvantage is that server-side scripting requires a more complex server installation. Although PHP is relatively simple to set up, a Java Servlet container install (such as Apache Tomcat) is typically more complex. For example, you typically want to put it behind a reverse proxy that serves static content more efficiently.

Furthermore, executing server-side code for each request is also significantly more expensive (in terms of processing power) than serving static files.

The interesting aspect of using JavaScript as an implementation language is that we can use the framework both on the client-side (in the browser) as well as on the server-side (with Node.js). The former aspect makes it possible to host applications on a web servers that only serve static content, making web hosting considerably easier and cheaper.

Writing an application model

As explained earlier, my layout framework separates the model from a view. An application layout model can be implemented in JavaScript as follows:

import { Application } from "js-sblayout/model/Application.mjs";

import { StaticSection } from "js-sblayout/model/section/StaticSection.mjs";
import { MenuSection } from "js-sblayout/model/section/MenuSection.mjs";
import { ContentsSection } from "js-sblayout/model/section/ContentsSection.mjs";

import { StaticContentPage } from "js-sblayout/model/page/StaticContentPage.mjs";
import { HiddenStaticContentPage } from "js-sblayout/model/page/HiddenStaticContentPage.mjs";
import { PageAlias } from "js-sblayout/model/page/PageAlias.mjs";

import { Contents } from "js-sblayout/model/page/content/Contents.mjs";

/* Create an application model */

export const application = new Application(
    /* Title */
    "My application",

    /* Styles */
    [ "default.css" ],

    /* Sections */
    {
        header: new StaticSection("header.html"),
        menu: new MenuSection(0),
        submenu: new MenuSection(1),
        contents: new ContentsSection(true)
    },

    /* Pages */
    new StaticContentPage("Home", new Contents("home.html"), {
        "404": new HiddenStaticContentPage("Page not found", new Contents("error/404.html")),

        home: new PageAlias("Home", ""),

        page1: new StaticContentPage("Page 1", new Contents("page1.html"), {
            page11: new StaticContentPage("Subpage 1.1", new Contents("page1/subpage11.html")),
            page12: new StaticContentPage("Subpage 1.2", new Contents("page1/subpage12.html")),
            page13: new StaticContentPage("Subpage 1.3", new Contents("page1/subpage13.html"))
        }),

        page2: new StaticContentPage("Page 2", new Contents("page2.html"), {
            page21: new StaticContentPage("Subpage 2.1", new Contents("page2/subpage21.html")),
            page22: new StaticContentPage("Subpage 2.2", new Contents("page2/subpage22.html")),
            page23: new StaticContentPage("Subpage 2.3", new Contents("page2/subpage23.html"))
        }),
    }),

    /* Favorite icon */
    "favicon.ico"
);

The above source code file (appmodel.mjs) defines an ECMAScript module exporting an application object. The application object defines the layout of a web application with the following properties:

• The title of the web application is: "My application".
• All pages use: default.css as a common stylesheet.
• Every page consists of a number of sections that have a specific purpose:
  
  • A static section (header) provides content that is the same for every page.
  • A menu section (menu, submenu) display links to sub pages part of the web application.
  • A content section (contents) displays variable content, such as text and images.
• An application consists of multiple pages that display the same sections. Every page object refers to a file with static HTML code providing the content that needs to be displayed in the content section.
• The last parameter refers to a favorite icon that is the same for every page.

Pages in the application model are organized in a tree-like data structure. The application constructor only accepts a single page parameter that refers to the entry page of the web application. The entry page can be reached by opening the web application from the root URL or by clicking on the logo displayed in the header section.

The entry page refers to two sub pages: page1, page2. The menu section displays links to the sub pages that are reachable from the entry page.

Every sub page can also refer to their own sub pages. The submenu section will display links to the sub pages that are reachable from a selected the sub page. For example, when page1 is selected the submenu section will display links to: page11, page12.

In addition to pages that are reachable from the menu sections, the application model also has hidden error pages and a home link that is an alias for the entry page. In many web applications, it is a common habit that in addition to clicking on the logo, a home button can also be used to redirect a user to the entry page.

Besides using the links in the menu sections, any sub page in the web application can be reached by using the URL as a selector. A common convention is to use the path components in the URL to determine which page and sub page need to be displayed.

For example, by opening the following URL in a web browser:

http://localhost/page1/page12

Brings the user to the second sub page of the first sub page.

When providing an invalid selector in the URL, such as http://localhost/page4, the framework automatically redirects the user to the 404 error page, because the page cannot be found.

Displaying sub pages in the application model

As explained earlier, to display any of the sub pages that the application model defines, we must invoke a view function.

A reasonable strategy (that should suit most needs) is to generate an HTML page, with a title tag composed the application and page's title, globally include the application and page-level stylesheets, and translate every section to a div using the section identifier as its id. The framework provides a view function that automatically performs this translation.

As a sidenote: for pages that require a more complex structure (for example, to construct a layout with more advanced visualizations), it is also possible to develop a custom view function.

We can create a custom style sheet: default.css to position the divs and give each section a unique color. By using such a stylesheet, the application model shown earlier may be presented as follows:

As can be seen in the screenshot above, the header section has a gray color and displays a logo, the menu section is blue, the submenu is red and the contents section is black.

The second sub page from the first sub page was selected (as can be seen in the URL as well as the selected buttons in the menu sections). The view functions that generate the menu sections automatically mark the selected sub pages as active.

With the Java and PHP versions (described in my previous blog post), it is a common practice to generate all requested pages server-side. With the JavaScript port, we can also use it on the client-side in addition to server-side.

Constructing an application that generates pages server-side

For creating web applications with Node.js, it is a common practice to create an application that runs its own web server.

(As a sidenote: for production environments it is typically recommended to put a more mature HTTP reverse proxy in front of the Node.js application, such as nginx. A reverse proxy is often more efficient for serving static content and has more features with regards to security etc.).

We can construct an application that runs a simple embedded HTTP server:

import { application } from "./appmodel.mjs";
import { displayRequestedPage } from "js-sblayout/view/server/index.mjs";
import { createTestServer } from "js-sblayout/testhttpserver.mjs";

const server = createTestServer(function(req, res, url) {
    displayRequestedPage(req, res, application, url.pathname);
});
server.listen(process.env.PORT || 8080);

The above Node.js application (app.mjs) performs the following steps:

• It includes the application model shown in the code fragment in the previous section.
• It constructs a simple test HTTP server that serves well-known static files by looking at common file extensions (e.g. images, stylesheets, JavaScript source files) and treats any other URL pattern as a dynamic request.
• The embedded HTTP server listens to port 8080 unless a PORT environment variable with a different value was provided.
• Dynamic URLs are handled by a callback function (last parameter). The callback invokes a view function from the framework that generates an HTML page with all properties and sections declared in the application layout model.

We can start the application as follows:

$ node app.mjs

and then use the web browser to open the root page:

http://localhost:8080

or any sub page of the application, such as the second sub page of the first sub page:

http://localhost:8080/page1/page12

Although Node.js includes a library and JavaScript interface to run an embedded HTTP server, it is very low-level. Its only purpose is to map HTTP requests (e.g. GET, POST, PUT, DELETE requests) to callback functions.

My framework contains an abstraction to construct a test HTTP server with reasonable set of features for testing web applications built with the framework, including serving commonly used static files (such as images, stylesheets and JavaScript files).

For production deployments, there is much more to consider, which is beyond the scope of my HTTP server abstraction.

It is also possible to use the de-facto web server framework for Node.js: express in combination with the layout framework:

import { application } from "./appmodel.mjs";
import { displayRequestedPage } from "js-sblayout/view/server/index.mjs";
import express from "express";

const app = express();
const port = process.env.PORT || 8080;

// Configure static file directories
app.use("/styles", express.static("styles"));
app.use("/image", express.static("image"));

// Make it possible to parse form data
app.use(express.json());
app.use(express.urlencoded({ extended: true }));

// Map all URLs to the SB layout manager
app.get('*', (req, res) => {
    displayRequestedPage(req, res, application, req.url);
});

app.post('*', (req, res) => {
    displayRequestedPage(req, res, application, req.url);
});

// Configure listening port
app.listen(port, () => {
    console.log("Application listening on port " + port);
});

The above application invokes express to construct an HTTP web server that listens to port 8080 by default.

In addition, express has been configured to serve static files from the styles and image folders, and maps all dynamic GET and POST requests to the displayRequestedPage view function of the layout framework.

Using the model client-side and dynamically updating the DOM

As already explained, using JavaScript as an implementation language also makes it possible to directly consume the application model in the browser and dynamically generate pages from it.

To make this possible, we only have to write a very minimal static HTML page:

<!DOCTYPE html>

<html>
    <head>
        <title>My page</title>
        <script type="module">
import { application } from "./appmodel.mjs";
import { initRequestedPage, updateRequestedPage } from "js-sblayout/view/client/index.mjs";

document.body.onload = function() {
    initRequestedPage(application);
};

document.body.onpopstate = function() {
    updateRequestedPage(application);
};
        </script>
    </head>

    <body>
    </body>
</html>

The above HTML page has the following properties:

• It contains the bare minimum of HTML code to construct a page that is still valid HTML5.
• We include the application model (shown earlier) that is identical to the application model that we have been using to generate pages server-side.
• We configure two event handlers. When the page is loaded (onload) we initially render all required page elements in the DOM (including the sections that translate to divs). Whenever the user clicks on a link (onpopstate), we update the affected sections in the DOM.

To make the links in the menu sections work, we have to compose them in a slightly different way -- rather than using the path to derive the selected sub page, we have to use hashes instead.

For example, the second sub page of the first page can be reached by opening the following URL:

http://localhost/index.html#/page1/page21

The popstate event triggers whenever the browser's history changes, and makes it possible for the user to use the back and forward navigation buttons.

Generating dynamic content

In the example application model shown earlier, all sections are made out of static HTML code fragments. Sometimes it may also be desired to generate the sections' content dynamically, for example, to respond to user input.

In addition to providing a string with a static HTML code as a parameter, it is also possible to provide a function that generates the content of the section dynamically.

new StaticContentPage("Home", new Contents("home.html"), {
    ...
    hello: new StaticContentPage("Hello 10 times", new Contents(displayHello10Times))
})

In the above code fragment, we have added a new sub page the to entry page that refers to the function: displayHello10Times to dynamically generate content. The purpose of this function is to display the string: "Hello" 10 times:

When writing an application that generates pages server-side, we could implement this function as follows:

function displayHello10Times(req, res) {
    for(let i = 0; i < 10; i++) {
        res.write("<p>Hello!</p>\n");
    }
}

The above function follows a convention that is commonly used by applications using Node.js internal HTTP server:

• The req parameter refers to the Node.js internal HTTP server's http.IncomingMessage object and can be used to retrieve HTTP headers and other request parameters.
• The req.sbLayout parameter provides parameters that are related to the layout framework.
• The res parameter refers to the Node.js internal HTTP server's http.ServerResponse object and can be used to generate a response message.

It is also allowed to declare the function above async or let it return a Promise so that asynchronous APIs can be used.

When developing a client-side application (that dynamically updates the browser DOM), this function should have a different signature:

function displayHello10Times(div, params) {
    let response = "";

    for(let i = 0; i < 10; i++) {
        response += "<p>Hello!</p>\n";
    }

    div.innerHTML = response;
}

In the browser, a dynamic content generation function accepts two parameters:

• div refers to an HTMLDivElement in the DOM that contains the content of the section.
• params provides layout framework specific properties (identical to req.sbLayout in the server-side example).

Using a templating engine

Providing functions that generate dynamic content (by embedding HTML code in strings) may not always be the most intuitive way to generate dynamic content. It is also possible to configure template handlers: the framework can invoke a template handler function for files with a certain extension.

In the following server-side example, we define a template handler for files with an .ejs extension to use the EJS templating engine:

import { application } from "./appmodel.mjs";
import { displayRequestedPage } from "js-sblayout/view/server/index.mjs";
import { createTestServer } from "js-sblayout/testhttpserver.mjs";

import * as ejs from "ejs";

function renderEJSTemplate(req, res, sectionFile) {
    return new Promise((resolve, reject) => {
        ejs.renderFile(sectionFile, { req: req, res: res }, {}, function(err, str) {
            if(err) {
                reject(err);
            } else {
                res.write(str);
                resolve();
            }
        });
    });
}

const server = createTestServer(function(req, res, url) {
    displayRequestedPage(req, res, application, url.pathname, {
        ejs: renderEJSTemplate
    });
});
server.listen(process.env.PORT || 8080);

In the above code fragment, the renderEJSTemplate function is used to open an .ejs template file and uses ejs.renderFile function to render the template. The resulting string is propagated as a response to the user.

To use the template handlers, we invoke the displayRequestedPage with an additional parameter that maps the ejs file extension to the template handler function.

In a client-side/browser application, we can define a template handler as follows:

<!DOCTYPE html>

<html>
    <head>
        <title>My page</title>
        <script type="text/javascript" src="ejs.js"></script>
        <script type="module">
import { application } from "./appmodel.mjs";
import { initRequestedPage, updateRequestedPage } from "js-sblayout/view/client/index.mjs";

const templateHandlers = {
  ejs: function(div, response) {
      return ejs.render(response, {});
  }
}

document.body.onload = function() {
    initRequestedPage(application, templateHandlers);
};

document.body.onpopstate = function() {
    updateRequestedPage(application, templateHandlers);
};
        </script>
    </head>

    <body>
    </body>
</html>

In the above code fragment, we define a templateHandlers object that gets propagated to the view function that initially renders the page (initRequestedPage) and dynamically updates the page (updateRequestedPage).

By adding the following sub page to the entry page, we can use an ejs template file to dynamically generate a page rather than a static HTML file or function:

new StaticContentPage("Home", new Contents("home.html"), {
    ...
    stats: new StaticContentPage("Stats", new Contents("stats.ejs"))
})

In a server-side application, we can use stats.ejs to display request variables:

<h2>Request parameters</h2>

<table>
    <tr>
        <th>HTTP version</th>
        <td><%= req.httpVersion %></td>
    </tr>
    <tr>
        <th>Method</th>
        <td><%= req.method %></td>
    </tr>
    <tr>
        <th>URL</th>
        <td><%= req.url %></td>
    </tr>
</table>

resulting in a page that may have the following look:

In a client-side application, we can use stats.ejs to display browser variables:

<h2>Some parameters</h2>

<table>
    <tr>
        <th>Location URL</th>
        <td><%= window.location.href %></td>
    </tr>
    <tr>
        <th>Browser languages</th>
        <td>
        <%
        navigator.languages.forEach(language => {
            %>
            <%= language %><br>
            <%
        });
        %>
        </td>
    </tr>
    <tr>
        <th>Browser code name</th>
        <td><%= navigator.appCodeName %></td>
    </tr>
</table>

displaying the following page:

Strict section and page key ordering

In all the examples shown previously, we have used an Object to define sections and sub pages. In JavaScript, the order of keys in an object is somewhat deterministic but not entirely -- for example, numeric keys will typically appear before keys that are arbitrary strings, regardless of the insertion order.

As a consequence, the order of the pages and sections may not be the same as the order in which the keys are declared.

When the object key ordering is a problem, it is also possible to use iterable objects, such as a nested array, to ensure strict key ordering:

import { Application } from "js-sblayout/model/Application.mjs";

import { StaticSection } from "js-sblayout/model/section/StaticSection.mjs";
import { MenuSection } from "js-sblayout/model/section/MenuSection.mjs";
import { ContentsSection } from "js-sblayout/model/section/ContentsSection.mjs";

import { StaticContentPage } from "js-sblayout/model/page/StaticContentPage.mjs";
import { HiddenStaticContentPage } from "js-sblayout/model/page/HiddenStaticContentPage.mjs";
import { PageAlias } from "js-sblayout/model/page/PageAlias.mjs";

import { Contents } from "js-sblayout/model/page/content/Contents.mjs";

/* Create an application model */

export const application = new Application(
    /* Title */
    "My application",

    /* Styles */
    [ "default.css" ],

    /* Sections */
    [
        [ "header", new StaticSection("header.html") ],
        [ "menu", new MenuSection(0) ],
        [ "submenu", new MenuSection(1) ],
        [ "contents", new ContentsSection(true) ],
        [ 1, new StaticSection("footer.html") ]
    ],

    /* Pages */
    new StaticContentPage("Home", new Contents("home.html"), [
        [ 404, new HiddenStaticContentPage("Page not found", new Contents("error/404.html")) ],

        [ "home", new PageAlias("Home", "") ],

        [ "page1", new StaticContentPage("Page 1", new Contents("page1.html"), [
            [ "page11", new StaticContentPage("Subpage 1.1", new Contents("page1/subpage11.html")) ],
            [ "page12", new StaticContentPage("Subpage 1.2", new Contents("page1/subpage12.html")) ],
            [ "page13", new StaticContentPage("Subpage 1.3", new Contents("page1/subpage13.html")) ]
        ])],

        [ "page2", new StaticContentPage("Page 2", new Contents("page2.html"), [
            [ "page21", new StaticContentPage("Subpage 2.1", new Contents("page2/subpage21.html")) ],
            [ "page22", new StaticContentPage("Subpage 2.2", new Contents("page2/subpage22.html")) ],
            [ "page23", new StaticContentPage("Subpage 2.3", new Contents("page2/subpage23.html")) ]
        ])],
        
        [ 0, new StaticContentPage("Last page", new Contents("lastpage.html")) ]
    ]),

    /* Favorite icon */
    "favicon.ico"
);

In the above example, we have rewritten the application model example to use strict key ordering. We have added a section with numeric key: 1 and a sub page with key: 0. Because we have defined a nested array (instead of an object), these section and page will come last (if we would have used an object, then they will appear first, which is undesired).

Internally, the Application and Page objects use a Map to ensure strict ordering.

More features

The framework has full feature parity with the PHP and Java implementations of the layout framework. In addition to the features described in the previous sections, it can also do the following:

• Work with multiple content sections. In our examples, we only have one content section that changes when picking a menu item, but it is also possible to have multiple content sections.
• Page specific stylesheets and JavaScript includes. Besides including CSS stylesheets and JavaScript files globally it can also be done on page level.
• Using path components as parameters. Instead of selecting a sub page, it is also possible to treat a path component as a parameter and dynamically generate a response.
• Internationalized pages. Each sub page uses a ISO localization code and the framework will pick the most suitable language in which the page should be displayed by default.
• Security handlers. Every page can implements its own method that checks whether it should be accessible or not according to a custom security policy.
• Controllers. It is also possible to process GET or POST parameters before the page gets rendered to decide what to do with them, such as validation.

Conclusion

In this blog post, I have described the features of the JavaScript port of my layout framework. In addition to rendering pages server-side, it can also be directly used in the web browser to dynamically update the DOM. For the latter aspect, it is not required to run any server-side scripting language making application deployments considerably easier.

One of the things I liked about this experiment is that the layout model is sufficiently high-level so that it can be used in a variety of application domains. To make client-side rendering possible, I only had to develop another view function. The implementation of the model aspect is exactly the same for server-side and client-side rendering.

Moreover, the newer features of the JavaScript language (most notably ECMAScript modules) make it much easier to reuse code between Node.js and web browsers. Before ECMAScript modules were adopted by browser vendors, there was no module system in the browser at all (Node.js has CommonJS) forcing me to implement all kinds of tricks to make a reusable implementation between Node.js and browsers possible.

As explained in the introduction of this blog post, web front-end technologies do not have a separated layout concern. A possible solution to cope with this limitation is to generate pages server-side. With the JavaScript implementation this is no longer required, because it can also be directly done in the browser.

However, this does still not fully solve my layout frustrations. For example, dynamically generated pages are poorly visible to search engines. Moreover, a dynamically rendered web application is useless to users that have JavaScript disabled, or a web browser that does not support JavaScript, such as text browsers.

Using JavaScript also breaks the declarative nature of web applications -- HTML and CSS allow you to write what the structure and style of a page without specifying how to render it. This has all kinds of advantages, such as the ability to degrade gracefully when certain features cannot be used, such as graphics. With JavaScript some of these properties are lost.

Still, this project was a nice distraction -- I already had the idea to explore this for several years. During the COVID-19 pandemic, I have read quite a few technical books, such as JavaScript: The Definitive Guide and learned that with the introduction of new language JavaScript features, such as ECMAScript modules, it would be possible to exactly the same implementation of the model both server-side and client-side.

As explained in my blog reflection over 2021, I have been overly focused on a single goal for almost two years and it started to negatively affect my energy level. This project was a nice short distraction.

Future work

I have also been investigating whether I could use my framework to create offline web applications with a consistent layout. Unfortunately, it does not seem to be very straight forward to do that.

It seems that it is not allowed to do any module imports from local files for security reasons. In theory, this restriction can be bypassed by packing up all the modules into a single JavaScript include with webpack.

However, it turns out that there is another problem -- it is also not possible to open any files from the local drive for security reasons. There is a file system access API in development, that is still not finished or mature yet.

Some day, when these APIs have become more mature, I may revisit this problem and revise my framework to also make offline web applications possible.

Availability

The JavaScript port of my layout framework can be obtained from my GitHub page. To use this framework client-side, a modern web browser is required, such as Mozilla Firefox or Google Chrome.