Monday, May 2, 2011

Deployment abstractions for WebDSL

Four years ago, I joined the Software Engineering Research Group as a master's student, to work on a research assignment (which served as a preparation of my masters thesis about the initial prototype of Disnix). This is how I met my supervisor Eelco Visser and many colleagues I'm working with. Back then, my research was about the deployment aspects of WebDSL, a domain-specific language for developing web application with a rich data model, using Nix tooling.

Although I'm not a primary developer of WebDSL, I am involved in it's deployment related aspects. In this blog post I'd like to give some general information about WebDSL and some ideas I'm currently working on as part of my research.

WebDSL is a Domain Specific Language (DSL) for web applications with a rich data model started by Eelco Visser. While developing applications, people often make abstractions. This is done to deal with repetition and patterns which look similar, which otherwise have to be implemented over and over again. It also allows developers to talk about concrete (high-level) objects from their domain and prevents them to reinvent the wheel.

In many cases, programming language features are used to make abstractions, such as inheritance in an Object-oriented programming language. These conventional manners (e.g. method and classes) are not always sufficient. Sometimes libraries or frameworks have awkward interfaces. Some parts of a program are not checked by the compiler, such as SQL queries embedded in strings, which may be desirable. In many cases, there is a considerable distance between the programming language concepts and domain concepts. By developing a DSL you can deal with such issues and provide better abstractions for a particular domain. The definition of a DSL is as follows (according to [1]):

A DSL is a high-level software implementation language that supports concepts and abstractions that are related to a particular (application) domain.

WebDSL is a case study for DSL engineering. The web application domain is quite challenging. Many web applications are implemented in various languages, each used for a specific concern, e.g. CSS for styling, HTML for web pages (which are sometimes generated on the server using a PHP script or Java Servlet), JavaScript for manipulating the DOM on the client and so on. Moreover, sometimes a lot of boilerplate code has to be written, e.g. writing getters and setters in Java code.

WebDSL has a collection of intergrated sub languages for developing web applications, to allow a developer to specify various aspects of a web application. The WebDSL compiler generates all the implementation code for those aspects. Currently, WebDSL has sub languages to describe a data model capturing entities and their relationships, pages, access control and a work flow language.

Although building abstractions for the web is not really new (there are many frameworks available providing similar concepts) WebDSL has some unique aspects, such as the fact that it is a statically typed and a checked language and it uses a custom syntax, which can be transformed to various back-ends.

WebDSL also has a number of sister projects. Spoofax is an Eclipse based platform for developing textual domain-specific languages with full-featured Eclipse editor plugins. Spoofax itself is based on SDF2 and Stratego, which are used to implement the WebDSL grammars and compiler. Spoofax is used to provide an Eclipse IDE for WebDSL (shown in the screenshot above). As a sidenote, Spoofax is also used for other DSLs, such as Mobl, a DSL for mobile web applications and even for an IDE for Nix expressions (which is still highly experimental). Another sister project is Acoda, which can be used for automated coupled evolution of data models in WebDSL.

Although a high-level DSL for web applications and all its related features seem to be very nice, it will not solve all web application issues. Except for the development of web applications, web applications must also be deployed in a test environment or production environment, which is also quite challenging.

In order to make a WebDSL application ready for use, the WebDSL compiler itself and all its dependencies (such as SDF2 and Stratego) must be built and present. Apart for the compiler, we must also setup a necessary infrastructure capable of running a WebDSL application. For example, for the Java backend, you need to install a MySQL DBMS instance and an Apache Tomcat server (or another Java servlet container) configured with the right properties to host the generated Java web application and the underlying data model.

Moreover, in production environments, which may serve many users, other facilities must be present, such as multiple instances of Apache Tomcat and MySQL for load balancing (or fallbacks), a reverse proxy and so on. Furthermore, components of web applications are not always deployed on physical machines, but they may also be deployed in virtual machines for testing or in a cloud environment such as Amazon EC2, which have different characteristics and contraints.

Currently, apart from a prototype that deploys a WebDSL application on a single physical machine, there are no facilities to deal with complex networks of machines and cloud providers using virtual machines and their deployment processes. So in order to get a WebDSL application working in a complex distributed environment, deployment steps must be performed manually by a developer or system administrator which can be quite a burden.

I'm currently working on a solution that allows developers to deploy WebDSL applications in a custom infrastructure (consisting of physical machines and/or virtual machines) using simple high-level specifications. The tooling builds upon our earlier work on Nix, Disnix and NixOS.


The WebDSL researchr page gives some references to published WebDSL papers. [1] refers to the paper GTTSE paper titled: 'WebDSL: A Case Study in Domain-Specific Language Engineering' written by Eelco Visser.