On reading research papers and maintaining knowledge

Ten years ago I have obtained my PhD degree and made my (somewhat gradual) transition from academia to industry. Despite the fact that I made this transition a long time ago, I still often get questions from people who are considering doing a PhD.

Most of the discussions that I typically have with such people are about writing -- I have already explained plenty about writing in the past, including a recommendation to start a blog so that writing becomes a habit. Having a blog allows you to break up your work into manageable pieces and build up an audience for your work.

Recently, I have been elaborately reorganizing files on my hard-drive, a tedious task that I often do at the end of the year. This year, I have also been restructuring my private collection of research papers.

Reading research papers became a habit while working on my master's thesis and doing my PhD. Although I have left academia a long time ago, I have retained the habit, although the amount of papers and articles that I read today is much lower than my PhD days. I no longer need to study research works much, but I have retained the habit to absorb existing knowledge and put things into context whenever I intend to do something new, for example for my blog posts or software projects.

In 2020, during the first year of the COVID pandemic, I have increased my interest in research papers somewhat, because I had to revise some of the implementations of algorithms in the Dynamic Disnix framework that were based on work done by other researchers. Fortunately, the ACM temporarily opened their entire digital library to the public for free so that I could get access to quite an amount of interesting papers, without requiring to pay.

In addition to writing, reading in academic research is also very important, for the following reasons:

• To expand your knowledge about your research domain.
• To put your own work into context. If you want to publish a paper about your work, having a cool idea is not enough -- you have to explain what your research contributes: what the innovation is. As a result, you need to relate to earlier work and (optionally) to studies that motivate the relevance of your work. Furthermore, you can also not just take credit for work that has already been done by others. As a consequence, you need to very carefully investigate what is out there.
• You may have to peer review papers for acceptance in conference proceedings and journals.

Reading papers is not an easy job -- it often takes me quite a bit of time and dedication to fully grasp a paper.

Moreover, when studying a research paper, you may also have to dive into related work (by examining a paper's references, and the references of these) to fully get an understanding. You may have to dive several levels deep to gain enough understanding, which is not a straightforward job.

In this blog post, I want to share my personal experiences with reading papers and maintaining knowledge.

My personal history with reading

I have an interesting relationship with reading. Already at young age, I used to study programming books to expand my programming knowledge.

With only limited education and knowledge, I was very practically minded -- I would relentlessly study books and magazines to figure out how to get things done, but as soon as I figured out enough to get something done I stopped reading, something that I consider a huge drawback of the younger version of me.

For example, I still vividly remember how I used to program 2D side scroller games for the Commodore Amiga 500 using the AMOS BASIC programming language. I figured out many basic concepts by reading books, magazines and the help pages, such as how to load IFF/ILBM pictures as backgrounds, load ProTracker modules as background music, using blitter objects for moving actors, using a double buffer to smoothly draw graphics, side scrolling, responding to joystick input etc.

Although I have managed to make somewhat playable games by figuring out these concepts, the games were often plagued by bugs and very slow performance. A big reason that contributed to these failures is because I stopped reading after mastering the basics.

For example, to improve performance, I should have disabled the autoback feature that automatically swaps the physical and logical screens on every drawing command and do the screen swap manually after all drawing instructions were completed. I knew that using a double screen buffer would take graphics glitches away, but I never bothered to study the concepts behind it.

As I grew older and entered middle school, I became more critical of myself. For example, I learned that it is essential to properly cite where you get your knowledge from rather than "pretending" that you are pulling something out of your own hat. :)

Fast forwarding to my studies at the university: reading papers from the academic literature became something that I had to commonly do. For example, I still remember the real-time systems and software architecture courses.

The end goal of the former course was to write your own research paper about a subject in the real-time systems domain. In this course, I learned, in addition to real-time system concepts, how academic research works: writing a research paper is not just about writing down a cool idea (with references that you used as an inspiration), but you also need to put your work into context -- a research paper is typically based on work already done by others, and your paper typically serves as an ingredient that can be picked up by other researchers.

In the latter course, I had to read quite a few papers in the software architecture domain, write summaries and discuss my findings with other students. Here, I learned that reading papers is all but a trivial job:

• Papers are often densely written. As a result, I get overwhelmed with information and it requires quite a bit of energy from my side to consume all of it.
• The formatting of many papers is not always helpful. Papers are typically written for print, not for reading from a screen. Also, the formatting of papers are not always good for displaying code fragments or diagrams.
• There is often quite a bit of unexplained jargon in a paper. To get a better understanding you need to dive deeper into the literature, such as also studying the references of the papers or books that are related to the subject.
• Sometimes authors frequently use multi-syllable words.
• It is also not uncommon for authors to use logic and formulas to formalize concepts and mathematically prove their contributions. Although formalization helps to do this, reading formulas is often a tough job for me -- there is typically a huge load of information and Greek symbols. These symbols IMO are not always very helpful to relate to what concepts they represent.
• Authors often tend to elaborately stress out the caveats of their contributions, making things hard to read.

Despite having read many papers in the last 16 years and I got better at it, reading still remains a tough job because of the above reasons.

In the final year of my master's, I had to do a literature survey before starting the work on my master's thesis. The first time I heard about this, I felt scared, because of my past experiences with reading papers.

Fortunately, my former supervisor: Eelco Visser, was very practically minded about the process -- he wanted us to first work on practical aspects of their research projects, such as WebDSL: a domain-specific language for developing web applications with a rich data model and related tools, such as Stratego/XT and the Nix package manager.

After mastering the practical concepts of these projects, doing a literature survey felt much easier -- instinctively, while using these tools in practice, I became more interested in learning about the concepts behind them. Many of their underlying concepts were described in research papers published my my colleagues in the same research department. While studying these papers, I also got more motivated/interested into diving deeper in the academic literature by studying the papers' references and searching for related subjects in the digital libraries of the ACM, IEEE, USENIX, Springer, Elsevier etc.

During my PhD reading research papers became even more important. In the first six months of my PhD, I had a very good start. I published a paper about an important aspect of my master's thesis: atomic upgrading of the static parts of a distributed system, and a paper about the overall objective of the research project that I was in. I have to admit that, despite having these papers instantly accepted, I still had the wrong mindset -- I was basically just "selling my cool ideas" and finding support in the academic literature, rather than critically studying what is out there.

For my third paper, that covers a new implementation of Disnix (the third major revision to be precise), I learned an important/hard lesson. The first version of the paper got badly rejected by the program committee, because of my "advertising cool ideas mindset" that I always used to have -- I failed to study the academic literature well enough to explain what the innovation of my paper is in comparison to other deployment solutions. As a consequence, I got some very hard criticisms from the reviewers.

Fortunately, they gave me good feedback. For example, I had to study papers from the Working Conference on Component Deployment. I have addressed their criticisms and the revised paper got accepted. I learned what I had to do in the future -- it is a requirement to also study the academic literature well enough to explain what your contribution is and demonstrate its relevance.

This rejection also changed my attitude how I deal with research papers. Previously, after my work for a paper was done, I would typically discard the artifacts that I no longer needed, including the papers that I used as a reference. After this rejection, I learned that I need to build my own personal knowledge base so that for future work, I could always relate to the things that I have read previously.

Reading research papers

I have already explained that for various reasons, reading research papers is all but an easy job. For some papers, in particular the ones in my former research domain: software deployment, I got better as I grew more familiar to the research domain.

Nonetheless, I still sometimes find reading papers challenging. For example, studying algorithmic papers is extremely hard IMO. In 2021, I had to revise my implementations of approximation solutions for the multi-way cut, and graph coloring problems in the Dynamic Disnix framework. I had to re-read the corresponding papers again. Because they were so hard to grasp, I wrote a blog post that explains how I practically applied them.

To fully grasp a paper, reading it a single time is often not enough. In particular the algorithmic papers that I mentioned earlier, I had to read them many times.

Interestingly enough, I learned that reading papers is also a subject of study. A couple of years ago I discovered a paper titled: "How to Read a Paper" that explains a strategy for reading research papers using a three-pass approach:

• First pass: bird's eye view. Study the title, abstract, introduction, headings, conclusions. A single pass is often already enough to decide whether a paper is relevant to read or not.
• Second pass: study in greater detail, but ignore the big details, such as mathematical proofs.
• Third pass: read everything in detail by attempting to virtually re-implement the paper.

After discovering this paper, I have also been using the three pass approach. I have studied most of my papers in my collection in two passes, and some of them in detail in three passes.

Another thing that I discovered by accident is that to extensively study literature, a continuous approach works better for me (e.g. reserving certain timeslots in a week) than just reserving longer periods of time that consist of only reading papers.

Also, regularly discussing papers with your colleagues helps. During my PhD days, I did not do it that often (we had no formal "process" for it) but there were several good sessions, such as a program committee simulation organized by Arie van Deursen, head of our research group.

In this simulation, we organized a program committee meeting of the ICSE conference in which the members of the department represented program committee members. We have discussed submitted papers and voted for acceptance or rejection. Moreover, we also had to leave the room if there was a conflict of interest.

I also learned that Edsger Dijkstra, a famous Dutch computer scientist, organized the ETAC (Eindhoven Tuesday Afternoon Club) and ATAC (Austin Tuesday Afternoon Club) in which amongst other activities, reading and discussing research papers was a recurring activity.

Building up your personal knowledge base

As I have explained earlier, I used to throw away my downloaded papers when the work for a paper was done, but I changed that habit after that hard paper rejection.

There are many good reasons to keep and organize the papers that you have read, even if they do not seem to be directly relevant to your work:

• As I have already explained, in addition to reading a single paper and writing your own research papers, you need to maintain your knowledge base so that you can put them into context.
• It is not always easy to obtain papers. Many of them are behind a paywall. Without a subscription you cannot access them, so once you have obtained them it is better to think twice before you discard them. Fortunately, open access becomes more common but it still remains a challenge. Arie van Deursen has written a variety of blog posts about open access.
• Although many papers are challenging to read, I also started to appreciate certain research papers.

My own personal paper collection has evolved in an interesting way. In the beginning, I just used to put any paper that I have obtained into a single folder called: papers until it grew large enough that I had to start classifying them.

Initially, there was a one-level folder structure, consisting of categories such as: deployment, operating systems, programming languages, DSL engineering etc. At some point, the content of some of these folders grew large enough and I introduced a second level directory structure.

For example, the sub folder for my former research domain: software deployment (the process that consists of all activities to make a software system available for use) contains the largest amount of papers. Currently, I have collected 168 deployment papers that I have divided over the following sub categories:

• Deployment models. Papers whose main contribution is a means to model various deployment aspects of a system, such as the structure of a system and deployment activities.
• Deployment planning. Papers whose main contribution are algorithms that decide a suitable/optimal deployment architecture based on functional and non-functional requirements of a system.
• Empirical studies. Papers containing empirical studies about deployment in practice.
• Execution. Papers in which the main contribution is executing deployment activities. I have also sub categorized this folder into technology-specific solutions (e.g. a solution is specific to a programming language, such as Java or component technology, such as CORBA) and generic solutions.
• Practice reports. Papers that report on the use of deployment technologies in practice.
• Surveys. Papers that analyse the literature and draw conclusions from them.

A hierarchical directory structure is not perfect for organizing papers -- for many papers there is an overlap between multiple sub domains in the software engineering domain. For example, deployment may also be related to a certain component technology, in service of optimizing the architecture of a system, related to other configuration management activities (versioning, status accounting, monitoring etc.) or an ingredient in integration testing. If there is an overlap, I typically look at the strongest kind of contribution that the paper makes.

For example, in the deployment domain, Eelco Dolstra wrote a paper about maximal laziness, an important implementation aspect of the Nix expression language. The Nix package manager is a deployment solution, but the contribution of the paper is not deployment, but making the implementation of a purely functional DSL efficient. As a result, I have categorized the paper under DSL engineering rather than deployment.

The organization of my paper collection is always in motion. Sometimes I gain new insights causing me to adjust the classifications, or when a collection of papers for a sub domain grows, I may introduce a second-level classification.

Some practical tips to get familiar with a certain research subject

So what is my recommended way to get familiar with a certain research subject in the software engineering domain?

I would start by doing something practical first. In software engineering research domain, often the goal is to develop or examine tools. Start by using these tools first and see if you can contribute to them from a practical point of view -- for example, by improving features, fixing bugs etc.

As soon as I have mastered the practical aspects, I may typically already get motivated to dive into their underlying concepts by studying the papers that cover them. Then I will apply the three pass reading strategy and eventually study the references of the papers to get a better understanding.

After my orientation phase has finished, the next thing I would typically look at is the conferences/venues that are directly related to the subject. For software deployment, for example, there used to be only one subject-related conference: the Working Conference On Component Deployment (that unfortunately was no longer organized after 2005). It is typically a good thing to have examined all the papers of the related conferences/venues, by at least using a first-pass approach.

Then a potential next step is to search for "early defining papers" in that research area. In my experience, many research papers are improving on concepts pioneered by these papers, so it is IMO a good thing to know where it all started.

For example, in the software deployment domain the paper: "A Characterization Framework for Software Deployment Technologies" is such an early defining paper, covering a deployment solution called "The Software Dock". The paper comes with a definition for the term: "software deployment" that is considered the canonical definition in academic research.

Alternatively, the paper: "Software Deployment, Past, Present and Future" is a more recent yet defining paper covering newer deployment technologies and also offers its own definition of the term software deployment.

For unknown reasons, I always seem to like early defining papers in various software engineering domains. These are some of my recommendations of early defining papers in other software engineering domains:

• General purpose programming languages: The IBM 701 Speedcoding System, The FORTRAN Automatic Coding System, Recursive Functions of Symbolic Expressions and Their Computation by Machine, Part I, Modified Report on the Algorithmic Language ALGOL 60.
• Domain-specific languages: Programming Pearls: Little Languages, Domain-Specific Languages: An Annotated Bibliography
• Programming: A case against the GO TO statement, Notes on Structured Programming
• Operating systems: The Structure of the "THE"-multiprogramming System, The Multics Input/Output System, A General-Purpose File System for Secondary Storage, The UNIX Time-Sharing System
• Software engineering in general: Research paradigms in computer science

After studying all these kinds of papers, your knowledge level should already be decent enough to find your way to study the remaining papers that are out there.

Literature surveys

In addition to research papers that need to put themselves into context, extensive literature surveys can also be quite valuable to the research community. During my PhD, I learned that it is also possible to publish a paper about a literature survey.

For example, some of my former colleagues did an extensive and systematic literature survey in the dynamic analysis domain. In addition to the results, the authors also explain their methodology, that consists of searching on keywords, looking for appropriate conferences and journals and following the papers' references. From these results they have derived an attribute framework and classified all the papers into this attribute framework.

I have kept the paper as a reference for myself, because I like the methodology. I am not so interested in dynamic analysis or program comprehension from a research perspective.

Literature surveys also exist in my former research domain, such as a survey of deployment solutions for distributed systems.

Conclusions

In this blog post, I have shared my experiences with reading papers and maintaining knowledge. In research, it is quite important and you need to take it seriously.

Fortunately, during my PhD I have learned a lot. In summary, my recommendations are:

• Archive your papers and build up a personal knowledge base
• Start with something practical
• Follow paper references
• Study early defining papers
• Find people to discuss with
• Study continuously in small steps

Although I never did an extensive literature survey in the software deployment domain (it is not needed for submitting papers that contribute new techniques) I can probably even write a paper about software deployment literature myself. The only problem is that I am not quite up to date with work that has been published in the last few years, because I no longer have access to these digital libraries.

Moreover, I also need to find the time and energy to do it, if I really want to :)

In memoriam: Eelco Visser (1966-2022)

On Tuesday 5 April 2022 I received the unfortunate news that my former master's and PhD thesis supervisor: Eelco Visser has unexpectedly passed away.

Although I made my transition from academia to industry almost 10 years ago and I have not been actively doing academic research anymore (I published my last paper in 2014, almost two years after completing my PhD), I have always remained (somewhat) connected to the research world and the work carried out by my former supervisor, who started his own programming languages research group in 2013.

He was very instrumental in the domain-specific programming languages research domain, but also in the software deployment research domain, a very essential part in almost any software development process.

Without him and his ideas, his former PhD student: Eelco Dolstra would probably never have started the work that resulted in the Nix package manager. As a consequence, my research on software deployment (resulting in Disnix and many other Nix-related tools and articles) and this blog would also not exist.

In this blog post, I will share some memories of my time working with Eelco Visser.

How it all started

The first time I met Eelco was in 2007 when I was still a MSc student. I just completed the first year of the TU Delft master's programme and I was looking for an assignment for my master's thesis.

Earlier that year, I was introduced to a concept called model-driven development (also ambiguously called model-driven engineering/architecture, the right terminology is open to interpretation) in a guest lecture by Jos Warmer in the software architecture course.

Modeling software systems and automatically generating code (as much as possible), was one of the aspects that really fascinated me. Back then, I was already convinced that working from a higher abstraction level, with more "accessible" building blocks could be quite useful to hide complexity, reduce the chances on errors and make developers more productive.

In my first conversation with Eelco, he asked me why I was looking for a model-driven development assignment and he asked me various questions about my past experience.

I told him about my experiences with Jos Warmer's lecture. Although he seemed to understand my enthusiasm, he also explained me that his work was mostly about creating textual languages, not visual languages such as UML profiles, that are commonly used in MDA development processes.

He also specifically asked me about the compiler construction course (also part of the master's programme), that is required for essential basic knowledge about textual languages.

The compiler construction course (as it was taught in 2007) was considered to be very complex by many students, in particular the practical assignment. As a practical assignment, you had to rewrite the a parser from using GNU Bison (a LARL(1) parser) to LLnextgen (a LL(1) parser) and extend the reference compiler with additional object-oriented programming features. Moreover, the compiler was implemented in C, and relied on advanced concepts, such as function pointers and proper alignment of members in a struct.

I explained Eelco that despite the negative image of the course because of its complexity, I actually liked it very much. Already at a young age I had the idea to develop my own programming language, but I had no idea how to do it, but when I was exposed to all these tools and concepts I finally learned about all the missing bits and pieces.

I was also trying to convince him that I am always motivated to deep dive into technical details. As an example, I explained him that one of my personal projects is creating customized Linux distributions by following the Linux from Scratch book. Manually following all the instructions in the book is time consuming and difficult to repeat. To make deploying a customized Linux distribution doable, I developed my own automated solution.

After elaborating about my (somewhat crazy) personal project, he told me that there is an ongoing research project that I will probably like very much. A former PhD student of his: Eelco Dolstra developed the Nix package manager and this package manager was used as the foundation for an entire Linux distribution: NixOS.

He gave me a printed copy of Eelco Dolstra's thesis and convinced me that I should give NixOS a try.

Research assignment

After reading Eelco Dolstra's PhD thesis and trying out NixOS (that was much more primitive in terms of features compared to today's version), Eelco Visser gave me my first research assignment.

When he joined Delft University of Technology in 2006 (a year before I met him) as an associate professor, he started working on a new project called: WebDSL. Previously, most of his work was focused on the development of various kinds meta-tools for creating domain specific languages, such as:

• SDF2 is a formalism used to write a lexical and context free syntax. It has many interesting features, such as a module system and scannerless parsing, making it possible to embed a guest language in an host language (that may share the same keywords). SDF2 was originally developed by Eelco Visser for his PhD thesis for the ASF+SDF Meta Environment.
• ATerm library. A library that implements the annotated terms format to exchange structured data between tools. SDF2 uses it to encode parse/abstract syntax trees.
• Stratego/XT. A language and toolset for program transformation.

WebDSL was a new step for him, because it is an application language (built with the above tools) rather than a meta language.

With WebDSL, in addition to just building an application language, he also had all kinds of interesting ideas about web application development and how to improve it, such as:

• Reducing/eliminating boilerplate code. Originally WebDSL was implemented with JBoss and the Seam framework using Java as an implementation language, requiring you to write a lot of boilerplate code, such as getters/setters, deployment descriptors etc.
  
  WebDSL is declarative in the sense that you could more concisely describe what you want in a rich web application: a data model, and pages that should render content and data.
• Improving static consistency checking. Java (the implementation language used for the web applications) is statically typed, but not every concern of a web application can be statically checked. For example, for interacting with a database, embedded SQL queries (in strings) are often not checked. In JSF templates, page references are not checked.
  
  With WebDSL all these concerns are checked before the deployment of an web application.

By the time I joined, he already assembled several PhD and master's students to work on a variety of aspects of WebDSL and the underlying tooling, such as Stratego/XT.

Obviously, in a development process of a WebDSL application, like any application, you will also eventually face a deployment problem -- you need to perform activities to make the application available for use.

For solving deployment problems in our department, Nix was already quite intensively used. For example, we had a Nix-based continuous integration service called the Nix buildfarm (several years later, its implementation was re-branded into Hydra), that built all bleeding edge versions of WebDSL, Stratego/XT and all other relevant packages. The Nix package manager was used by all kinds of people in the department to install bleeding edge versions of these tools.

My research project was to automate the deployment of WebDSL applications using tooling from the Nix project. In my first few months, I have packaged all the infrastructure components that a WebDSL application requires in NixOS (JBoss, MySQL and later Apache Tomcat). I changed WebDSL to use GNU Autotools as build infrastructure (which was a common practice for all Stratego/XT related projects at that time) and made subtle modifications to prevent unnecessary recompilations of WebDSL applications (such as making the root folder dynamically configurable) and wrote an abstraction function to automatically build WAR files.

Thanks to Eelco I ended up in a really friendly and collaborative atmosphere. I came in touch with his fellow PhD and master's students and we frequently had very good discussions and collaborations.

Eelco was also quite helpful in the early stages of my research. For example, whenever I was stuck with a challenge he was always quite helpful in discussing the underlying problem and bringing me in touch with people that could help me.

My master's thesis project

After completing my initial version of the WebDSL deployment tool, that got me familiarised with the basics of Nix and NixOS, I started working on my master's thesis which was a collaboration project between Delft University of Technology and Philips Research.

Thanks to Eelco I came in contact with a former master's thesis student and postdoc of his: Merijn de Jonge who was employed by Philips Research. He was an early contributor to the Nix project and collaborated on the first two research papers about Nix.

While working on my master's thesis I developed the first prototype version of Disnix.

During my master's thesis project, Eelco Dolstra, who was formerly a postdoc at Utrecht University joined our research group in Delft. Eelco Visser made sure that I got all the help from Eelco Dolstra about all technical questions about Nix.

Becoming a PhD student

My master's thesis project was a pilot for a bigger research project. Eelco Visser, Eelco Dolstra and Merijn de Jonge (I was already working quite intensively with them for my master's thesis) were working on a research project proposal. When the proposal got accepted by NWO/Jacquard for funding, Eelco Visser was the first to inform me about the project to ask me what I thought about it.

At that moment, I was quite surprised to even consider doing a PhD. A year before, I attended somebody's else's PhD defence (someone who I really considered smart and talented) and thought that doing such a thing myself was way out of my grasp.

I also felt a bit like an impostor because I had interesting ideas about deployment, but I was still in the process of finishing up/proving some of my points.

Fortunately, thanks to Eelco my attitude completely changed in that year -- during my master's thesis project he convinced me that the work I was doing is relevant. What I also liked is the attitude in our group to actively build tools, have the time and space to explore things, and eat our own dogfood with it to solve relevant practical problems. Moreover, much of the work we did was also publicly available as free and open source software.

As a result, I easily grew accustomed to the research process and the group's atmosphere and it did not take long to make the decision to do a PhD.

My PhD

Although Eelco Visser only co-authored one of my published papers, he was heavily involved in many aspects of my PhD. There are way too many things to talk about, but there are some nice anecdotes that I really find worth sharing.

OOPSLA 2008

I still remember the first research conference that I attended: OOPSLA 2008. I had a very quick publication start, with a paper covering an important aspect of master's thesis: the upgrade aspect for distributed systems. I had to present my work at HotSWUp, an event co-located with OOPSLA 2008.

(As a sidenote: because we had to put all our efforts in making the deadline, I had to postpone the completion of my master's thesis a bit, so it started overlap with my PhD).

It was quite an interesting experience, because in addition to the fact that it was my first conference, it was also my first time to travel to the United States and to step into an airplane.

The trip was basically a group outing -- I was joined by Eelco and many of his PhD students. In addition to my HotSWUp 2008 paper, we also had an OOPSLA paper (about the Dryad compiler), a WebDSL poster, and another paper about the implementation of WebDSL (the paper titled: "When frameworks let you down") to present.

I was surprised to see how many people Eelco knew at the conference. He was also actively encouraging us to meet up with people and bringing us in touch with people that he know that could be relevant.

We were having a good time together, but I also remember him saying that it is actually much better to visit a conference alone, rather than in a group. Being alone makes it much easier and more encouraging to meet new people. That lesson stuck and in many future events, I took the advantage of being alone as an opportunity to meet up.

Working on practical things

Once in a while I had casual discussions with him about ongoing things in my daily work. For my second paper, I had to travel to ICSE 2009 in Vancouver, Canada all by myself (there were some colleagues traveling to co-located events, but took different flights).

Despite the fact that I was doing research on Nix-related things, NixOS at that time was not my main operating system yet on my laptop because it was missing features that I consider a must-have in a Linux distribution.

In the weeks before the planned travel date, I was intensively working on getting all the software packaged that I consider important. One major packaging effort was getting KDE 4.2 to work, because I was dissatisfied with only having the KDE 3.5 base package available in NixOS. VirtualBox was another package that I consider critical, so that I could still run a conventional Linux distribution and Microsoft Windows.

Nothing about this work is considered scientific "research" that may result in a paper that we can publish. Nonetheless, Eelco recognized the value of making NixOS more usable and encouraged me to get all that software packaged. He even asked me: "Are you sure that you have packaged enough software in NixOS so that you can survive that week?"

Starting my blog

Another particularly helpful advice that he gave me is that I should start a blog. Although I had a very good start of my PhD, having a paper accepted in my first month and another several months later, I slowly ran into numerous paper rejections, with reviews that were not helpful at all.

I talked to him about my frustrations and explained that software deployment research is generally a neglected subject. There is no research-related conference that is specifically about software deployment (there used to be a working conference on component deployment, but by the time I became a PhD student it was no longer organized), so we always had to "package" our ideas into subjects for different kinds of conferences.

He gave me the advice to start a blog to increase my interaction with the research community. As a matter of fact, many people in our research group, including Eelco, had their own blogs.

It took me some time to take that step. First, I had to "catch up" on my blog with relevant background materials. Eventually, it paid off -- I wrote a blog post titled: Software deployment complexity to emphasize software deployment as an important research subject, and thanks to Eelco's Twitter network I came in touch with all kinds of people.

Lifecycle management

For most of my publication work, I intensively worked with Eelco Dolstra. Eelco Visser left most of the practical supervision to him. The only published paper that we co-authored was: "Software Deployment in a Dynamic Cloud: From Device to Service Orientation in a Hospital Environment".

There was also a WebDSL-related subject that we intensively worked on for a while, that unfortunately never fully materialized.

Although I had already had the static aspects of a WebDSL application deployment automated -- the infrastructure components (Apache Tomcat, MySQL) as well as a function to compile a Java Web application Archive (WAR) with the WebDSL compiler, we also had to cope with the data that a WebDSL application stores -- WebDSL data models can evolve, and when this happens, the data needs to be migrated from an old to a new table structure.

Sander Vermolen, a colleague of mine, worked on a solution to make automated data migrations of WebDSL possible.

At some point, we came up with the idea to make this all work together -- deployment automation and data migration from a high-level point of view hiding unimportant implementation details. Due to the lack of a better name we called this solution: "lifecycle management".

Although the project seemed to look straight forward to me in the beginning, I (and probably all of us) heavily underestimated how complex it was to bring Nix's functional deployment properties to data management.

For example, Nix makes it possible to store multiple variants of the same packages (e.g. old and new versions) simultaneously on a machine without conflicts and makes it possible to cheaply switch between versions. Databases, on the other hand, make imperative modifications. We could manage multiple versions of a database by making snapshots, but doing this atomically and in an portable way is very expensive, in particular when databases are big.

Fortunately, the project was not a complete failure. I have managed to publish a paper about a sub set of the problem (automatic data migrations when databases move from one machine to another and a snapshotting plugin system), but the entire solution was never fully implemented.

During my PhD defence he asked me a couple of questions about this subject, from which (of course!) I understood that it was a bummer that we never fully realized the vision that we initially came up with.

Retrospectively, we should have divided the problem into a smaller chunks and solve each problem one by one, rather than working on the entire integration right from the start. The integrated solution would probably still consist of many trade-offs, but it still would have been interesting to have come up with at least a solution.

PhD thesis

When I was about to write my PhD thesis, I was making the bold decision to not compose the chapters directly out of papers, but to write a coherent story using my papers as ingredients, similar to Eelco Dolstra's thesis. Although there are plenty of reasons to think of to not do such a thing (e.g. it takes much more time for a reading committee to review such a thesis), he was actually quite supportive in doing that.

On the other hand, I was not completely surprised by it, considering the fact that his PhD thesis was several orders of magnitude bigger than mine (over 380 pages!).

Spoofax

After I completed my PhD, and made my transition to industry, he and his research group relentlessly kept working on the solution ecosystem that I just described.

Already during my PhD, many improvements and additions were developed that resulted in the Spoofax language workbench, an Eclipse plugin in which all these technologies come together to make the construction of Domain Specific Languages as convenient as possible. For a (somewhat :-) ) brief history of the Spoofax language workbench I recommend you to read this blog post written by him.

Moreover, he also kept dogfooding his own practical problems. During my PhD, three serious applications were created with WebDSL: researchr (a social network for researchers sharing publications), Yellowgrass (an issue tracker) and Weblab (a system to facilitate programming exams). These applications are still maintained and used by the university as of today.

A couple of months after my PhD defence in 2013 (I had to wait for several months to get feedback and a date for my defence), he was awarded the prestigious Vici grant and became a full professor, starting his own programming language research group.

In 2014, when I was already in industry for two years, I was invited for his inauguration ceremony and was given another demonstration of what Spoofax has become. I was really impressed by all the new meta languages that were developed and what Spoofax looked like. For example, SDF2 evolved into SDF3, a new meta-language for developing Name Address bindings (NaBL) was developed etc.

Moreover, I liked his inauguration speech very much, in which he briefly demonstrated the complexities of computers and programming, and what value domain specific languages can provide.

Concluding remarks

In this blog post, I have written down some of my memories working with Eelco Visser. I did this in the spirit of my blog, whose original purpose was to augment my research papers with practical information and other research aspects that you normally never read about.

I am grateful for the five years that we worked together, that he gave me the opportunity to do a PhD with him, for all the support, the things he learned me, and the people who he brought me in touch with. People that I still consider friends as of today.

My thoughts are with his family, friends, the research community and the entire programming languages group (students, PhD students, Postdocs, and other staff).

Dr. Sander

As a follow up story on the previous two blog posts, I can tell you that I have defended my PhD thesis last Monday, which went successfully. Now I can officially call myself a Doctor.

It was quite a tense, busy and interesting day. I'm not really used to days in which you are the centre of attention for most of the time. In the morning, our foreign guests: Sam Malek and Roberto di Cosmo (who are members of my committee) arrived to tell a bit about their ongoing research. We had some sort of a small software deployment workshop with quite some interesting ideas and discussions. Having them visiting us, gave me some renewed excitement about ongoing research and some interesting future ideas, since software deployment (as I have explained) is typically a misunderstood and under appreciated research subject.

After our small workshop, I had to quickly pick up our suits, and then return to the university. In the early afternoon, I had to give a laymen's talk to my family and friends in which I tried to explain the background and the goal of my PhD thesis. Then the actual defence started lasting for exactly one hour (no more, no less) in which the committee members asked me questions about my thesis and the accompanied propositions.

At Delft University of Technology and any other Dutch university, there is a lot of ceremonial stuff involved with the PhD defence. Professors have to be dressed up in gowns. Me and my paranimphs (the two people sitting in front of me who help me and take over my defence if I pass out) had to be dressed up in suits. I have to address to committee members formally depending on their roles, such as: "hooggeleerde opponent" and the committee members had to formally address me with "waarde promovendus".

I received some interesting questions during my defence round. To have an idea what these questions were, Vadim Zaytsev has made 140 character transcriptions of each question and answer on Twitter.

Although I was experiencing some nervousness at the beginning of the day, as I didn't know what exactly was going to happen and what kind of questions I would receive, in the end it was fun and I liked it very much. After the defence I received my PhD diploma:

For people that want to know what my thesis is exactly about:

• My PhD thesis can be obtained from my homepage.
• I have also written a blog post about my PhD thesis for people familiar with Nix that is significantly shorter.
• For people who think that even the blog post is too long, there is also 140 character version of my PhD thesis.

My PhD thesis propositions and some discussion

Apart from the contents of my PhD thesis, theses at our university are usually accompanied by a list of propositions. According to our university's Doctorate regulations, they must be defendable and opposable, at least six of the propositions are not supposed to be directly related to the research subject and two of them may be slightly playful. Besides the contents of my thesis, committee members are also allowed to ask questions about the propositions.

A colleague of mine: Felienne Hermans, has covered her propositions on her blog to elaborate about them. I have decided to do the same thing, although I'm not planning to create separate blog posts for each individual proposition. Instead, I cover all of them in a single blog post.

Propositions

1. Many of the non-functional requirements of a deployed service-oriented system can be realized by selecting an appropriate software deployment system.
   
   As I have explained earlier in a blog post about software deployment complexity, systems are rarely self-contained but composed of components. An important property of a component is that it can be deployed independently, significantly complicating a software deployment process.
   
   Components of service-oriented systems are called "services". It's a bit of an open debate to exactly tell what they are, since people from industry often think in terms of web services (things that use SOAP, WSDL, UDDI), while I have also seen the description "autonomous platform independent entities that can be loosely coupled" in the academic literature.
   
   Although web services are some sort of platform independent entities, they still have implementations behind their interfaces depending on certain technology and can be deployed to various machines in a network. We have seen that deployment on a single machine is hard and that deploying components into networks of machines is even more complicated, time consuming and error prone.
   
   Besides deployment activities, there are many important non-functional requirements a system has to meet. Many of them can be achieved by designing an architecture, e.g. components, connectors and the way they interact through architectural patterns/styles. Architectural patterns (e.g. layers, pipes and filters, blackboard etc.) implement certain quality attributes.
   
   For service-oriented systems, it's required to deploy components properly into a network of machines to be able to compose systems. In other words: we have to design and implement a proper deployment architecture. This has several technical challenges, such as the fact that we have to deploy components in such a way that they exactly match the deployment architecture, the deployment activities themselves, and the fact that we have to determine whether a system is capable of running a certain component (i.e. a service using Windows technology cannot run on a Linux machine or vice-versa).
   
   In addition to technical constraints, there are also many non-functional issues related to deployment that require attention, i.e. where to place components and how to combine them to achieve certain non-functional requirements? For example, privacy could be achieved by placing services providing access to privacy-sensitive data in a restricted zone and robustness by deploying multiple redundant instances of the same service.
   
   It can also be hard to manually find a deployment architecture that satisfies all non-functional requirements. In such cases, deployment planning algorithms are very helpful. In some cases it's even too hard or impossible to find an optimal solution.
   
   Because of all these cases, an automated deployment solution taking all relevant issues into account is very helpful in achieving many non-functional requirements of a deployed service-oriented system. This is what I have been trying to do in my PhD thesis.
   
   
2. The intention of the Filesystem Hierarchy Standard (FHS) is to provide portability among Linux systems. However, due to ambiguity, over-specification, and legacy support, this standard limits adoption and innovation. This phenomenon applies to several other software standards as well.
   
   There are many standards in the software engineering domain. In fact, it's dominated by them. One standard that is particularly important in my research is the Filesystem Hierarchy Standard (FHS) defining the overall filesystem structure of Linux systems. I have written a blog post on the FHS some time ago.
   
   In short: the FHS defines the purposes of directories, it makes a distinction between static and variable parts of a system, it defines hierarchies (e.g. / is for boot/recovery, /usr is for user software and for /usr/local nobody really knows (ambiguity)). Moreover, it also defines the contents of certain directories, e.g. /bin should contain /bin/sh (over-specification).
   
   I have problems with the latter two aspects -- the hierarchies do not provide support for isolation, allowing side-effects to easily manifest themselves while deploying and enabling destructive upgrades. I also have a minor problem with strict requirements of the contents of directories, as they easily allow builds to trigger side-effects while assuming that certain tools are always present.
   
   For all these reasons, we deviate on some aspects of the FHS in NixOS. Some people consider this unacceptable, and therefore they will not be able to incorporate most of our techniques to improve the quality of deployment processes.
   
   Moreover, as the FHS itself has issues, we observe that although the filesystem structure is standardized, file system layouts in many Linux distributions are still slightly different and portability issues still arise.
   
   In other domains, I have also observed various issues with standards:
   
   
   • Operating systems: nearly every operating system is more or less forced to implement POSIX and/or the Single UNIX Specification, taking a lot of effort. Furthermore, by implementing these standards they UNIX is basically reimplemented. These standards have many strict requirements on how certain library calls should be implemented, although it also specifies undefined behaviour at the same time. Apart from the fact that it's difficult and time consuming to implement these standards, there is little room to implement an operating system that is conceptually different from UNIX as it conflicts with portability.
   • The Web (e.g. HTML, CSS, DOM etc.): First, a draft is written in a natural language (which is inherently ambiguous) and sometimes underspecified. Then vendors start implementing these drafts. As initially these standards are ambiguous and underspecified, implementations behave very differently. Slowly these implementations converge into something that is uniform by collaborating with other vendors and the W3C to improve the draft version of a standard. Some vendors intentionally or accidentally implement conformance bugs, which don't get fixed for quite some time.
     
     These buggy implementations may become the de-facto standard, which has happened in the past, e.g. with Internet Explorer 6, requiring web developers to implement quirks code. Since the release of Internet Explorer 6 in 2001, Microsoft had 95% market share and did not release a new version until 2006. This was seriously hindering innovation in web technology. It also took many years before other implementations with better web standards conformance and more features gained acceptance.

So are standards bad? Not necessarily, but I think we have to critically evaluate them and not consider them as holy books. Moreover, standards need to be developed with some formality and elegance in mind. If junk gets standardized, it will remain junk and requires everybody to cope with junk for quite some time.

One of the things that may help is using good formalisms. For example, a good one I can think of is BNF that was used in the ALGOL 60 specification.


3. To move the software engineering community as a whole forward, industry and academia should collaborate. Unfortunately, their Key Performance Indicators (KPIs) drive them apart, resulting in a prisoner's dilemma.
   
   This proposition is related to an earlier rant blog post about software engineering fractions and collaborations. If I would use stereotypes, then the primary KPI of academia are the amount of (refereed) publications and the primary KPI of industry is how much they sell.
   
   It's obvious that, if both fractions would let themselves go a bit from their KPIs, i.e. academia does a bit more in engineering tools and transferring knowledge, while industry spends some of their effort in experimenting and paying attention to "secondary tasks", that both parties would benefit. However, in practice often the opposite happens (although there are exceptions of course).
   
   This is analogous to a prisoner's dilemma, which is a peculiar phenomenon. Visualize the following situation: two prisoners have jointly committed a crime and got busted. If both confess their crime then the amount of time they have to spend in prison are five years. If one prisoner confesses while the other does not, then the prisoner that confesses goes ten years into jail and the other remains free. If none of them confess, they both have to spent twenty years in prison.
   
   In this kind of situation the (obvious) win-win situation for both criminals is that they both confess. However, because they both give priority to their self-interests, none of them confesses as they assume that they remain free. But instead, the situation has the worst outcome: both have to spent twenty years in prison.
   
   
4. In software engineering, the use of social media, such as blogging and twitter, are an effective and efficient way to strengthen collaboration between industry and academia.
   
   This proposition is related to the previous one. How can we create a win-win situation? Often I hear people saying: "Well collaboration is interesting, but it costs time and money, which we don't have right now and we have other stuff to do".
   
   I don't think the barrier has to be that high. Social media, such as blogging and twitter, can be used for free and allows one to easily share stories, thoughts, results and so on. Moreover, recipients can also share these with people they know.
   
   My blog for example, has attracted many more readers and has given me much more feedback then all my research papers combined. Moreover, I'm not limited by all kinds of contraints that program committee members impose on me.
   
   However these observations are not unique to me. Many years ago, a famous Dutch computer scientist named Edsger Dijkstra wrote many manuscripts that he sent to his peers directly. He wrote about subjects that he found relevant. His peers spread these manuscripts through their colleagues allowing him to reach a wide range of people, eventually reaching thousands of people.
   
   
5. While the vision behind the definition of free software as described by the Free Software Foundation to promote freedom is compelling, the actual definition is ambiguous and inadequately promoted.
   
   The free software definition defines four freedoms. I can rephrase them in one sentence: "Software that can be used, studied, adapted and shared for any purpose". An essential precondition for this is the availability of the source code. I think this definition is clear and makes sense.
   
   However, there is a minor issue with the definition. The word 'free' is ambiguous in English. In the definition, it refers to free as in freedom not free in price (gratis). In Dutch or French it's not a problem. In these languages free software translates to 'vrije software' and 'libre software'.
   
   Moreover, although (almost) all free software is gratis, it's also allowed to sell free software for any price, which is often misunderstood.
   
   I have seen that the ambiguity of the word free is often used as an argument why the definition is not attracting a general audience.
   
   I think there is a bigger problem: the way free software is advertised. Most advertisements are not about what's good about free software, but about what's bad about proprietary software and how evil certain practices of certain companies are.
   
   Although I don't want to say that they are not right and we should tolerate such bad practices, I think it would also help to pay more attention to the good aspects of free software. The open source definition has much more care for this. For example, being able to improve quality of software. That's something I think that would attract people from the other side, whereas negative campaigning does not.
   
   
6. Compared to the definition of free software provided by the Free Software Foundation, the definition of Open Source as provided by the Open Source Initiative, fails to improve on freedom. While it has been more effectively promoted, it lacks a vision and does not solve ambiguity.
   
   The open source definition lists ten pragmatic points with the intention of having software that is free (as in freedom), e.g. availability of source code, means to share modified versions, and so on. However, it does not explain why it's desired for others to respect these ten pragmatic points and what their rationale is (although there is an annotated definition that does).
   
   Because of these reasons, I have seen that sometimes software is incorrectly advertised as being open-source, while in reality they are not. For example, there is also software available with source code, for which it is not allowed to do commercial redistributions, such as the LCC compiler. That's not open source (nor free software). Another prominent example is Microsoft's Shared Source initiative, only allowing someone to look at code, but not to modify or redistribute it.
   
   A very useful aspect of open source is the way it's advertised. It pays a lot of attention in selling its good points. For example, that everyone is able to improve its quality, and allowed to collaborate etc. Companies (even those that sell proprietary software) acknowledge these positive aspects and are sometimes willing to work with open-source people on certain aspects or to "open-source" pieces of their proprietary products. Examples of this are the Eclipse platform and the Quake 3 arena video game.
   
   
7. Just like making music is more than translating notes and rests into tones and pauses with specific durations, developing software systems is more than implementing functional requirements. In both domains, details, collaboration and listening to others are most important.
   
   I have observed that in both domains we make estimations. In software development, we try to estimate how much effort it takes to develop something and in music we try to estimate how much effort it takes to practice and master a composition.
   
   In software development, we often look at functional requirements (describing what a system should do) to estimate. I have seen that sometimes functional requirements may look ridiculously simple, such as displaying tabular data on a web page. Nearly every software developer would say: "that's easy".
   
   But even if functional requirements are simple, certain non-functional requirements (describing how and where) could make it very difficult. For example, properly implementing security facilities, a certain quality standard (such as ISO 9126) or to provide scalability. These kind of aspects may be much more complicated than the features of a system itself.
   
   Moreover, software is often developed in a team. Good communication and being able to divide work properly is important. In practice, you will almost always see that something goes wrong with that, because people have assumptions that all details are known by others or there is no clear architecture of a system so that work can be properly divided among team members.
   
   These are all kinds of reasons that may result in development times that are significantly longer than expected and failure to properly deliver what clients have asked.
   
   In my spare time I'm a musician and in music I have observed similar things. People make effort estimations by looking at the notes written on paper. Essentially, you could see those as functional requirements as they tell you what to play.
   
   However, besides playing notes and pausing, there are many more aspects that are important, such as tempo, dynamics (sudden and gradual loudness) and articulation. You could compare these aspects to non-functional requirements in software, as they tell somebody how to play (series of) notes.
   
   Moreover, making music can also be a group effort, such as a band or an orchestra, requiring people to properly interact with each other. If others make mistakes they may confuse you as well.
   
   I vividly remember a classical composition of 15 years ago. I just joined an orchestra and we were practising: "Land of the Long White Cloud" by Philip Sparke. In the middle of the composition, there is a snare drum passage consisting of only sixteenth notes. I already learned playing patterns like these in my first drum lesson, so I thought that it would be easy.
   
   However, I had to play these notes in a very fast tempo and very quietly, which are usually conflicting constraints for percussionists. Furthermore, I had to keep up the right tempo and don't let the other members distract me. Unfortunately, I couldn't cope with all these additional constraints, and that particular passage had to be performed by somebody else. I felt like an idiot and I was very disappointed. However, we did win the contest in which we had to perform that particular composition.
   
   As a side note: "The land of the long white cloud" is also known as Aotearoa or New Zealand.
   
   
8. Multilingualism is a good quality for a software engineer as it raises awareness that in natural languages as well as in software languages and techniques, there are things that cannot be translated literally.
   
   It's well known that some words or sentences cannot be literally translated from one natural language to another. In such cases, we have to reformulate a sentence into something that has an equivalent meaning, which is not always trivial.
   
   For example, non-native English speakers, like Dutch people, tend to make (subtle) mistakes now and then, which sometimes have hilarious outcomes. Make that the cat wise is a famous website that elaborates on this, calling the Dutch variant of English: Dunglish.
   
   Although we are aware of the fact that we cannot always translate things literally in natural languages, I have observed that in the software domain the same phenomenon occurs. One particular programming language may be more useful for a certain goal, than another programming language. Eventually, code written in a programming language gets compiled into machine code or another programming language (having an equivalent reformulation in a different language) or interpreted by an interpreter.
   
   However, I have also observed that in the software engineering domain there is a lot of programming language conservatism. Most conventional programming languages used nowadays (Python, C++, Java, C# etc.) use structured and imperative programming concepts in combination with class-based OO techniques. Unconventional languages such as purely functional programming languages (Haskell) or declarative languages (Prolog, Erlang) only get little mainstream acceptance, although they have very powerful features. For example, programs implemented in a purely functional language easily scale across multiple cores/processors.
   
   Instead, many developers use conventional languages to achieve the same, imposing many additional problems that need to be solved and more chances on errors. Our research based on the purely functional deployment model also suffers from conservatism. Therefore, I think multilingualism is very powerful asset for an engineer, as he is not limited by a solution set that is too narrow.
   
   
9. Stubbornness is both a positive as well as a negative trait of a researcher.
   
   I think that when you do research and if you discover something that is uncommon, others may reject it or tell you to do something that they consider more relevant. Some researchers choose to comply and give up stuff that they think is relevant. If every scientist would do that, then I think certain things would have never been discovered. I think it's a scientist's duty to properly defend important discoveries.
   
   In the middle ages it was even worse. For example, a famous scientist named Galileo Galilei revealed that not the Earth but the Sun was the centre of our solar system. He was sentenced to house arrest for the rest of his life by the catholic church.
   
   However, stubbornness also has a negative aspect. It often comes with ignorance and sometimes that's bad. For example, I have "ignored" some advice about properly studying related work and taking evaluation seriously, resulting in a paper that was badly rejected.
   
   

Conclusion

In this blog post, I have described some thoughts on my PhD propositions. The main reason of writing this down is to prepare myself for my defence. I know this blog post is lengthy, but that's good. This will probably prevent my committee members to read all the details, so that they cannot use everything I have just written against me :-) (I'm very curious to see if anyone has notice that I just said this :P).

My post-PhD career, a.k.a. leaving academia

It's been quiet here on my blog for a while now. Meanwhile, I'm more or less finished writing my PhD thesis. I still have to defend my thesis -- currently, it's being reviewed by my supervisors and my reading committee. As I have decided not to write a thesis in which (apart from the introduction and conclusion) the chapters are composed of my papers (as most people do in our research group), this will take a bit longer than usual.

Furthermore, my employment contract with the university has expired (in the Netherlands PhD students are typically employed a university). Since this month, I have started my new job at an industrial company. So essentially, I left academia.

On leaving academia

I think that some people that know me and haven't seen me in a while may be surprised (or perhaps shocked :-) ) by this announcement, so I think it would be a good idea to give an explanation why I have decided not to pursue a research career.

First, I'd like to stress out that the last four years were great, although there were a few periods that could have been better. I have no regrets becoming a PhD student (as opposed to working for an industrial company) -- I did many interesting things, I have been at many interesting places around the world, presented my research and software at a number of international software engineering conferences, learned interesting/useful stuff and I have met quite a number of interesting people. I'm really thankful for that to my supervisors, who gave me this opportunity.

Apart from research, one of the other things I'm really proud of is the practical impact of the work I'm involved with. For example, I gave a presentation at FOSDEM about NixOS -- a non-academic event -- which was quite successful. Apart from FOSDEM, I also gave a number of talks and demonstrations at industrial companies that were very well received. Finally, I'm also happy about the impact of my blog.

After listing all these positive traits, it almost sounds a bit odd that I'm not continuing the same path, which I often hear from outsiders. I have the following reasons for this:

1. My working style does not completely map to the job description of a researcher. Apart from doing the work to get things published, I have also invested a significant amount of time and effort in the tools and their practical usage.
   
   As I have also presented frequently to people outside the academic community, I typically use a lot of practical examples and I often have a number of demos showing good practical usage. These aspects are nice, but are (in principle) not part of my job description. Quite often, outsiders become happy after seeing these aspects and think that my research is about doing that kind of stuff, which is not true.
   
   
2. It's about publishing. As I have explained earlier in a number of blog posts, researchers have to publish and often it's used as the primary means to measure their productivity.
   
   I know some people that have left academia because they hate writing papers and the pressure that it sometimes gives. I don't really hate writing papers, although I've had my frustrating moments sometimes -- actually, if I have a good idea I'm very eager to write them down.
   
   What bothers me is that my research subject -- software deployment -- is not a very popular topic in the software engineering research world. If I would continue my career in research, I have to produce more publications, preferably at highly ranked conferences and journals.
   
   It's difficult to achieve this goal with my current research topic and working style. As a consequence, I'm more or less forced to pick subjects that are better publishable, which basically forces me to check the interests of the program committee members and do stuff that I'm not always interested in. I'm not somebody who likes to work that way.
   
   Some research topics are hyper-specialized, which is not always a good thing. I have noticed that even for my research (that already looks quite practical to most people in the research community) is difficult to sell conceptually to non-academic people, unless I do it the right way. For a number of presentations that I have seen at conferences, I'm pretty sure that people for industry would lose their interest in a matter of minutes and have no clue what the speaker is talking about.
   
   For many of these papers, the "real world" does not care about and as a consequence they tend to become forgotten knowledge. I'm not somebody who likes to do stuff that nobody cares about and do not bring any value to the software engineering community as a whole.
   
   Furthermore, if I would spend my time to publish more, there is less time to do engineering work and to produce/improve tools that can actually do something.
   
   I've picked my current research project, because I like solving problems that I really believe that they are relevant and I want to add value. However, the "deliverables" of my style unfortunately don't always map to publishable units at academic events.
   
   
3. I want to add more practical value. I have always been a person that likes to build stuff. If I have a nice idea in mind, I'm always very eager to try it out and to make piece of software that does something, although it is not always very relevant. For the last 6 months, I had no time for doing stuff like this. Nearly all my time was consumed by finishing publications, writing my thesis and preparing/visiting a conference.
   
   

Looking for a new job

As my time was completely consumed by finishing publications and writing my thesis the last 6 months, I didn't really have the time to think about a new career perspective. At the end of August, I have decided to put my CV online and I have used the entire month of September to visit and talk to companies.

I was approached by many companies and I have visited all of them in a circle of Rotterdam, Amsterdam, Eindhoven and Utrecht. So I have traveled quite a lot :-).

In the beginning, I felt a bit worried, because it's been a long time ago that I officially applied for a job and I knew that there are a number of prejudices that industry people have on academics like me. This a bit funny, because when I was still a bachelor's student I was not worried about application procedures or getting a job at all :-)

First it looked like my view on industry turned out to be true -- some potential employers did not care about my research career, just asked me: "can you program in Java?" and offered me a salary, which was way lower that my current salary. Some of them also did not know what a PhD is and treated me the same as an undergrad student.

Furthermore, at one company I had to perform a number of capacity tests to check whether I was smart enough. I had to do tests, such as verbal analogies, number sequences, and abstract reasoning. Fortunately, I did the test well enough. There were a few applicants that immediately had to leave because their score was too low and the company was not even interested talking to them. I have made it to the last round and they gave me a job offer, but I refused it. :-)

Fortunately, I did not unlearn my application skills (nearly all of them went quite successfully) and not all companies that I talked to were disappointing -- in the last 2 weeks of September I had "collected" 4 interesting job offers, with good salaries and interesting job aspects. I have met a number of interesting people at these companies and it was very hard for me to turn 3 of them down. A few of them were very disappointed that I rejected them. :-)

In total, I received 6 job offers, one application turned out to be a mismatch of interests from both sides, and 1 rejection after the second round. Apparently, that company was looking for somebody that just wants to write code and not somebody who's critical like me :-).

My new job: Software Architect @ Conference Compass

Currently, I have joined Conference Compass, a Delft-based startup company developing applications (mostly Apps for mobile devices) to improve the experience of conference participants.

I will fulfill the role of Software Architect -- I'm involved with many technical aspects, ranging from design, implementation, documentation and (interestingly enough) deployment.

The main reason of joining this company is that the company is small, there is a lot of interesting challenges that I can work on, a lot of room in things that I can do/improve, and the atmosphere -- 3 employees are PhDs (once I graduate they have 4 of them) and are -- apart from technical aspects -- very much interested in applying new concepts.

Even more interesting is that there is room in which I can use my experience from my research background and that they are very much interested in participation in free and open-source software projects.

Another interesting aspect is that they are located in the YES!Delft incubation centre, which is supported by various external parties including Delft University of Technology. In fact, I'm not that far away from my previous working spot (only 100 meters or so). I can see my former employer's building from the window here. :-)

In my past career, I have worked for medium sized companies as well as big ones. Joining a small group is a new experience to me.

The future of Disnix and other components of the Nix project

After reading this, you may possibly wonder what the future is of the software that I wrote during my PhD career, in particular Disnix and other components part of the Nix project.

As I may have already partially revealed on this blog, my new employer is also interested in the deployment technologies of my previous job. One of their biggest interests is having this technology supported in the mobile space and in the cloud.

Furthermore, my former employer is still using our Nix-based Hydra buildfarm and my two former colleagues (who are now also employed elsewhere) still maintain it and use the Nix technology at their company as well.

So in short: The Nix project is far from dead and is still actively being used. Now that we're not required to publish papers anymore, we even have more time to improve the tools and to make it even better applicable.

Also, my homepage and this blog will remain to exist. It will still be tech-related, cover deployment and other technical aspects that I may run into in the near future. My new job is not only about software deployment, so I will also be working on other aspects.

The future of research in software deployment

As the Mancoosi project has come to an end and because me and my former colleagues working on Nix (including the Nix author) have left the university, deployment research is pretty much dead now, which is a shame, because I think it's very important in practice.

However, I have agreed with my supervisors (and still have the intention!) to publish one journal paper that contains some unpublished stuff from my PhD thesis, in my spare time. Maybe (you'll never know) I have more stuff published when I can find the time and space for it, but I can't give you any time window. :-)

It's not that since I have switched to industry that I'm not interested in research anymore and that I don't want to read or write a paper. I'd still like to be involved in a healthy/doable way, but it's no longer my primary task.

In fact, if academia and industry invest some effort, it can result in fruitful collaborations that give both parties benefits. I wrote a very large blog post about this some time ago, titled 'Software engineering fractions, collaborations and "The System"' that I'd like to encourage both academic and industry people to read.

So to all my academic friends: I'd like to say that if you have a new idea, let me know and make sure that you show that it's relevant to read or even better -- something that I can actually use/try.

Concluding remarks

In this blog post, I have announced that I have moved from academia to industry. The title contains "leaving academia" which sounds very dramatic and a bit like a farewell message. Although I'm no longer employed by the university, I'm actually still located quite close to it, and I have visited my former colleagues twice this week. Furthermore, the technology that we have "invented" is still actively being used and maintained.

I also have the intention to retain the relationship with people from the research world, although I still have to see how I can properly integrate that into my new job description.

Finally, I still have to defend my PhD thesis. Once I have more details about this, I'll announce it here and at several other places of course :-).

A review of conferences in 2011-2012

It's time for the final episode in the "visited conference series". Fortunately, 2011 and 2012 were not as busy as 2010, so I can put the remaining ones in a single blog post.

FOSDEM: Free and Open Source Software Developers' European Meeting (2011)

2011 started with FOSDEM as with 2010. As with all the previous editions, the FOSDEM experience is always the same as usual - the same peculiar building etc. etc. (see my previous blog posts).

In this edition, I have also given a talk about NixOS titled: "Using NixOS for declarative deployment and testing". I didn't really know what to expect, but it turned out that it was received quite well. After my talk, I saw some #fosdem tweets on Twitter, in which one of the attendants described my talk as one of the most exciting ones next to systemd and the Google Go language (personally, I also find these two other talks quite good).

SEAMS: Software Engineering for Adaptive and Self-Managing Systems (2011)

The next event was another academic one: SEAMS, which is about self-adaptive and self-managing systems. SEAMS was co-located with ICSE and a number of other events, such as MSR, and held in Honolulu, Hawaii.

SEAMS looked like an interesting event to me, but I had some difficulties finding attendants that were looking into the same issues I did. Most of the presentations were about embedded software and covered issues I did not know about or understand. Although service-oriented systems were listed on their call for papers, I was the only one with a paper in that domain.

I missed some interaction between the participants in general. Perhaps, it's due to the fact that SEAMS was originally a workshop and upgraded to a symposium. Furthermore, this edition had a record number of participants and submissions. Also, the panel discussion was not very interesting and apart from the panel members, nobody took part.

Another disappointing thing is that after the panel discussion a number of attendants left and I had to present my paper in the final slot, with only half of the people left.

The social event was at a Chinese restaurant in Honolulu and was much better. Here, I finally had the opportunity to meet people and to have discussions about research. One attendant suggested a partial solution to me, I was struggling with for a while.

In general, SEAMS was not a bad event although several aspects could be improved and the domain does not entirely map to the kind of research that I'm doing.

ICSE: International Conference on Software Engineering (2011)

Apart from SEAMS, I also attended ICSE, which was held at the same place (Honolulu, Hawaii).

The picture above shows how awesome the environment is. This picture has nothing to do with the conference of course, but I like to show it to you anyway :-)

The conference experience was good and actually a bit better than the 2009's edition, that I have attended previously. A colleague as well as a former colleague had to present a paper there about spreadsheets and testing cross browser compatibility. Apart from my colleague's presentations, there was another good presentation titled: "An Empirical Study of Build Maintenance Effort", which contains a collection of empirical results relevant to our research.

The social event was a dinner, which included an amazing show with lots of typical Hawaiian cultural aspects.

FOSDEM: Free and Open Source Software Developers' European Meeting (2012)

I started 2012 with FOSDEM, as with the previous two years. This year was a bit special compared to the others, as I had a lot of difficulties getting there due to the weather conditions. In 2012, there was a lot of snowfall. As a result, I arrived in Brussels somewhere in the afternoon, although it gave me some pretty pictures, such as the one above, which I took from a small train station en route, because we had to wait until a defect was fixed.

You probably expect me to say that the FOSDEM experience is same as usual. Although it was held at the ULB campus like all the other years, several more buildings were used for the talks. These buildings also had a much better quality compared to the regular ones. As a result, the hallways were less crowded (which is a good thing). Normally, I was always struggling to get from one devroom to another.

The picture above shows a room in a newer building in which the Wayland talk was held.

HotSWUp: Workshop on Hot Topics in Software Upgrades (2012)

In 2012, I attended HotSWUp for the second time. HotSWUp was the first academic event that I have attended four years earlier. This time it was co-located with ICSE and held at the Irchel Campus of the University of Zürich, Switzerland.

I haven't attended any academic events for a while, and a day before the workshop I didn't expect it to be that exciting. The first HotSWUp was already great, and this one was even a bit better.

There were many interesting presentations, covering upgrading aspects in various domains ranging from programming language level, to component level and system level. As with the first HotSWUp, I liked the cross disciplinary aspect very much.

I was quite satisfied with my own presentation, which raised a number of interesting questions, such as how to version components. Moreover, I received an interesting suggestion that I could use to optimise the approach described in my paper.

The social event was a dinner in a vegetarian restaurant. Normally, I wouldn't pick such a location, but it was interesting anyway. The fake meat tasted like real meat. After the dinner, me and several other participants had a couple of beers somewhere else.

ICSE: International Conference on Software Engineering (2012)

In addition to HotSWup, I also attended ICSE - the host conference held in the Kongresshaus Zürich. It is the third time for me attending ICSE.

This years ICSE was nice, had some interesting talks (two of them were of my colleagues, one about code smells in spreadsheets and the other about test confessions of developers), but I'm a bit critical about this years' edition compared to my previous experiences.

The quality of the first two keynote presentations were a bit disappointing and I really had to force myself listening, which (of course) failed after a few minutes. The third keynote was much better, although the speaker could have included more recent examples of research in software architectures.

Another thing was the conference venue itself. The rooms for the research talks were much too small, overly crowded and without any air conditioning (so the temperatures were really high). As a result, I sometimes had difficulties listening to a talk and it was almost impossible to ask questions.

Furthermore, it was difficult to talk to new people on the hallways, also partially due to the weird structure of the conference venue. Fortunately, I did not limit me entirely to meet new people, but it could have been better.

We had two social events at ICSE. The last one was a dinner, which included a show with two singers and a keyboard player, as shown above.

Although I'm critical about this years' ICSE, it was not a bad conference, but some things could have been better.

Conclusions

In this blog posts I have described the events I have attended in 2011 and 2012, which were less busy as 2010. The only new event I have attended was SEAMS.

These were all the conferences I have visited in my period as as a PhD student. In a next blog post, I'll try to draw some general conclusions from these blog posts.