As a consequence of using callbacks, it has become much harder to structure code properly. Without any proper software abstractions, we may end up writing pyramid code that is hard to write, read, adapt and maintain. One of the solutions to alleviate such problems is by using software abstractions implemented by libraries such as async.
Recently, I've been experimenting with several JavaScript libraries/frameworks (such as AngularJS, Selenium JavaScript webdriver, and jQuery) and I went to a Titanium SDK (a cross-platform JavaScript framework for developing apps) developer meetup. I have noticed that yet another software abstraction has become quite popular in these worlds.
Promises
The software abstraction I am referring to is called "promises". To get some understanding on what they are supposed to mean, I will cite the Promises/A proposal, that is generally regarded as its official specification:
A promise represents the eventual value returned from the single completion of an operation.
To me this description looks quite abstract. Fortunately, the proposal also outlines a few practical bits. It turns out that promises are supposed to be encoded as objects having the following characteristics:
- A promise can be in one of the following three states: unfulfilled, fulfilled, and failed. A promise in an unfulfilled state can either transition to a fullfilled state or failed state. The opposite transitions are not possible -- once a promise is fulfilled or failed it should remain in that state.
- A promise object has a then member referring to a function with the following signature:
.then(onFullfilled, onError, onProgress);
-
The then function parameters are all optional and refer to functions with the following purposes:
- onFullfilled: A function that gets invoked once the promise reaches its fulfilled state.
- onError: A function that gets invoked once the promise reaches its failed state.
- onProgress: A function that may get invoked while the promise is still unfulfilled. Promises are actually not required to support this function at all.
If any of these parameters is not a function, then they are ignored.
- Each then function is supposed to return another promise (which also have a then property), allowing users to "chain" multiple then function invocations together. This is how promises allow someone to structure code more properly and prevent people from writing pyramid code, because chaining can be done at the same indentation level.
Semantic issues
Although some of the promise characteristics listed above may look useful, it has turned out that the Promises/A proposal has a number of weaknesses as well.
First of all, it is just a proposal containing some random notes and not an official specification. Furthermore, it has been written in an informal way and it is incomplete and underspecified. For example, it leaves a few semantic issues open, such as what should happen if a promise enters a failed state, when a particular then function invocation in a chain does not implement an onError callback? As a consequence, this has lead to several implementations having different and incompatible behaviour.
To address some of these issues, a second specification has been developed called the Promises/A+ specification. This specification is not intended to completely replace the Promises/A proposal, but instead it is concentrated mostly on the semantics of the then function (not including the onProgress parameter). Moreover, this specification is also written in a much more formal and concise way in my opinion.
For example, the Promises/A+ specification adds the following details:
- It specifies the ability to invoke a promise's then function multiple times.
- Propagation of results and errors in a then function invocation chain, if any of their callback function parameters are not set.
When using Promises/A+ compliant promise objects, error propagation works in a similar way compared to a synchronous try { } catch { } block allowing one to execute a group of instructions and use a separate block at the end to catch any exceptions that may be thrown while executing these.
An example: Using the Selenium webdriver
To demonstrate how promises can be used, I have developed an example case with the JavaScript Selenium webdriver, a web browser automation framework used frequently for testing web applications. This package uses promises to encode query operations.
Our testcase works on a webpage that contains a form allowing someone to enter its first and last name. By default, they contain my first and last name:
The corresponding HTML code looks as follows:
<!DOCTYPE html>
<html>
<head><title>Selenium test</title></head>
<body>
<h1>Selenium test</h1>
<p>Please enter your names:</p>
<form action="test.html" method="post">
<p>
<label>First name</label>
<input type="text" name="firstname" value="Sander">
</p>
<p>
<label>Last name</label>
<input type="text" name="lastname" value="van der Burg">
</p>
<button type="submit" id="submit">Submit</button>
</form>
</body>
</html>
To automatically retrieve the values of both input fields and to click on the submit button, I have written the following JavaScript code:
var webdriver = require('selenium-webdriver');
var remote = require('selenium-webdriver/remote');
var webdrvr = require('webdrvr');
/* Create Selenium server */
var server = new remote.SeleniumServer(webdrvr.selenium.path, {
args: webdrvr.args
});
/* Start the server */
server.start();
/* Set up a Chrome driver */
var driver = new webdriver.Builder()
.usingServer(server.address())
.withCapabilities(webdriver.Capabilities.chrome())
.build();
/* Execute some tests */
driver.get('file://'+process.cwd()+'/test.html')
.then(function() {
return driver.findElement(webdriver.By.name('firstname'));
})
.then(function(firstNameInput) {
return firstNameInput.getAttribute('value');
})
.then(function(firstName) {
console.log("First name: "+firstName);
return driver.findElement(webdriver.By.name('lastname'));
})
.then(function(lastNameInput) {
return lastNameInput.getAttribute('value');
})
.then(function(lastName) {
console.log("Last name: "+lastName);
return driver.findElement(webdriver.By.id('submit'));
})
.then(function(submitButton) {
return submitButton.click();
}, function(err) {
console.log("Some error occured: "+err);
});
/* Stop the server */
server.stop();
As may be observed from the code fragment above, besides setting up a test driver that uses Google Chrome, all the test operations are encoded as a chain of then function invocations each returning a promise:
- First, we instruct the test driver to get our example HTML page containing the form by creating a promise.
- After the page has been opened, we create and return a promise that queries the HTML input element that is used for filling in a first name.
- Then we retrieve the value of the first name input field, by creating and returning a promise that fetches the value attribute of the HTML input element.
- We repeat the previous two steps in a similar way to fetch the last name as well.
- Finally, we create and return promises that queries the submit button and clicks on it.
- In the last then function invocation, we define a onError callback that catches any errors that might occur when executing any of the previous steps.
As can be observed, we do not have to write any nested callback blocks that result in horrible pyramid code.
Implementing promises
The Selenium example shows us how asynchronous test code can be better structured using promises. The next thing I was thinking about is how can I use promises to fix structuring issues in my own code?
It turns out that there are many ways to do this. In fact, the Promise/A and Promise/A+ specifications are just merely interface specifications and leave it up to developers to properly implement them. For example, it can be done completely manually, but there are also several libraries available providing abstractions for that, such as Q, RSVP, and when.
As an experiment, I have restructured the following pyramid example code fragment from my previous blog post (that I originally used to restructure with async) into a better structured code fragment with promises:
var fs = require('fs');
var path = require('path');
fs.mkdir("out", 0755, function(err) {
if(err) throw err;
fs.mkdir(path.join("out, "test"), 0755, function(err) {
if (err) throw err;
var filename = path.join("out", "test", "hello.txt");
fs.writeFile(filename, "Hello world!", function(err) {
if(err) throw err;
fs.readFile(filename, function(err, data) {
if(err) throw err;
if(data == "Hello world!")
process.stderr.write("File is correct!\n");
else
process.stderr.write("File is incorrect!\n");
});
});
});
});
I have picked the RSVP library to implement promises, since it claims to be lightweight and fully Promises/A+ compliant. The restructured code looks as follows:
var fs = require('fs');
var path = require('path');
var Promise = require('rsvp').Promise;
/* Promise object definitions */
var mkdir = function(dirname) {
return new Promise(function(resolve, reject) {
fs.mkdir(dirname, 0755, function(err) {
if(err) reject(err);
else resolve();
});
});
};
var writeHelloTxt = function(filename) {
return new Promise(function(resolve, reject) {
fs.writeFile(filename, "Hello world!", function(err) {
if(err) reject(err);
else resolve();
});
});
};
var readHelloTxt = function(filename) {
return new Promise(function(resolve, reject) {
fs.readFile(filename, function(err, data) {
if(err) reject(err);
else resolve(data);
});
});
};
/* Promise execution chain */
var filename = path.join("out", "test", "hello.txt");
mkdir(path.join("out"))
.then(function() {
return mkdir(path.join("out", "test"));
})
.then(function() {
return writeHelloTxt(filename);
})
.then(function() {
return readHelloTxt(filename);
})
.then(function(data) {
if(data == "Hello world!")
process.stderr.write("File is correct!\n");
else
process.stderr.write("File is incorrect!\n");
}, function(err) {
console.log("An error occured: "+err);
});
The revised code fragment above is structured as follows:
- The overall structure consists of two parts: First we define a number of promises and then we execute them by composing a then function invocation chain.
- I have encapsulated most of the operations from the previous code fragment into reusable promises: one makes a directory, one writes a text file, one reads the text file.
- A promise is created by instantiating the rsvp.Promise prototype. The constructor function always takes the same function as a parameter taking resolve and reject as parameters. resolve is a callback that can be used to notify a caller that it has reached its fulfilled state. The reject parameter notifies a caller that it has reached its failed state. Both of these functions can take a value as parameter that can be passed to a then function invocation.
- To be able to parametrize these promises, I have encapsulated them in functions
- In the remainder of the code, I instantiate the promises and execute them in a then function invocation chain. As can be observed, we have no pyramid code.
- However, I observed that our refactored code has a big drawback as well -- it has grown significantly in size which can be considered a deterioration compared to the pyramid code fragment.
Discussion
So far, I have given an explanation of what promises are supposed to mean, how they can be used and how they can be created. We have seen that code structuring issues are "solved" by the fact that then function invocations can be chained.
In my previous blog post, I have explored the async library as a means to solve the same kind of issues, which make me wonder which approach is best? To me, the approach of chaining then function invocations looks quite similar to async's waterfall pattern. However, I find the promise approach slightly better, because it uses return values instead of callbacks preventing us to accidentally reach limbo state if we forget to call them.
However, I think async's waterfall pattern has a lower usage barrier because we do not have to capture all the steps we want into promise objects resulting in less overhead of code.
I think that using promises to structure code works best for things that can be encoded as reusable objects, such as those used to implement a system having a Model-View-Controller (MVC) pattern, that e.g. pass reusable objects from the model to the controller and view. In situations in which we just have to execute a collection of ad-hoc operations (such as those in my example) it simply produces a lot of extra overhead.
Finally, async's waterfall pattern as well as chaining then function invocations work well for things that consist of a fixed number of steps that have to be executed sequentially. However, it may also be desirable to execute steps in parallel, or to use asynchronous for and while loops. The promises specifications do not provide any solutions to cope with these kind of code structuring issues.
Some other thoughts
After doing these experiments and writing this blog post, I still have some remaining thoughts:
- I have noticed that promises are considered the "the next great paradigm" by some people. This actually makes me wonder what the "previous great paradigm" was. Moreover, I think promises are only useful in certain cases, but definitely not for all JavaScript's asynchronous programming problems.
- Another interesting observation I did is not related to JavaScript or asynchronous programming in general. I have observed that writing specifications, which purpose is to specify what kind of requirements a solution should fulfill, is a skill that is not present with quite a few people doing software engineering. It's not really surprising to me that the Promise/A proposal leads to so much issues. Fortunately, the Promises/A+ authors demonstrate how to properly write a specification (plus that they are aware of the fact that specifications always have limitations and issues).
- I'm still surprised how many solutions are being developed for a problem that is non-existent in other programming environments. Actually, I consider these software abstractions dealing with asynchronous programming "workarounds", not solutions. Because none of them is ideal there are many of them (the joyent Node wiki currently lists 79 libraries and I know there are more of them), I expect many more to come.
- Personally, I consider the best solution a runtime environment and/or programming language that has facilities to cope with concurrency. It is really surprising to me that nobody is working on getting that done. UPDATE: There is a bit of light at the end of the tunnel according to Zef's blog titled: "Callback-Free Harmonious Node.js" post that I just read.
No comments:
Post a Comment