In the first blog post, I explained some general asynchronous programming issues, code structuring issues and briefly demonstrated how the async library can be used to structure code more properly. Later, I have written a blog post about promises, another abstraction mechanism dealing with asynchronous programming complexities. Finally, I have developed my own abstraction functions by investigating how JavaScript's structured programming language constructs (that are synchronous) translate to the asynchronous programming world.
In these blog posts, I have used two kinds of function invocation styles -- something that I call the Node.js-function invocation style, and the promises invocation style. As the name implies, the former is used by the Node.js standard library, as well as many Node.js-based APIs. The latter is getting more common in the browser world. As a matter of fact, many modern browsers, provide a Promise prototype as part of their DOM API allowing others to construct their own Promise-based APIs with it.
In this blog post, I will compare both function invocation styles and describe some of their differences. Additionally, there are situations in which I have to mix APIs using both styles and I have observed that it is quite annoying to combine them. I will show how to alleviate this pain a bit by developing my own generically applicable adapter functions.
Two example invocations
The most frequently used invocation style in my blog posts is something that I call the Node.js-function invocation style. An example code fragment that uses such an invocation is the following:
fs.readFile("hello.txt", function(err, data) {
if(err) {
console.log("Error while opening file: "+err);
} else {
console.log("File contents is: "+data);
}
});
As you may see in the code fragment above, when we invoke the readFile() function, it returns immediately (to be precise: it returns, but it returns no value). We use a callback function (that is typically the last function parameter) to retrieve the results of the invocation (or the error if something went wrong) at a later point in time.
By convention, the first parameter of the callback is an error parameter that is not null if some error occurs. The remaining parameters are optional and can be used to retrieve the corresponding results.
When using promises (more specifically: promises that conform to the Promises/A and Promises/A+ specifications), we use a different invocation pattern that may look as follows:
Task.findAll().then(function(tasks) {
for(var i = 0; i < tasks.length; i++) {
var task = tasks[i];
console.log(task.title + ": "+ task.description);
}
}, function(err) {
console.log("An error occured: "+err);
});
As with the previous example, the findAll() function invocation shown above also returns immediately. However, it also does something different compared to the Node.js-style function invocation -- it returns an object called a promise whereas the invocation in the previous example never returns anything.
By convention, the resulting promise object provides a method called then() in which (according the Promises/A and A+ standards) the first parameter is a callback that gets invoked when the function invocation succeeds and the second callback gets invoked when the function invocation fails. The parameters of the callback functions represent result objects or error objects.
Comparing the invocation styles
At first sight, you may probably notice that despite having different styles, both function invocations return immediately and need an "artificial facility" to retrieve the corresponding results (or errors) at a later point in time, as opposed to directly returning a result in a function.
The major difference is that in the promises invocation style, you will always get a promise as a result of an invocation. A promise provides a reference to something which corresponding result will be delivered in the future. For example, when running:
var tasks = Task.findAll();
I will obtain a promise that, at some point in the future, provides me an array of tasks. I can use this reference to do other things by passing the promise around (for example) as a function argument to other functions.
For example, I may want to construct a UI displaying the list of tasks. I can already construct pieces of it without waiting for the full list of tasks to be retrieved:
displayTasks(tasks);
The above function could, for example, already start rendering a header, some table cells and buttons without the results being available yet. The display function invokes the then() function when it really needs the data.
By contrast, in the Node.js-callback style, I have no reference to the pending invocation at all. This means that I always have to wait for its completion before I can render anything UI related. Because we are forced to wait for its completion, it will probably make the application quite unresponsive, in particular when we have to retrieve many task records.
So in general, in addition to better structured code, promises support composability whereas Node.js-style callbacks do not. Because of this reason, I consider promises to be more powerful.
However, there is also something that I consider a disadvantage. In my first blog post, I have shown the following Node.js-function invocation style pyramid code example as a result of nesting callbacks:
var fs = require('fs');
var path = require('path');
fs.mkdir("out", 0755, function(err) {
if(err) throw err;
fs.mkdir(path.join("out, "test"), 0755, function(err) {
if (err) throw err;
var filename = path.join("out", "test", "hello.txt");
fs.writeFile(filename, "Hello world!", function(err) {
if(err) throw err;
fs.readFile(filename, function(err, data) {
if(err) throw err;
if(data == "Hello world!")
process.stderr.write("File is correct!\n");
else
process.stderr.write("File is incorrect!\n");
});
});
});
});
I have also shown in the same blog post, that I can use the async.waterfall() abstraction to flatten its structure:
var fs = require('fs');
var path = require('path');
filename = path.join("out", "test", "hello.txt");
async.waterfall([
function(callback) {
fs.mkdir("out", 0755, callback);
},
function(callback) {
fs.mkdir(path.join("out, "test"), 0755, callback);
},
function(callback) {
fs.writeFile(filename, "Hello world!", callback);
},
function(callback) {
fs.readFile(filename, callback);
},
function(data, callback) {
if(data == "Hello world!")
process.stderr.write("File is correct!\n");
else
process.stderr.write("File is incorrect!\n");
}
], function(err, result) {
if(err) throw err;
});
As you may probably notice, the code fragment above is much more readable and better maintainable.In my second blog post, I implemented a promises-based variant of the same example:
var fs = require('fs');
var path = require('path');
var Promise = require('rsvp').Promise;
/* Promise object definitions */
var mkdir = function(dirname) {
return new Promise(function(resolve, reject) {
fs.mkdir(dirname, 0755, function(err) {
if(err) reject(err);
else resolve();
});
});
};
var writeHelloTxt = function(filename) {
return new Promise(function(resolve, reject) {
fs.writeFile(filename, "Hello world!", function(err) {
if(err) reject(err);
else resolve();
});
});
};
var readHelloTxt = function(filename) {
return new Promise(function(resolve, reject) {
fs.readFile(filename, function(err, data) {
if(err) reject(err);
else resolve(data);
});
});
};
/* Promise execution chain */
var filename = path.join("out", "test", "hello.txt");
mkdir(path.join("out"))
.then(function() {
return mkdir(path.join("out", "test"));
})
.then(function() {
return writeHelloTxt(filename);
})
.then(function() {
return readHelloTxt(filename);
})
.then(function(data) {
if(data == "Hello world!")
process.stderr.write("File is correct!\n");
else
process.stderr.write("File is incorrect!\n");
}, function(err) {
console.log("An error occured: "+err);
});
As you may notice, because the then() function invocations can be chained, we also have a flat structure making the code better maintainable. However, the code fragment is also considerably longer than the async library variant and the unstructured variant -- for each asynchronous function invocation, we must construct a promise object, adding quite a bit of overhead to the code.
From my perspective, if you need to do many ad-hoc steps (and not having to compose complex things), callbacks are probably more convenient. For reusable operations, promises are typically a nicer solution.
Mixing function invocations from both styles
It may happen that function invocations from both styles need to be mixed. Typically mixing is imposed by third-party APIs -- for example, when developing a Node.js web application we may want to use express.js (callback based) for implementing a web application interface in combination with sequelize (promises based) for accessing a relational database.
Of course, you could write a function constructing promises that internally only use Node.js-style invocations or the opposite. But if you have to regularly intermix calls, you may end up writing a lot of boilerplate code. For example, if I would use the async.waterfall() abstraction in combination with promise-style function invocations, I may end up writing:
async.waterfall([
function(callback) {
Task.sync().then(function() {
callback();
}, function(err) {
callback(err);
});
},
function(callback) {
Task.create({
title: "Get some coffee",
description: "Get some coffee ASAP"
}).then(function() {
callback();
}, function(err) {
callback(err);
});
},
function(callback) {
Task.create({
title: "Drink coffee",
description: "Because I need caffeine"
}).then(function() {
callback();
}, function(err) {
callback(err);
});
},
function(callback) {
Task.findAll().then(function(tasks) {
callback(null, tasks);
}, function(err) {
callback(err);
});
},
function(tasks, callback) {
for(var i = 0; i < tasks.length; i++) {
var task = tasks[i];
console.log(task.title + ": "+ task.description);
}
}
], function(err) {
if(err) {
console.log("An error occurred: "+err);
process.exit(1);
} else {
process.exit(0);
}
});
For each Promise-based function invocation, I need to invoke the then() function and in the corresponding callbacks, I must invoke the callback of each function block to propagate the results or the error. This makes the amount of code I have to write unnecessary long, tedious to write and a pain to maintain.
Fortunately, I can create a function that abstracts over this pattern:
function chainCallback(promise, callback) {
promise.then(function() {
var args = Array.prototype.slice.call(arguments, 0);
args.unshift(null);
callback.apply(null, args);
}, function() {
var args = Array.prototype.slice.call(arguments, 0);
if(args.length == 0) {
callback("Promise error");
} else if(args.length == 1) {
callback(args[0]);
} else {
callback(args);
}
});
}
The above code fragment does the following:
- We define a function takes a promise and a Node.js-style callback function as parameters and invokes the then() method of the promise.
- When the promise has been fulfilled, it sets the error parameter of the callback to null (to indicate that there is no error) and propagates all resulting objects as remaining parameters to the callback.
- When the promise has been rejected, we propagate the resulting error object. Because the Node.js-style-callback requires a single defined object, we compose one ourselves if no error object was returned, and we return an array as an error object, if multiple error objects were returned.
Using this abstraction function, we can rewrite the earlier pattern as follows:
async.waterfall([
function(callback) {
prom2cb.chainCallback(Task.sync(), callback);
},
function(callback) {
prom2cb.chainCallback(Task.create({
title: "Get some coffee",
description: "Get some coffee ASAP"
}), callback);
},
function(callback) {
prom2cb.chainCallback(Task.create({
title: "Drink coffee",
description: "Because I need caffeine"
}), callback);
},
function(callback) {
prom2cb.chainCallback(Task.findAll(), callback);
},
function(tasks, callback) {
for(var i = 0; i < tasks.length; i++) {
var task = tasks[i];
console.log(task.title + ": "+ task.description);
}
}
], function(err) {
if(err) {
console.log("An error occurred: "+err);
process.exit(1);
} else {
process.exit(0);
}
});
As may be observed, this code fragment is more concise and significantly shorter.
The opposite mixing pattern also leads to issues. For example, we can first retrieve the list of tasks from the database (through a promise-style invocation) and then write it as a JSON file to disk (through a Node.js-style invocation):
Task.findAll().then(function(tasks) {
fs.writeFile("tasks.txt", JSON.stringify(tasks), function(err) {
if(err) {
console.log("error: "+err);
} else {
console.log("everything is OK");
}
});
}, function(err) {
console.log("error: "+err);
});
The biggest annoyance is that we are forced to do the successive step (writing the file) inside the callback function, causing us to write pyramid code that is harder to read and tedious to maintain. This is caused by the fact that we can only "chain" a promise to another promise.
Fortunately, we can create a function abstraction that wraps an adapter around any Node.js-style function taking the same parameters (without the callback) that returns a promise:
function promisify(Promise, fun) {
return function() {
var args = Array.prototype.slice.call(arguments, 0);
return new Promise(function(resolve, reject) {
function callback() {
var args = Array.prototype.slice.call(arguments, 0);
var err = args[0];
args.shift();
if(err) {
reject(err);
} else {
resolve(args);
}
}
args.push(callback);
fun.apply(null, args);
});
};
}
In the above code fragment, we do the following:
- We define a function that takes two parameters: a Promise prototype that can be used to construct promises and a function representing any Node.js-style function (which the last parameter is a Node.js-style callback).
- In the function, we construct (and return) a wrapper function that returns a promise.
- We construct an adapter callback function, that invokes the Promise toolkit's reject() function in case of an error (with the corresponding error object provided by the callback), and resolve() in case of success. In case of success, it simply propagates any result object provided by the Node.js-style callback.
- Finally, we invoke the Node.js-function with the given function parameters and our adapter callback.
With this function abstraction we can rewrite the earlier example as follows:
Task.findAll().then(function(tasks) {
return prom2cb.promisify(Promise, fs.writeFile)("tasks.txt", JSON.stringify(tasks));
})
.then(function() {
console.log("everything is OK");
}, function(err) {
console.log("error: "+err);
});
as may be observed, we can convert the writeFile() Node.js-style function invocation into an invocation returning a promise, and nicely structure the find and write file invocations by chaining then() invocations.
Conclusions
In this blog post, I have explored two kinds of asynchronous function invocation patterns: Node.js-style and promise-style. You may probably wonder which one I like the most?
I actually hate them both, but I consider promises to be the more powerful of the two because of their composability. However, this comes at a price of doing some extra work to construct them. The most ideal solution to me is still a facility that is part of the language, instead of "forgetting" about existing language constructs and replacing them by custom-made abstractions.
I have also explained that we may have to combine both patterns, which is often quite tedious. Fortunately, we can create function abstractions that convert one into another to ease the pain.
Related work
I am not the first one comparing the function invocation patterns described in this blog post. Parts of this blog post are inspired by a blog post titled: "Callbacks are imperative, promises are functional: Node’s biggest missed opportunity". In this blog post, a comparison between the two invocation styles is done from a programming language paradigm perspective, and is IMO quite interesting to read.
I am also not the first to implement conversion functions between these two styles. For example, promises constructed with the bluebird library implement a method called .asCallback() allowing a user to chain a Node.js-style callback to a promise. Similarly, it provides a function: Promise.promisify() to wrap a Node.js-style function into a function returning a promise.
However, the downside of bluebird is that these facilities can only be used if bluebird is used as a toolkit in an API. Some APIs use different toolkits or construct promises themselves. As explained earlier, Promises/A and Promises/A+ are just interface specifications and only the purpose of then() is defined, whereas the other facilities are extensions.
My function abstractions only make a few assumptions and should work with many implementations. Basically it only requires a proper .then() method (which should be obvious) and a new Promise(function(resolve, reject) { ... }) constructor.
Besides the two function invocation styles covered in this blog post, there are others as well. For example, Zef's blog post titled: "Callback-Free Harmonious Node.js" covers a mechanism called 'Thunks'. In this pattern, an asynchronous function returns a function, which can be invoked to retrieve the corresponding error or result at a later point in time.
References
The two conversion abstractions described in this blog post are part of a package called prom2cb. It can be obtained from my GitHub page and the NPM registry.
No comments:
Post a Comment