Flexible callback arguments

Recently, I released a small library called Nimble, which is an experiment in unifying a synchronous and asynchronous API. Its early days yet, but I thought I'd share my approach so others can explore the viability of the technique too.

I wanted to combine the functions of Underscore.js and my own Async library, so a single function can act both synchronously and asynchronously. This is easy enough to achieve by adding an optional callback to each function, but its not so simple when modifying the iterator. Below is an example of the various ways people might use a synchronous function:

_.map([1,2,3], function (value) { ... });
_.map([1,2,3], function (value, index) { ... });
_.map([1,2,3], function (value, index, arr) { ... });

You have the option of specifying all the arguments, or just the ones you need. However, with an asynchronous map, we need to pass in a callback. This is conventionally the last argument to a function in node.js. The problem is, we can't omit parameters from our iterator because we always need the last argument:

_.map([1,2,3], function (value, index, arr, callback) { ... });

That's looking a bit verbose! Most of the time, however, the arr and index arguments are not used. Becase of this, the original Async library's map function just used the following arguments for iterators:

async.map([1,2,3], function (value, callback) { ... });

That saves you some typing, but it's annoying being unable to use the other parameters. It also means the async API differs even more from the original synchronous one. My approach to this problem when writing Nimble was to inspect the iterator's arity. For those of you not familiar with the term, arity is the number of arguments a function accepts. To find out the arity of a function, just look at its .length property.

var fn = function (one, two, three) { ... };
// fn.length == 3

var fn = function (one) { ... };
// fn.length == 1

This works cross-browser, and allows us to modify the arguments we pass to the iterator. First, I define a list of the full arguments, then I remove elements not used by the async iterator and add a callback to the end:

var test = function (iterator) {
    // the full list of available arguments
    var args = ['value', 'index', 'arr'];

    // remove the unused arguments
    args = args.slice(0, iterator.length - 1);

    // add the callback to the end
    args.push('callback');

    // run the iterator with the new arguments
    return iterator.apply(this, args);
};

console.log('\nExample one:\n');

test(function (value, index, arr, callback) {
    console.log(value);
    console.log(index);
    console.log(arr);
    console.log(callback);
});

console.log('\nExample two:\n');

test(function (value, callback) {
    console.log(value);
    console.log(callback);
});

Running this file would result in the following output:

Example one:

value
index
arr
callback

Example two:

value
callback

As you can see, we're now able to vary the number of arguments a function accepts, and the callback is always just the last argument accepted. This is not without its problems however. If an iterator does not define its arguments in the normal way, and instead uses the arguments object, we won't know how many arguments to provide.

test(function () {
    console.log(arguments[0]);
    console.log(arguments[1]);
    console.log(arguments[2]);
    console.log(arguments[3]);
});

Would give the following output:

value
index
callback
undefined

Ok, that's not what we'd expect. What you would probably expect in this circumstance, is for the whole list of possible arguments to be passed in. Lets update our test function to handle iterators with an arity of zero.

var test = function (iterator) {
    // the full list of available arguments
    var args = ['value', 'index', 'arr'];

    if (iterator.length) {
        // remove the unused arguments
        args = args.slice(0, iterator.length - 1);
    }

    // add the callback to the end
    args.push('callback');

    // run the iterator with the new arguments
    return iterator.apply(this, args);
};

You should now see the following output when running the example above:

value
index
arr
callback

Success! If you want to see the above technique in action, check out the Nimble library. I'd also like to hear about any potential issues introduced by this idea in the comments.