monocle-js

monocle-js is a Node library for using a blocking-like syntax when writing asynchronous code. In other words, it's one way to avoid the 'callback hell' so many Javascript developers love to hate. It's a port of the Monocle library for event-driven Python, made possible by ES6's new generators and the yield keyword. (For present-day javascript, we use Facebook's Regenerator to bundle in a generator runtime.)

Install with: npm install monocle-js

The problem

A lot of Node libraries and Javascript libraries in general follow the callback pattern. This isn't bad in and of itself but it encourages developers to write code that drifts rightward and becomes difficult to read. Let's say we want to read some data from the web and write it to a file:

var request = require('request')
  , fs = require('fs');

var myLibraryFunction = function(jsonUrl, cb) {
    request(jsonUrl, function(err, resp, body) {
        if (err) {
            return cb(err);
        }
        fs.writeFile('/path/to/my/file.json', body, function(err) {
            if (err) {
                return cb(err);
            }
            cb(null, resp, body);
        });
    });
};

myLibraryFunction('http://somesite.com/json/data', function(err, resp, body) {
    if (err) {
        console.log("Downloading and writing file failed!");
    } else {
        console.log("Downloading and writing file was successful!");
        console.log(resp);
        console.log(body);
    }
});

This is obviously a contrived example, but when building up a library of project-specific functionality, you often find yourself doing this in Node.

Here's what the same code could look like using Monocle:

var request = require('monocle-request')
  , fs = require('monocle-fs')
  , monocle = require('monocle-js')
  , o_O = monocle.o_O;

var myLibraryFunction = o_O(function*(jsonUrl) {
    var data = yield request(jsonUrl);
    yield fs.writeFile('/path/to/my/file.json', data[1]);
    return data;
});

var main = o_O(function*() {
    try {
        var data = yield myLibraryFunction('http://somesite.com/json/data');
        console.log("Downloading and writing file was successful!");
        console.log(data[0]);
        console.log(data[1]);
    } catch (err) {
        console.log("Downloading and writing file failed!");
    }
});

monocle.launch(main);

As you can see, the points at which you would have created an anonymous function to handle the asynchronous callback, you now simply use the yield keyword to block until the callback's result is ready. And the library functions you create are 'monoclized' by wrapping them (or 'decorating them') with the o_O method.

Also, notice that you don't need to do any explicit error handling if you don't want to. Errors will be thrown like synchronous JS code, and can be caught and handled appropriately.

It's important to understand that Monocle methods cannot simply be called like normal functions--to actually begin executing them, pass them as arguments to monocle.launch. There's also a convenience method for launching a monoclized generator directly: monocle.run. With monocle.run, you can avoid having to define a main function in the example above, like so:

monocle.run(function*() {
    try {
        var data = yield myLibraryFunction('http://somesite.com/json/data');
        console.log("Downloading and writing file was successful!");
        console.log(data[0]);
        console.log(data[1]);
    } catch (err) {
        console.log("Downloading and writing file failed!");
    }
});

Using callback-based methods

Of course, in the previous examples, I've required monocle-fs and monocle-request, libraries which didn't exist until I created them. What if you want to make use of arbitrary callback-based library methods? You can do that with Monocle as well. Here's the previous example without the assumption that request and fs have already been 'monoclized'.

var request = require('request')
  , fs = require('fs')
  , monocle = require('monocle-js')
  , o_O = monocle.o_O
  , o_C = monocle.callback;

var myLibraryFunction = o_O(function*(jsonUrl) {
    var cb = o_C();
    request(jsonUrl, cb);
    var data = yield cb;
    cb = o_C();
    fs.writeFile('/path/to/my/file.json', data[1], cb);
    yield cb;
    return data;
});

monocle.run(function*() {
    try {
        var data = yield myLibraryFunction('http://somesite.com/json/data');
        console.log("Downloading and writing file was successful!");
        console.log(data[0]);
        console.log(data[1]);
    } catch (err) {
        console.log("Downloading and writing file failed!");
    }
});

The way it works is that first we create a special callback to be used in the library method. We do this by calling monocle.callback() or monocle.o_C() for short. This creates a callback we pass to the asynchronous method. Then, on the next line, we simply yield to the callback, which blocks execution until the asynchronous method is done and we have a result. Using this strategy, it's easy to incorporate library methods which have not yet been monoclized.

Functions which have been monoclized are called 'o-routines', and, from within other o-routines, can simply be yielded to without creating a callback. This is why we simply yield myLibraryFunction(jsonUrl) in the example above.

Yield! As in traffic

Reading through the examples above, you may have noticed that we're using yield in an interesting way. In a typical Javascript generator, yield is used to send data from the generator to the caller of next(). You could think of this as "yield as in crops". In Monocle, yield is a sign that we should wait for the result of an asynchronous function. It's much better to think of this as "yield as in traffic" (hat tip to the other monocle for this explanation). Take a look at this code from the example above:

var data = yield request(jsonUrl);
yield fs.writeFile('/path/to/my/file.json', data[1]);
return data;

In the first line, we're using yield as a way to retrieve the result of an asynchronous o-routine called request. This is what we mean by "yield as in traffic". In the second line, we're doing the same kind of thing, only we're not assigning the result to anything. These lines essentially say, "wait until the asynchronous process is finished, and give me the result".

The third line, using the return statement, is how we actually send a result back to whoever is calling this particular function. So, the rule of thumb is this:

• yield when you want to wait for an o-routine or a callback and get its result
• return when you want to send back the result of an o-routine

Because of these semantics, Monocle checks to make sure the only type of thing you're yielding is an o-routine or callback. You can also yield iterators (what you get when you call generators) directly--they're converted into o-routines on the fly. See the examples below:

var myFunc = o_O(function*() {
    // this is good, monocle.utils.sleep is an o-routine
    yield monocle.utils.sleep(1000);

    // this is good, cb is a monocle callback
    var cb = o_C();
    setTimeout(cb, 1000);
    yield cb;

    // this is good; pauseBriefly() returns an iterator
    var pauseBriefly = function*() {
        yield monocle.utils.sleep(500);
    };
    yield pauseBriefly();


    // this is bad, we should be returning instead; Math.pow is not an o-routine.
    // Monocle will throw an error
    yield Math.pow(2, 3);

    // this is bad, we should be returning instead.
    // Monocle will throw an error
    yield 5;
});

Porting async libraries

We saw above how to make use of pre-existing async functions in o-routines, using monocle callbacks. Monocle also provides a helper function which can be used to turn a node-style async function into an o-routine automatically: monocle.monoclize() or monocle.o_M() for short. By "node-style async function", I mean one which takes a series of parameters, the last of which is a callback. Monocle assumes this callback takes at least two parameters: the first of which is an error object (or null) used to determine whether the original function completed successfully.

Let's look at an example. fs.readFile() is a node-style async function. We can convert it into an o-routine like this:

var monocle = require('monocle-js')
  , o_O = monocle.o_O
  , o_C = monocle.o_C
  , fs = require('fs');

var monoclizedRead = o_O(function*(filePath) {
    var cb = o_C();
    fs.readFile(filePath, cb);
    return (yield cb);
});

We can eliminate this boilerplate by using monoclize():

var monocle = require('monocle-js')
  , o_O = monocle.o_O
  , o_M = monocle.o_M
  , fs = require('fs');

var monoclizedRead = o_M(fs.readFile);

For an example of how this is used to port entire Node libraries, check out monocle-fs.

Using Promises

Promises are another way of handling asynchronous control flow in Javascript. There are many popular promises implementations, including Q and Bluebird. Library methods which return promises can be used without modification in Monocle. For example, here's a bit of functional testing code using Wd.js's promise library:

var wd = require('wd')
  , browser = wd.promiseRemote()
  , should = require('chai').should();

browser
    .init({ browserName: 'chrome' })
    .then(function () {
        return browser.get("http://admc.io/wd/test-pages/guinea-pig.html");
    })
    .then(function () { return browser.title();})
    .then(function (title) {
        title.should.equal('WD Tests');
    })
    .fin(function () { browser.quit(); })
    .done();

This code opens up a Chrome browser, navigates to a URL, and asserts that the title is as expected. We could rewrite this using Monocle as follows:

var wd = require('wd')
  , run = require('monocle-js').run
  , browser = wd.promiseRemote()
  , should = require('chai').should();

run(function*() {
    yield browser.init({ browserName: 'chrome' });
    yield browser.get("http://admc.io/wd/test-pages/guinea-pig.html");
    var title = yield browser.title();
    title.should.equal('WD Tests');
    yield browser.quit();
});

Many libraries exist which make use of promises, and Monocle makes it easy to take advantage of them in a more streamlined and elegant fashion.

Running o-routines in parallel

Having a blocking syntax is great, but we can also take advantage of the fact that we're not really blocking! If we have a bunch of o-routines we want to run in parallel, it's really easy:

var monocle = require('monocle-js')
  , run = monocle.run
  , ll = monocle.parallel
  , sleep = monocle.utils.sleep
  , o_O = monocle.o_O;

var method1 = o_O(function*() {
    yield sleep(500);
    console.log("Hello");
});

var method2 = o_O(function*(sleepTime, text) {
    yield sleep(sleepTime);
    console.log(text);
});

run(function*() {
    yield ll([
        method1,
        [method2, 250, "World!"]
    ]);
});

In the above example, we have two methods. The first prints out "Hello" after waiting half a second. The second prints out arbitrary text after waiting an arbitrary amount of time. Using the parallel (or ll) method, we can run both of these simultaneously. Because of this parallelism, the result will actually look like this:

World!
Hello

And, everything will be printed out in half a second, not three quarters of a second. You can see in the call how to pass parameters to parallel methods: simply wrap the method and it's parameters in an array.

Chaining o-routines

It's common practice in Javascript to run asynchronous methods on the results of other asynchronous methods. This can be visualized like a chain:

obj1.foo() --> obj2
  obj2.bar() --> obj3
    obj3.baz() --> result

Or in code:

var result = obj1.foo().bar().baz()

Of course, if foo, bar, and baz are asynchronous, this would really look like:

obj1.foo(function(err, obj2) {
    obj2.bar(function(err, obj3) {
        obj3.baz(function(err, result) {
            // do something with result
        });
    });
});

Out of the box, Monocle makes this easier, but we still have to yield each call separately, since they're each going to be o-routines:

var result = yield (yield (yield obj1.foo()).bar()).baz();

This can admittedly get a bit ugly. However, if we define foo as a chainable o-routine, then we can get around this limitation:

Clazz.prototype.foo = monocle.chainable(function*() {
    // do some stuff
    return objWithBarMethod;
});

var obj1 = new Clazz();
var result = yield obj1.foo().bar();

All that matters is that the first function be defined as chainable. There's also a nice alias for monocle.chainable: o_P;

Getting errors out of Monocle

Despite the coolness of Monocle, Javascript is still, at the end of the day, callback-based. We haven't descended into fibers! What this means is that this will throw an unhandled error:

try {
    monocle.run(function*() {
        yield myFunc();
        yield sleep(50);
        throw new Error("whoops");
    });
} catch (e) {
    // we'll never get here, instead the error will crash the script
}

Remember: monocle.run and monocle.launch are not synchronous! So if you want to do something with an error inside run, catch it with .fin or .nodeify:

monocle.run(function*() {
    yield myFunc();
    yield sleep(50);
    throw new Error("whoops");
}).fin(function(err) {
    console.log(err);  // whoops
});

If you don't, we'll throw the error in the general context, which might cause strange undefined behavior, as mentioned above.

Enabling Javascript generators

By default, generators and proxies (used for chaining) are not enabled in the V8 Javascript engine which powers Node. In Node 11, generators are available but not enabled unless you pass the --harmony flag. If you're using monocle-js, make sure to do that!

Making libraries compatible with vanilla Node-style async methods

Let's say you want to use monocle-js to write your code, but you also want to export library methods for people who don't have monocle-js to use. Basically, what we need is a way to take an o-routine (or generator) and turn it into a regular Node-style callback-based method. This is super easy with monocle.nodeify (or monocle.no for short--get it? For when you want to say "no" to monocle!). Take this simple o-routine, for example:

var sleepTwice = o_O(function*(x, y) {
    yield sleep(x);
    yield sleep(y);
});

If we wanted to export this for others to use, we can simply add this to our exports:

module.exports.sleepTwice = monocle.no(sleepTwice);

Now someone can call it as expected in their own callback-based code:

var sleepTwice = require('yourlibrary').sleepTwice;

sleepTwice(50, 100, function(err) {
    // do stuff
});

Running tests

Monocle's tests are written in Mocha. Simply run this command:

mocha --harmony test/

Case study

I ported monocle-js for use in Yiewd, a generator-based WebDriver client library. All WebDriver calls are HTTP-based, and given Node's callback-based HTTP library, WebDriver test code descends quickly into callback hell. Yiewd is a good example of how an existing callback-based library can be wrapped easily and its methods turned into o-routines for use with Monocle.

Once you've 'monoclized' an existing library, or created a new library using o-routines, it's easy to write asynchronous Javascript code in an easy-to-read synchronous fashion.

Monocle-enabled libraries

A list of Node libraries that export o-routines:

• Yiewd
• monocle-fs
• monocle-request

Fashion

One of the awesome things about the original monocle was that the decorator (@_o) looked like a monocle-bearing individual! We can't start names with @ in Javascript, hence the use of o_O in the port. But we have options. These are all exported for your particular taste:

monocle.o0
monocle.o_0
monocle.oO
monocle.o_P  // for chaining