generator-async
v0.1.6
Published
Generator-based asynchronous control flow for Node.js
Downloads
14
Maintainers
Readme
generator-async
Flexible generator-based asynchronous control flow for Node.js.
Introduction
With generator-async
use both callback-based and promise-based libraries all together with a clean, consistent interface. Use the yield
keyword before an asynchronous function call that would normally take a callback or return a promise, and skip the callback or call to then()
.
var async = require('generator-async');
// load `fs` with async's `require`
var fs = async.require('fs');
async.run(function*() {
// read a file
var passwd = yield fs.readFile('/etc/passwd');
// then read another file
var group = yield fs.readFile('/etc/group');
// then spit out the contents of both files
console.log(passwd, group);
});
In this example, the contents of /etc/passwd
are read from disk and assigned to passwd
. Program execution waits on the I/O and assignment, while the process event loop keeps on moving. That's in contrast to fs.readFileSync
for example, where the entire process waits.
Notice fs
was loaded through async.require
, which wraps async methods to be compatible. If you'd rather not wrap, you can load modules directly like normal, and just refer to async.cb
wherever a standard callback would be expected. See the Alternative Explicit Callback Interface section below for more.
Running in Parallel
Run functions concurrently with async.parallel
and then yield in the same order.
async.run(function*() {
// kick off read operations with async.parallel
async.parallel( fs.readFile('/etc/passwd') );
async.parallel( fs.readFile('/etc/group') );
// yield in order
var passwd = yield async.yield;
var group = yield async.yield;
});
Examples
Working with Request
var request = async.require('request');
async.run(function*() {
var data = yield request('http://www.google.com');
console.log(data);
});
Working with Redis
var redis = async.require('redis');
async.run(function*() {
var client = redis.createClient(6379, '172.17.42.1', {});
yield client.hset("hash key", "hashtest 1", "some value");
yield client.hset("hash key", "hashtest 2", "some other value");
var data = yield client.hkeys("hash key");
console.log(data);
});
Working with Express
Use generator functions directly as route handlers.
var express = async.require('express');
var fs = async.require('fs');
var app = express();
app.get('/', function*(req, res) {
var data = yield fs.readFile('/etc/passwd');
res.end(data);
});
Working with the file system
var fs = async.require('fs');
var mkdirp = async.require('mkdirp');
var rimraf = async.require('rimraf');
var path = require('path');
var assert = require('assert');
async.run(function*() {
// come up with a random-ish nested dir name and see that it doesn't exist yet
var name = path.join('mkdirp-' + parseInt(Math.random() * 10000), 'nested', 'dir');
assert.equal(yield fs.exists(name), false);
// create the new directory and see that it exists
var err = yield mkdirp(name);
assert.equal(yield fs.exists(name), true);
// remove our nascent dir and see that it no longer exists
yield rimraf(name);
assert.equal(yield fs.exists(name), false);
});
Wrapping Classes
Wrapped classes that define methods as generator functions will automatically have those generator functions wrapped in a generator-async context, meaning they'll be invoked and executed when called by consumers.
Consider a Class representing a file:
var File = function() {
this.initialize.apply(this, arguments);
};
File.prototype = {
initialize: function(filename) {
this.filename = filename;
},
read: function*() {
var contents = yield fs.readFile(this.filename);
return contents;
},
size: function*() {
var stat = yield fs.stat(this.filename);
return stat.size;
}
};
File = async(File);
Consume this functionality in an async.run
context:
async.run(function*() {
var file = new File('/etc/passwd');
var size = yield file.size();
console.log(size);
});
Or, consume this functionality as-is just like normal with conventional callbacks.
var file = new File('/etc/passwd');
file.size(function(err, size) {
console.log(size);
};
Alternative Explicit Callback Interface
Sometimes it's not feasible to wrap a module or method to be yieldable since it may have a non-standard callback scheme. Or you may prefer the more verbose interface in order to use the module directly and shed some of the intermediary magic. In that case, node's require
like normal, and refer to async.cb
where a callback is expected:
// require `fs` directly
var fs = require('fs');
async.run(function*() {
// refer to async.cb where the callback would go
var contents = yield fs.readFile('/etc/passwd', async.cb);
console.log(contents);
});
Use async.cb
where a standard node callback would be expected (a function that accepts err
and data
parameters). For non-standard callbacks refer to async.raw
to get back all the values (and handle errors yourself).
API
async(input)
Implementation depends on the type of input. Given a function, or generator, returns a function that is both yieldable in an async.run
context, and also compatible with a standard callback interface. Given an object or class, returns the input with each of its methods wrapped to be yieldable. Aliased as async.wrap
.
async.require(module, [hints])
Imports a module Like node's native require
, but also wraps that module so that its methods are yieldable in an async.run
context with no callbacks necessary.
That's the goal at least. Wrapping involves making a heuristic best guess about which methods are asynchronous and what is their callback signature etc. So while it often works well, you may sometimes need to give hints about the makeup of the module. See module-async-map for more.
If you'd rather, feel free to use node's native require
instead, and refer to async.cb
where a callback is expected.
async.run(fn*)
Invokes and executes the supplied generator function.
async.proxy(fn)
Returns a wrapped version of the supplied function compatible to be run either in an async.run
context or standard node callback style.
async.fn(fn*)
Returns a function that when called will invoke and execute the supplied generator function.
Collection Methods
Since the yield
keyword is only valid directly inside of generator functions, we can't yield
inside of stock Array
methods, which might be exactly what you want to do sometimes. Instead, use these collection methods, which accept generator functions as iterator functions, so you can yield
from within them. Underlying implementations courtesy of the fantastic async library, which has more documentation.
async.forEach(arr, fn*)
Applies fn*
as an iterator to each item in arr
, in parallel. Aliased as async.each
.
var fs = async.require('fs');
async.run(function*() {
var filenames = [...];
var totalBytes = 0;
yield async.forEach(filenames, function*(filename) {
var stat = yield fs.stat(filename);
totalBytes += stat.size;
console.log(filename, stat.size);
});
console.log("Total bytes:", totalBytes);
});
async.eachLimit(arr, limit, fn*)
Same as async.forEach
except that only limit
iterators will be simultaneously running at any time.
async.map(arr, fn*)
Produces a new array of values by mapping each value in arr through the generator function.
async.run(function*() {
var filenames = [...];
var existences = yield async.map(filenames, function*(filename) {
return yield fs.exists(filename);
});
console.log(existences); // => [ true, true, false, true, ... ]
});
async.mapLimit(arr, limit, fn*)
Same as async.map
except that only limit iterators will be simultaneously running at any time.
async.filter(arr, fn*)
Returns a new array of all the values in arr which pass an async truth test.
async.run(function*() {
var filenames = [...];
var bigFiles = yield async.filter(filenames, function*(filename) {
var stat = yield fs.stat(filename);
return stat.size > 1024;
});
console.log("Big files:", bigFiles);
});
async.reject(arr, fn*)
The opposite of async.filter
. Removes values that pass an async truth test.
async.reduce(arr, memo, fn*)
Reduces arr
into a single value using an async iterator to return each successive step. memo
is the initial state of the reduction. Runs in series.
async.reduceRight(arr, memo, fn*)
Same as async.reduce
, only operates on arr in reverse order.
async.detect(arr, fn*)
Returns the first value in arr
that passes an async truth test. The generator function is applied in parallel, meaning the first iterator to return true will itself be returned.
async.sortBy(arr, fn*)
Sorts a list by the results of running each arr value through an async generator function.
async.run(function*() {
var filenames = [...];
var sortedFiles = yield async.filter(filenames, function*(filename) {
var stat = yield fs.stat(filename);
return stat.size;
});
console.log("Sorted files:", sorted);
});
async.some(arr, fn*)
Returns true if at least one element in the arr satisfies an async test.
async.every(arr, fn*)
Returns true if every element in arr satisfies an async test.
async.concat(arr, fn*)
Applies iterator to each item in arr, concatenating the results. Returns the concatenated list.
History and Inspiration
This is an evolution of gx, with inspiration from other generator-based control flow libraries such as co, genny, galaxy, and suspend.
License
Copyright (c) 2015 David Chester
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.