batches

You have to make thousands of database queries or API calls? You don't want to throw those millions of queries all at once at your poor backend? Batches is here to help you out.

Examples

For one example is worth a thousand words:

var b = require('batches').batch({ concurrent: 10, context: 'Jimmy', collectData: true });
for (var i = 0; i < 100; ++i) {
	b.add(function(next, idx) {
		setTimeout(function() {
			console.log('This is call #' + (idx + 1) + ', ' + this);
			next(null, 'Result #' + (idx + 1));
		}, 101-idx);
	});
}
b.when(function(errors, data) {
	console.log('All done.');
	console.log(data);
});

API

batches.batch(options)

Create a new batch. Available options:

• concurrent: How many jobs to run at the same time. Default: 100
• context: Context to call the job-functions in. Default: the batch-object itself
• exitOnError: If true, no new jobs are started after the first error occurred. Default: true
• collectData: If true, the data passed to next-callbacks is collected and passed to the when-callback when all jobs are finished. Default: false

batch.add(callback)

Add a new job to a batch. The callback gets called immediately if concurrency allows it. The arguments of the callback are:

• next: Has to be called when the job is done. Can be passed an error object and data.
• index: The index of the job (the jobs get called in the order they are added to the batch)

batch.when(callback)

Add a callback that gets called when all jobs have finished (or exitOnError is set to true and an error occurred). First argument is either null or an array of errors (where array indices and job-index match up), the second argument is either an empty array (when collectData is set to false) or an array containing all data passed to the next-callbacks, again with indices matching the index-parameter of the jobs.