task-pool
v0.1.1
Published
A generic pool to limit number of running asynchronous tasks or promises.
Downloads
938
Readme
task-pool
A generic pool to limit number of running asynchronous tasks or promises.
Installation
npm install task-pool
Usage
Consider you are going to do lots of asynchrounous CPU/IO intensive tasks. Lets say running a huge number of external tasks using exec calls, or reading a lot of files in a loop. In this case you may want to limit the number of running tasks at a time to avoid high load and memory usages.
task-pool does the job for you. It makes a queue of tasks and manages to run only a limited number of them at a time.
var Pool = require('task-pool').Pool;
//Create a new pool with a maximum number of 20 tasks at a time.
//Tasks taking longer than 5 seconds continue their journey out of the pool to leave space for new tasks.
var pool = new Pool({limit: 20, timeout: 5000});
//Create a wrapper around exec
var exec = pool.wrap(require('child_process').exec);
//An array of commands to be executed using exec.
var tasks = [ ... ];
for (i = 0; i < tasks.length; i++) {
exec(tasks[i], function(error, stdout, stderr) {
//Do some process on the results
});
}
As you can see exec wrapper acts like the original exec method. The only exception is that it doesn't return the ChildProcess object immediately. It returns a Task
object.
For example if you want to grab the pid of the child process, do it like this:
var task = exec(tasks[i], function(error, stdout, stderr) {
//Do some process on the results
});
task.on('run', function(child) {
console.log(child.pid);
});
You can also wrap a function that returns a promise:
//Create a wrapper around exec
var exec = pool.promise.wrap(require('child-process-promise').exec);
//An array of commands to be executed using exec.
var tasks = [ ... ];
for (i = 0; i < tasks.length; i++) {
exec(tasks[i]).then(result) {
//Do some process on the result
}).catch(error, function(error) {
//Do something when an error occurs
});
}
License
task-pool is released under the MIT license.