ForkQueue creates a queue where the items are removed via a 'next' message received from child processes. The number of child processes is passed to the constructor along with the full path of the module to run. The processes are spawned via child_process.fork

API

Constructor

var numWorkers = 5,
    module = 'worker.js';

var queue = new Queue(5, 'worker.js');

Enqueue

Add a value to the queue

for (var i = 0; i < 100; i++) {
  queue.enqueue(i);
}

End

Wait for the child processes to work through the queue then kill them.

queue.end(callback);

Worker modules

Worker modules are spawned with child_process.fork. In order to request a value off the queue, they send a 'next' message to the parent with process.send('next'). The only message sent to them contains the value off the queue. They will exit with 'SIGTERM' sent from the parent after queue.end is called.

The simplest possible worker is below.

  process.send('next');

  process.on('message', function(value) {
    // Do something with value

    // Tell the parent to return the next value off the queue
    process.send('next');
  });