easycluster

A cluster wrapper that makes distributing tasks to workers easy

How to install?

npm i -S easycluster

Which is the wee equivalent of:

npm install --save easycluster

How to use it?

Cluster = require 'easycluster'

do ->
  cluster = new Cluster
    workers: 4
    racing: 2
    jobs: () -> [1..10]
    job: (job) ->
      console.log "Job: #{job}"
      job * 2
  result = await cluster.run()
  console.log "Result: #{result}"

Will output something like:

Job: 3
Job: 4
Job: 9
Job: 10
Job: 1
Job: 2
Job: 5
Job: 6
Job: 7
Job: 8
Result: 6,8,18,20,2,4,10,12,14,16

How does it work?

You give 4 pieces of information: Workers: basically, the number of child processes to create. Racing: Each process then can have "fake" concurrency (Node.js doesn't have threads, but you can have sort of a racing condition) Jobs: A method that will return an array. Each element of this array will be given to a worker. It can be async. Job: A method that receives an element of the array. This method is used by workers.

Then you await run and get results.

Oui, très bien, bravo!

Take a look at the source, I tried to make it very simple, plus, it smells coffee.

Own use cases

I personally use it to analyse some data from a MongoDB. I get a list of items, and for each of these items, I have to do some work with other objects. First try I took a look at my CPU and it was not going crazy, so I thought "Hey, why not do something ???". I ended creating a module. I reduced the time of my algorithm by 4 (yep, 4 CPUs) and my system monitor goes crazy 100% CPU (network too, average 60 MiB/s for few secs, which is fun). Plus, it was the occasion to create an NPM module that looks almost serious. Almost. (Feel my CoffeeScript !11one)