npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

work-queue

v0.0.6

Published

Process scheduled items from a queue held in MongoDB.

Downloads

3

Readme

The Work Queue

Control a queue of tasks to be scheduled, completed, repeated, etc.

Connecting to the Queue

Use WorkQueue.connect(url, [opts]).

Default opts shown here.




WorkQueue = require('work-queue').connect("mongodb://localhost:27017/test", {
	collection: "workQueue",
	readerId: [ "reader-", 5 ], // 5 chars of randomness
	// consider, readerId: "app-server-56"
})

Scheduling Jobs


WorkQueue.push({
	type: "my-type",
	schedule: { at: timestamp }
})

You can schedule when the job is due using:

  • at with an absolute timestamp in ms
  • every with an interval in ms, example:
    	WorkQueue.push({ type: "foo", schedule: { every: 30*1000} })
  • after with a delay in ms, example:
    	WorkQueue.push({ type: "foo", schedule: { after: 5*60*1000 } })

You can combine every and after, to control when the first iteration occurs.

Working the Queue

This is not an abstract Queue. It is meant to hold items that need to process, complete, fail, retry, and recur.

Documents in the queue always have these fields:

* type: string
* ctime: timestamp
* mtime: timestamp
* status: "new"|"complete"|"failed"|<worker-id>
* schedule: object

Plus any fields given when pushed.

To turn the current process into a Worker, first you must teach it how to handle each type of item it will find there.

	WorkQueue.register('my-type', function(item, done) {
		// item has all the fields shown above
		doWorkOnItem(item, function (err) {
			if(err) { done(err) }
			else { done() }
		})
	})
	
	worker = WorkQueue.createWorker({
		idle_delay: 100 // polling interval if nothing to do
	})
	
	worker.resume()
	// run this example for 10 seconds, then pause
	setTimeout(worker.pause, 10000)
	

A usable example can be found in bin/queue-reader.coffee.

bin/queue-reader.coffee

Usage: queue-reader [options...] mongodb://host:port/db_name

Options:
  -c, --collection  the collection to hold work orders in                                                         [default: "workQueue"]
  -i, --interval    when idle, how often to look for new work                                                     [default: 100]
  -r, --require     require this/these module(s), which should export type handlers                               [default: ""]
  -d, --demo        DANGEROUS: load an example queue as a test, will flush all jobs in the specificed collection  [default: false]

The -r or --require option is the most important if you want to do real work. It can be given multiple times, and each string given to it will be passed to require() within the reader script.

Each module required in this way should export an object full of { type: handler } pairs.

Example:

module.exports['echo'] = function (item, done) {
	console.log(item)
	done()
}

The -i or --interval option is only meaningful when the queue is empty. When each work item is completed, a check for new work is performed immediately.