npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@smartrent/heretic

v0.5.2

Published

An opinionated background job runner. Job payloads, logs, and metadata are backed by Postgres. Job queueing and execution are backed by RabbitMQ.

Downloads

47

Readme

Heretic

An opinionated background job runner. Job payloads, logs, and metadata are backed by Postgres. Job queueing and execution are backed by RabbitMQ.

Also, ES6/7.

Why?

Lots of job queues are backed by Redis, but in order to leverage Redis's queueing in a consistent way, you have to write a lot of Lua scripts, and you can still easily run into issues with jobs getting stuck in weird, inconsistent states.

Redis is also not a great option if you want to be able to search and filter jobs (by type, by payload, etc). It's up to you (or your queueing library) to implement your own indexing.

Should I use it?

Probably not. It wouldn't be my first Terrible Idea™.

Usage

Not implemented yet:

  • Automatic failure retries
  • Tests

Database Schema

Feel free to add any other triggers/indexes you'd like.

CREATE TABLE heretic_jobs (
  id SERIAL NOT NULL PRIMARY KEY,
  queue_name text NOT NULL,
  status text DEFAULT 'pending',
  payload jsonb,
  attempt_logs jsonb[] DEFAULT '{}',
  max_attempts int NOT NULL DEFAULT 1,
  created_at timestamptz NOT NULL DEFAULT now(),
  updated_at timestamptz NOT NULL DEFAULT now(),
  last_attempted_at timestamptz
);

CREATE INDEX ON heretic_jobs (queue_name);
CREATE INDEX ON heretic_jobs (status);

CREATE FUNCTION heretic_updated_at_timestamp() RETURNS TRIGGER AS $$
BEGIN
  NEW.updated_at = now();
  RETURN NEW;
END;
$$ LANGUAGE 'plpgsql';

CREATE TRIGGER update_heretic_jobs_updated_at
  BEFORE UPDATE ON heretic_jobs
  FOR EACH ROW EXECUTE PROCEDURE heretic_updated_at_timestamp();

Example

import Promise from 'bluebird';
import Heretic from 'heretic';
import knex from 'knex';

const heretic = new Heretic('amqp://127.0.0.1', knex({
  client : 'pg',
  connection : 'postgres://127.0.0.1',
}));

heretic.process('my.job', function(job, message, done) {
  Promise.resolve()
    .then(() => {
      // do some processing or something. whatever you want!
    })
    .nodeify(done);
});

heretic.process('other.job', function(job, message, done) {
  setTimeout(() => {
    done(new Error('job failed'));
  }, 1000);
});

// the 'jobError' event happens when a job message is published in RabbitMQ that
// can never be handled correctly (malformed JSON, job id doesn't exist in the
// database, etc.). The message will be dead-lettered for later inspection (by you)
heretic.on('jobError', (err) => {
  console.error('Error with job!', err.stack);
});

// the 'jobFailed' event happens when a job fails, but in a recoverable way. it
// will be automatically retried up to the maximum number of retries.
heretic.on('jobFailed', (err) => {
  console.error('Job execution failed!', err.stack);
});

// enqueue a job
heretic.enqueue('email.send', {
  to : '[email protected]',
  body : 'Hi',
});