covey

Covey is a Node.js manager for long- or short-running background processes.

Using Covey, you declaratively state what work you want done, and Covey will make sure that the work is completed.

Covey is similar to forever or worker-manager in that it is a useful way of keeping a long-running process alive and retrying on exits. It is different in that it is only meant for managing Node.js processes that adhere to a specific interface that need to convey status and completion information.

Covey is useful for splitting up an application into multiple independent pieces in situations where you need processes to be resilient to crashing and block the event loop more than normal, or when you simply want to declaratively sequence an application.

Limitations

Covey is probably better-suited for CLI utilities and Electron applications than server applications. Currently covey will only run one worker per process, and will not reuse processes.

In addition, errors are usually thrown globally instead of simply notifying interested parties. This makes a CoveyManager in "host mode" very vulnerable to crashes, which is acceptable for usage in Electron apps under controlled environment, but not with many users.

Because Covey uses introspection to get worker constructor names, you cannot currently use UglifyJS name mangling with your workers.

Roadmap

• option to completely nuke a group, reset, stop, and clear all workers inside
• Test multiple groups - no duplicate workers across groups
• Test running bogus headless runner
• Add support for results and data but as a proper pubsub channel, readable stream, promise
  • support streaming data within a reduxy app, to stream input
• Add support for health checks
• Add support for logging, especially across IPC boundaries
• Add examples, in vanilla JS and TypeScript
• Better story and testing around error handling in general
• Better story around handling workers that refuse to stop
• Maybe get rid of client / host manager stuff or simplify it
• Document foreign process workers, managers, manager clients/hosts, headless runners

Concepts

Covey Manager

A Covey Manager is an object that manages covey workers - usually one process per application will be responsible for managing workers.

Covey Workers

A class that adheres to the CoveyWorker interface and a process that runs the worker.

Covey Group

A Covey Group is a named grouping of CoveyWorkers within a CoveyManager. You can set a group declaratively ("here is a list of all workers I want run in this group"): the manager will diff declared workers with existing workers; new workers will be started, workers that are no longer listed are torn down, and workers in both sets are untouched. You can also use groups in a less declarative way to add new workers or remove existing workers.

Usage

Declare a worker:

import { CoveyWorker, CoveyWorkerLifecycleState } from 'covey';
import SerialPort from 'serialport';
import http from 'http';

const { Crashed, Stopping, Stopped } = CoveyWorkerLifecycleState;

class StreamSerialPortToHttp extends CoveyWorker {
  params: {
    serialport:string,
    baudrate:number,
    port:number,
  };

  /**
   * A worker constructor must complete synchronously, and should take
   * configuration. It may synchronously validate input.
   * All parameters must be JSON-serializable!
   */
  constructor (serialport: string, baudrate: number, port: number) {
    super();
    this.params = {
      serialport,
      baudrate,
      port
    };
  }

  /**
   * Called a maximum of once per worker instance.
   */
  runWorker () {
    this.starting('begin'); // status update

    const s = new SerialPort(this.serialport, { baudRate: this.baudrate });

    this.state.serialLink = s;

    s.on('open', () => {
      this.starting('serial port opened'); // status update

      const responder = (req, res) => {
        s.pipe(res);
      };

      const server = http.createServer(responder).listen(this.port);

      this.state.server = server;

      server.on('close', () => {
        this.state.server = null;
        if (this.lifecycle.state !== Stopping) {
          this.crashed('http server closed');
        } else if (!this.state.serialLink) {
          this.stopped('stopped cleanly (from server close)');
        }
      });
    });

    s.on('close', () => {
      this.state.serialLink = null;
      if (this.lifecycle.state !== Stopping) {
        this.crashed('serial port closed');
      } else if (!this.state.server) {
        this.stopped('stopped cleanly (from serialLink close)');
      }
    });
  }

  /**
   * Called by worker manager or end-user to stop this worker.
   */
  stopWorker () {
    this.stopping('closing serial link and server');
    this.state.serialLink && this.state.serialLink.close();
    this.state.server && this.state.serialLink.close();
    // should be closed cleanly once server and serial link are closed
  }
}

Declare available workers in a manifest:

import { CoveyManifest } from 'covey';

const manifest = new CoveyManifest();

// Declare previously defined worker
manifest.declare(StreamSerialPortToHttp);

Then, create a manager somewhere with a manifest of available workers, and the standalone script that can be called to start foreign-process workers (optional):

import { CoveyManager } from 'covey';

const mgr = new CoveyManager(manifest, './worker.js');

Finally, communicate with your manager to start a group of workers:

import { CoveyGroup } from 'covey';

mgr.runGroup(new CoveyGroup('myWorkers', [
  new StreamSerialPortToHttp('/dev/tty.123', 9600, 7770).setOptions({ foreign: true }),
  new StreamSerialPortToHttp('/dev/tty.234', 115200, 7771).setOptions({ foreign: true }),
  new StreamSerialPortToHttp('/dev/tty.345', 921600, 7772).setOptions({ foreign: true })
]));

In addition to the options specific to a type of worker, all workers have common options like foreign, which defaults to true (foreign workers are run in their own process; non-foreign workers are run in the same process as the manager).

Then, you can subscribe to manager events to receive notifications about workers, query state, and perform other actions: TODO