rulesEngine.io

Isomorphic JavaScript rules engine based on spoken language verb tenses.

• Features
• Definitions
• Notational Convention
• Installation
• Basic Usage
  • Default Step Configuration
  • Caching
  • Pretty Print Workflow Plan
• Configuration
  • Rules
    • Conditions
    • Prerequisites
    • Logic
    • Error Handling
  • Rules Provider
  • Logging Provider
    • Debugging (Workflow Stack)
  • Dispatch Provider
    • Rabbit MQ
    • React Redux
  • Expanding Verb Support
    • Anti-Patterns
• Questions & Answers

Features

• Automatic runtime construction of hierarchical workflows.
• Multi-tenant environment and feature flags compatible.
• Asynchronous hooks on success and failure of operation
• Implicit support for any form of storage, e.g. PostgreSQL, MySQL, MongoDB, Redis, etc.
• Overridable strategy for logging.
• Support for promises and async/await.
• Complete Test Suite.
• Caching for optimized workflow retrieval

Definitions

We will be using several terms throughout this library. For clarification, we will use the following definition of those terms: | Term | Description | | --- | --- | | Workflow | A (pre)defined set of tasks to be applied, in the defined order on its subject. | | Rules Engine | An automated means to evaluate criteria associated with (a collection of) rules at runtime, based on incoming data to select and apply the applicable tasks to that data. | | Task | A predefined amount of logic to be applied to data. | | Rule | A task with criteria under which to execute it. | | Isomorphic | Executable/behaving the same in both browser, as well as a server-side (nodeJS) environments. | | spoken language verb tenses | Past, present/imperative, and future tense of a verb. Note, in the English language there is no real difference between a verb's present and future tense, so, we will combine the verbs with "will", "doing", and "did" prefixes to clearly distinguish the phases. | | Namespace | A grouping of related relations. Used as an isomorphic replacement instead of e.g. "database", or "domain". | | Relation | A specific "datatype" in whatever representation; Compare with e.g a MongoDB "Collection", a SQL "Schema", or a "class" or "data structure" in code. |

Notational Convention

• verb tense should always be provided,
• namespace and relation can be omitted,
• status, is only added when applicable.

I.e.: willCreate_item_type is equivalent to verb = "willCreate", namespace = "item", relation = "type", while doingUpdate__ is equivalent to verb = "doingUpdate", namespace = any, relation = any. Finally remove___success would reference the rules that should be run asynchronously after any remove operation succeeds (independent of namespace or relation).

Installation

npm install rulesEngine.io

To use it in the Browser or any other (non CJS) environment, use your favorite CJS bundler. No favorite yet? Try: Browserify, Webmake or Webpack.

Usage

Below is a quick example how to use rulesEngine.io:

const RulesEngine = require('rulesEngine.io');
const rulesArray = require('./yourLocalRulesArray');

const rulesEngine = new RulesEngine(rulesArray);

const context = {
    verb: 'doingGet',
    namespace: 'item',
    relation: 'type'
};

const output = await rulesEngine.execute({}, context);

In this example, the RulesEngine constructor takes one parameter:

1. An array of rules objects. See Rules Format

Subsequently, rulesEngine.execute takes two parameters:

1. The request object. Passed into the selected tasks as is.
2. The context object. Used to select the appropriate rules to apply to the request object.

Out of the box, rulesEngine.io comes with support for 4 verbs: create, get, update and delete, and will therefore look for any rules in rulesArray for willGet, doingGet, and didGet.

Default Step Configuration

const steps = {
    get: ['willGet', 'doingGet', 'didGet'],
    create: ['willCreate', 'doingCreate', 'didCreate'],
    update: ['willUpdate', 'doingUpdate', 'didUpdate'],
    remove: ['willRemove', 'doingRemove', 'didRemove'],
    count: ['willCount', 'doingCount', 'didCount']
}

You can pass in support for additional verbs by passing in a similarly formatted object

const steps = {
    import: ['willImport', 'doingImport', 'didImport'],
    refresh: ['willRefresh', 'doingRefresh', 'didRefresh']
}

const RulesEngine = require('rulesEngine.io');
const rulesArray = require('./yourLocalRulesArray');

const rulesEngine = new RulesEngine(rulesArray, {steps});

The result is support for the both the original, and the passed in steps.

Caching

const RulesEngine = require('rulesEngine.io');
const rulesArray = require('./yourLocalRulesArray');

const rulesEngine = new RulesEngine(rulesArray);

async function execute({ query, payload }, context) {
    //generate workflow using cached workflow if available
    const workflow = await rulesEngine.createWorkflow(context);
    // execute workflow
    const output = await rulesEngine.execute({ query, payload }, workflow, context);
}

Pretty Print Workflow Plan

async function execute(data, context){
    //pre-generate workflow using cached workflow if available
    const workflow = await rulesEngine.createWorkflow(context);
    // output JSON object:
    console.log(workflow.toJSON());
    //print multi-line plain text (e.g. for inclusion of plain text log files):
    console.log(workflow.toString());
    // ... execute workflow
}

Configuration

• Rules
• Settings

All settings are optional: | Setting | Usage | default behavior | |---|---|---| | logging | Pass in a Logging provider | logging to console | | enableWorkflowStack | Enable provision of Workflow Stack| false | | dispatch | Wire up Dispatch provider | non-awaited promise call | | steps | Add support for additional verbs | see Default Step Configuration | | states | Use State identifier in your own language| ['success', 'fail' ]| | cacheAge | Duration to cache workflows in milliSeconds | 5000 ms |

Rules

"Rules" in rulesEngine.io are a combination of a set of conditions and logic. The conditions are evaluated from the Rules Provider, and to some extend included on the workflowStack. However, after the initial construction of the workflow, it serves no functional purpose anymore. This allows 2 separate rule-styles:

1. Conditions and logic in one file
2. Conditions in a separate (query-able) source, with references to the logic files

While technically you could store the logic in a query-able resource as well, we strongly recommend against storing code in a database as e.g. strings and interpreting those strings as code and executing the result.

In the examples provided, we will combine conditions and logic as if in 1 file, but as stated, they can be separated.

Example Rule

{
    verb: 'willCreate',
    namespace: undefined,
    relation: undefined,
    status: undefined,
    description: 'Prevent the creation of the record if one already exists with the same `title` property.'
    // obtain any additional info needed to perform the logic
    prerequisites:[{
        context:{
            verb:'count',
            namespace: undefined,
            relation: undefined,
            status: undefined,
        }
        query: ({data:{query, payload}, context}) => {
            return { title: payload.title };
        },
        payload:undefined
    }],
    //this is the actual logic:
    logic: async ({data:{query, payload}, prerequisiteResults, context, workflowStack}) => {
        const [countResult] = prerequisiteResults;
        if (countResult > 0){
            throw new Error('Duplicate Exception');
        }
        return {query, payload};
    }
}

The undefined values are for clarity of the example, in reality the properties can be omitted entirely.

Conditions

Finally, any other property that is passed in on the context will be compared to the rule's value for that same property, if that property also exists with a non-undefined value on the rule.

The idea behind this approach is that, beyond the basic verb-based workflow construction, it allows for a versatile inclusion/exclusion mechanism by any other property. Examples could be general concepts as a tenantId, or as specific as e.g. specific properties of the data. It is completely controlled by you in what you pass in as "context" to rulesEngine.createWorkflow(context) or rulesEngine.execute(data, workflow, context).

Note: If you need any more advanced rule selection mechanisms: it is only limited by your own creativity if you implement your own Rules Provider.

Prerequisites

1 In your particular system it may or may not be more efficient to do a get and check if anything is returned.

A prerequisite is an object with at least a context:

• context - This object is required. It has the following properties itself:
  • verb - Required. One of the base verbs. E.g. count, or get, but never one of the future or past tense forms.
  • namespace - Optional. If empty, the original request's namespace will be used.
  • relation - Optional. If empty, the original request's relation will be used.
  • status - Optional. If absent will NOT use the original request's relation will be used.

Any other properties are optional, and control what is passed into the actual prerequisite workflow. They can either be a fixed value, or a function receiving the data and (parent) context.

For Example:

 {
   //...
   prerequisites: [{
        context:{
            verb: 'update',
            namespace: 'devices',
            relation: 'iotDevice'
        },
        query:({data, context})=>{ return {_id:data._id}; },
        payload:[{op:'replace', path:'/online', value:true}]
   }]
   logic: async (...) => {...}
 }

with the following data object:

{
    "_id":"2001"
}

will result in the following object that is passed into the update_devices_iotDevice workflow as data:

{
    "query":{"_id":"2001"},
    "payload":[{"op":"replace", "path":"/online", "value":true}]
}

Aborting a prerequisite workflow

Rules logic

 {
   //...
   /**
    * @param {object} parameters
    * @param {any} parameters.data the data object to work on
    * @param {any[]} parameters.prerequisiteResults Array of resulting objects from each prerequisite, or an empty array if there were none.
    * @param {Context} parameters.context the context for the current request
    * @param {WorkflowStack} [parameters.workflowStack] for debugging only, the workflowStack as applicable at the start of executing `logic()` I.e. the current rule is marked with _ACTIVE, if enabled.
    * @param {(data:object,context:Context)=>Promise<void>} parameters.dispatch the dispatch function to emit events
    * @param {LoggingProvider} parameters.log Logging object to output your logging needs
    **/
   logic: async ({data, prerequisiteResults, context, workflowStack, dispatch, log}) => {
        const [countResult] = prerequisiteResults;
        if (countResult > 0){
            throw new Error('Duplicate Exception');
        }
        const output = data;
        return output;
    }
 }

During execution, the logic method is passed 5 or 6 parameters:

• data - This is the the data you passed in to the execute() method, after all previous tasks (if any) have been applied to it. If this is a task in a prerequisite workflow, this includes any transformations as defined on the prerequisite definition of the parent rule.
• prerequisiteResults - a (possibly empty) array with the results from all prerequisites workflows, in the same order as they are defined on the prerequisites.
• context - the context as passed into execute(), or, in case of a prerequisite workflow, as constructed in the prerequisite definition.
• workflowStack - If enabled, the workflowStack.
• dispatch - the dispatch function to emit events
• log - Logging object to output your logging needs

Aborting a workflow

Error Handling

Sometimes, when an error is thrown in the rules logic, it makes sense to perform some error handling right on the spot (rather than in a __fail rule). Either because recovery is possible, or because you need to undo some thing done in that rule's logic. rulesEngine.io allows defining an onError method on a rule to perform either tasks:

 {
   //...
   /**
    * @param {object} parameters
    * @param {Error} parameters.error the error object throw in `logic()`
    * @param {any} parameters.data the data object as passed into `logic()`
    * @param {Context} parameters.context the context object as passed into `logic()`
    * @param {WorkflowStack} [parameters.workflowStack] for debugging only, the workflowStack as applicable at the start of`logic()` I.e. the current rule is marked with _ACTIVE, if enabled
    * @param {(data:object,context:object)=>Promise<void>} parameters.dispatch the dispatch function to emit events
    * @param {LoggingProvider} parameters.log Logging object to output your logging needs
    **/
   onError: async ({error, data, context, workflowStack, dispatch, log}) => {
        if (error.message.includes('Warning')){
            //recovery possible, or error can be ignored, return a result for the next rule to continue with.
            const output = data;
            return output;
        }
        //no recovery possible, rethrow original error (or throw a new one)
        throw new Error('Non Recoverable State');
    }
 }

Unlike logic() which does not need to return a result, onError() needs to either throw or re-throw an error, or return a result. If nothing is returned, rulesEngine.io will throw an error and abort the workflow, to guarantee proper thought has been put into the error handling

Rules Provider

// rulesRepository.js

module.exports = {
  /**
   * @param {object} context the context object as passed into `rulesEngine.execute`
   * @param {string} context.verb the verb tense to retrieve rules for
   * @param {string} context.namespace
   * @param {string} context.relation
   * @param {string} [context.status]
   * ... any other property you provide on the context object, examples could be tenant, user,
   */
  async find: ({verb, namespace, relation, status, ...context}) => {
      //... perform your own logic to retrieve and filter rules.
      //E.g. query a database for a list of applicable rules, then retrieve the rules from the file system
  }

}

const RulesEngine = require('rulesEngine.io');
const rulesRepository = require('./rulesRepository');

const rulesEngine = new RulesEngine(rulesRepository);

Logging Provider

Logging provider template:

// logging.js
module.exports = {
  /**
  * Called with detailed debugging information
  * @param {string | object} message
  * @param {object} context
  * @param {object} [workflowStack] workflow Stack, if enabled
  */
  debug: (message, context, workflowStack) => console.log(message),

  /**
  *
  * @param {string} message
  * @param {object} context
  * @param {object} [workflowStack] workflow Stack, if enabled
  */
  info: (message, context, workflowStack) =>  console.info(message),

  /**
  * Called for logging expected error conditions
  * @param {string} warning
  * @param {object} context
  * @param {object} [workflowStack] workflow Stack, if enabled
  */
  warn: (warning, context, workflowStack) =>  console.warn(warning),

  /**
  * Called for unhandled errors.
  * @param {Error} error Javascript Error object. Note: this object is not serializable by default
  * @param {string} error.message
  * @param {string} error.stack
  * @param {object} context
  * @param {object} [workflowStack] workflow Stack, if enabled
  */
  error: (error, context, workflowStack) => console.error(error),
}

const RulesEngine = require('rulesEngine.io');
const logging = require('./logging');

const rulesEngine = new RulesEngine(rules, { logging });

If a logging provider is specified, any non-implemented log levels will default to log to the console as depicted above.

Debugging (Workflow Stack)

To enabled it, set the enableWorkflowStack parameter on the configuration to true:

const rulesEngine = new RulesEngine(rules, { enableWorkflowStack:true });

When enabled, rules will get called with an additional workflowStack parameter available for logging or inspecting during debugging. Additionally, logging from rulesEngine.io itself will include the workflow stack.

The workflowStack is a JSON object, including key information about the constructed workflow, as well as the progress through that workflow.

Example The following is example output of a workflow stack

[
    // first task
  {
    "verb": "doingGet",
    "description": "Perform the `find` operation on the database",
    //this task was executed, and this was the value returned by this task.
    //As such, this value is what was used as payload for the second task, as well as for that task's prerequisites
    "RESULT": [
      {
       //... some result
      }
    ]
  },
  // second task
  {
    "verb": "didGet",
    "namespace": "deviceManagement",
    "relation": "settings",
    "description": "Assure existence of settings record",
    "prerequisites": [
      [
        {
          "TRANSFORMATIONRESULT": "pending",
          "description": "Message transformation towards create_deviceManagement_settings ",
          "_ABORTED": "Settings Exists. No need to create."  //The transformation step aborted the rest of this sub-workflow
        },
        {
          "verb": "willCreate",
          "namespace": "deviceManagement",
          "relation": "settings",
          "description": "Prevent creating 2nd global settings",
          "_SKIPPED": true  // this task was never executed as an earlier step aborted the (sub) workflow
        },
        {
          "verb": "doingCreate",
          "description": "Perform the `create` operation on the database",
          "_SKIPPED": true  // this task was never executed as an earlier step aborted the (sub) workflow
        }
      ]
    ],
    "_ACTIVE": true  // This stack was generated entering THIS rule
  }
]

Dispatch Provider

RabbitMQ

const rabbit = require('foo-foo-mq');
// ... rabbit configuration omitted for brevity

const dispatch = async (message, context) => rabbit.publish('rulesEngine.exchange', {message, context});

const rulesEngine = new RulesEngine(rules, { dispatch });

React Redux

function someAction(store, action, next){
    const rulesEngine = new RulesEngine(rules, { dispatch: store.dispatch });
    const context = ...
    const output = await rulesEngine.execute(action, context);
    return next(output);
}

module.exports = closeAction;

Expanding Verb Support

const steps = {
    import: ['willImport', 'doingImport','didImport'],
    heartbeat: ['heartbeat']
};

const rulesEngine = new RulesEngine(rules, { steps });

Any rules mappings defined this way will be combined with the standard operations, with any mappings you provide taking replacing the default ones if applicable.

Note: like heartbeat in the above example, it is possible to define less than 3 steps for a verb. While there might be exceptions where you know you will never use a before and after phase, generally this should be avoided as it prevents easy extensions in the future. (If you are worried about performance when you have a slow rules provider, use rulesEngine.createWorkflow to take advantage of its caching abilities).

Anti-Patterns

const steps = {
    import: ['splitInBatches', 'validateValues', 'createDependencies', 'import', 'upsertImportRecord'],
};

The steps in this configuration are very specific, and predefine the complete workflow, completely bypassing the flexibility of the rules engine.

Another likely code-smell is having more than 1 rule for the present tense.

Questions and Answers

Question: Why are you use spoken language verb tenses?

Question: What logic should I implement for each verb tense?

| Verb Tense | Usage | CRUD examples | | ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | | present/imperative (doing + verb) | Only the core operation the verb indicates2. E.g. the actual operation on the database, or the actual http call. Typically we only have 1 rule per namespace/relation for this tense. | doingCreate, doingGet, doingupdate, doingDelete | | future (will + verb) | Anything you might want to do in preparation for doing the actual operation. E.g. validation, adding timestamps, etc | willCreating, willGet, willUpdate, willDelete | | past (did + verb) | Anything you want to do synchronously after the actual operation has completed. E.g. adding computed or default values before the value is returned to the client. | didCreate, didGet, didUpdate, didDelete | | verb___success | Any logic to run asynchronously from the original request upon successfully executing the original request. E.g. sending a welcome email when a new user account is created. | create___success, update___success, get___success3, delete___success | | verb___fail | Any logic to run asynchronously from the original request when that original request failed to execute. E.g. | create___fail, update___fail, get__fail3, delete___fail |

2 Keeping the present logic as generic, simple, and single-responsible as possible will help drive reuse and simplicity. 3 get___success and get___fail will rarely have implementations. Examples would be very specific logic for e.g. failing or succeeding in retrieving a user for login, for audit/tracking purposes.

Question: Can I have more than 1 rule for the present tense?

If you are doing multiple things, look at the primary thing you are doing ("save something"), and put that in the present-tense rule. Then look at the secondary action ("update it's dependencies"). If that fails, should the primary action still fail? If so, put it in a past-tense ( did{verb} ) rule. If not, trigger it from a {verb}__success rule.

Note, if you would need to roll-back the primary action if the secondary action fails, you could put the secondary action in the past-tense (did{verb}) rule. However, if the request came from a client that is waiting for a response, it might be better to just return the result from the primary action, then trigger the secondary action(s) from the {verb}__success rule, and IF the secondary actions fail, trigger a new update to roll back the primary action. It will be much easier to debug, and (if applicable) it load-balances better.

Question: I have a deploy verb, and need to do multiple actions during our deployment. Can I have multiple rules for the present tense (doing{verb})?

• run database migrations
• deploy code
• run regression test suite

If so, you might want to introduce a migrate verb, and have a willDeploy__ rule with that as the prerequisite, and a regress verb that you dispatch from a deploy__success rule.

Question: I like the tool, but we do everything in our code in .... (fill in your own native language). Can we use verb tenses in our language?

const steps = {
    creeer: ['gaatCreeren', 'creeer','gecreerd'],
    vind: ['gaatVinden', 'vind','gevonden'],
    verander: ['gaatVeranderen', 'verander', 'verandered'],
    verwijder: ['gaatVerwijderen', 'verwijder', 'verwijderd']
};

const rulesEngine = new RulesEngine(rules, {steps});

Just make sure to match the verb properties on your rules with your the step names as you configure in the options

Question: I don't want "success" and "fail", I want the spanish "exito" and "fallar".

const estado = ['exito', 'fallar']

const motorDeReglas = new RulesEngine(rules, {states:estado});
// without any further configuration, this will result in e.g. `create_item_type_exito`

Just make sure to have the "success" state first, and the "fail" state second.

Question: Where should I implement creating a patch record, in a 2nd rule in doingCreate, the did{verb} or in the {verb}__success?

Question: You seem rather specific about what goes in each verb-tense, why??

• Single Responsibility - keeping rules small forces you to keep the number of things you do in them limited
• Open Closed - it is easy to add new rules without interfering with other rules if they all only do a very specific thing
• DRY - it is easy to re-use small rules. When rules become more complex, they often also become harder to re-use
• KISS - just keep it simple...

And perhaps most of all, it intentionally forces you as developer to think about how you should break up functionality, before you begin. Pausing a moment and thinking about what you are now exactly going to code, before starting to code is usually a good thing...