subscribe-for-data

Fast implementation of multiple related models properties fetching & mixing to your model.

• DB agnostic.
• flexible query condition
• one DB query per subscription.
• all queries running parallel, starting at same moment, you choose then
• No loops, stream based.
• zero dependencies

Workflow description

You create subscription around your related model with makeSubscription(model, options)

You can create any count of subscriptions you need.

Then you can to subscription.add(target) target objects you want to mix in properties from related model data.

After you've added all needed targets to all subscriptions you can anytime run fillSubscriptions()

fillSubscriptions() assigns data as it goes via stream with auto parallelization if multiple subscriptions created. One query per subscription is executed.

It generates mongo condition. If you return from options.getCondition(target) scalar value then is generated $in query. I
to query your source,

Mongo query generation is just default behavior, you can alter it as you want.

Installation

npm i subscribe-for-data subscribe-for-data-from-mongoose

Import

const mongoosePlugin = require('subscribe-for-data-from-mongoose');
const { 
  makeSubscription, fillSubscriptions 
} = require('subscribe-for-data').use(mongoosePlugin);

Example

By default it works with mongoose. This behavior can be easily overriden by setting custom getStream option callback.

const mongoosePlugin = require('subscribe-for-data-from-mongoose');
const { makeSubscription, fillSubscriptions } = require('subscribe-for-data').use(mongoosePlugin);
const RootModel = require('./MainModel');
const RelatedModel = require('./RelatedModel');
const AnotherRelatedModel = require('./AnotherRelatedModel');

(async () => {
  const relatedSubscription = makeSubscription(RelatedModel, { // single field attach
    targetField: 'position', // key of property to be created on roots
    foreignField: 'root_id', // field to build condition against
    sourceField: 'position'
  });
  const anotherRelatedSubscription = makeSubscription(AnotherRelatedModel, { // something completely different
    getCondition({ mysteriousTimestamp, type }) {
      return { type, updatedAt: { $gt: mysteriousTimestamp} };
    }, 
    assignData(root, { someField, otherType }) {
      (root.someFields = root.someField || []).push(someField);
      (root.otherTypes = root.otherTypes || new Set()).add(otherType);
    },
  });
  const roots = [];
  await RootModel.find({}).cursor().eachAsync((root) => {
    [relatedSubscription, anotherRelatedSubscription]
      .forEach(subscription => subscription.add(root)); // subscribed
    roots.push(root);
  });
  await fillSubscriptions(); // 2 DB queries executed in parallel, no loops then
  console.log(roots[0]);
})();

Expected output:

{
  "_id": "000000",
  "position": 42,
  "someFields": [2, 5, 8, 5],
  "otherTypes": [23, 42, 78]
}

API Reference

• SubscribeForData : object
  • .makeSubscription(source, options) ⇒ Object
  • .fillSubscriptions() ⇒ Promise
  • .assignDefaultOptions(mixin)

• assignData : function
• getKey ⇒ *
• extractKey ⇒ *
• getCondition ⇒ *
• getDataHandler ⇒ function
• getAddingMethod ⇒ function
• getStream : function

SubscribeForData : object

subscribe-for-data

• SubscribeForData : object
  • .makeSubscription(source, options) ⇒ Object
  • .fillSubscriptions() ⇒ Promise
  • .assignDefaultOptions(mixin)

SubscribeForData.makeSubscription(source, options) ⇒ Object

Creates subscription for related model data

| Param | Type | Description | | --- | --- | --- | | source | Object | Source model | | options | Object | Options | | options.targetField | String | field data to be saved into (optional) | | options.baseCondition | Object | Base condition | | options.defaultValue | * | Default value for field | | options.getKey | getKey | Callback which returns unique key from target model (model.id by default) | | options.getCondition | getCondition | returns condition, using target model (model.id by default) | | options.extractKey | extractKey | returns unique key of target model from foreign model | | options.isMultiple | Boolean | if one to many relation | | options.useEachAsync | Boolean | only for mongoose cursor | | options.parallel | Number | parallel parameter for eachAsync if useEachAsync is true | | options.foreignField | String | If getCondition returns scalar values this field will be used for $in | | options.sourceField | String | field to use of foreign model | | options.assignData | assignData | Do model filling by itself, otherwise use targetField | | options.getStream | getStream | returns stream from source and condition (using mongoose model by default) | | options.getDataHandler | getDataHandler | Get data handler for processing related models | | options.getAddingMethod | getAddingMethod | Get add() method of future subscription |

SubscribeForData.fillSubscriptions() ⇒ Promise

Fill subscribed targets

SubscribeForData.assignDefaultOptions(mixin)

change default options

| Param | | --- | | mixin |

assignData : function

Assigns data from foreign model to target

| Param | Type | Description | | --- | --- | --- | | target | Object | your target model | | foreign | Object | foreign model |

getKey ⇒ *

get unique identifier of target for internal indexing

Returns: * - target identifier

| Param | Type | Description | | --- | --- | --- | | target | Object | your target model |

extractKey ⇒ *

get unique identifier of target from foreign model

Returns: * - target identifier

| Param | Type | Description | | --- | --- | --- | | foreign | Object | Foreign model data |

getCondition ⇒ *

get condition

Returns: * - condition, can be scalar or object

| Param | Type | Description | | --- | --- | --- | | target | Object | your target model |

getDataHandler ⇒ function

get foreign data handler

Returns: function - Callback handling data assignment

| Param | Type | Description | | --- | --- | --- | | options | Object | Options | | options.targets | Object | targets index | | options.targetField | String | field data to be saved into | | options.extractKey | extractKey | returns unique key of target model from foreign model | | options.isMultiple | Boolean | if one to many relation | | options.sourceField | String | field to use of foreign model | | options.assignData | assignData | Do model filling by itself, otherwise use targetField |

getAddingMethod ⇒ function

get future subscription.add() method

Returns: function - Callback handling data assignment

| Param | Type | Description | | --- | --- | --- | | options | Object | Options | | options.targets | Object | targets index | | options.getKey | getKey | Callback which returns unique key from target model (model.id by default) | | options.getCondition | getCondition | returns condition, using target model (model.id by default) | | options.defaultValue | * | Default value for field | | options.targetField | String | field data to be saved into | | options.condition | object | DB Query condition, being prepared | | options.extractKey | extractKey | returns unique key of target model from foreign model | | options.foreignField | String | If getCondition returns scalar values this field will be used for $in | | options.inner | Array | Internal array for condition storing |

getStream : function

get stream from model using condition

| Param | Description | | --- | --- | | source | Source model | | condition | Query condition |