What's this?

This is a set of functions specialized in comparing and merging arrays of elements signed by unique set of IDs.

Although it was originally developed to facilitate server ⇒ client data-set replication (changes coming from database side are merged with the client-side date), it can be used in many other scenarios.

Why would I use this?

As mentioned in the into, this lib is specialized for working with database-style arrays, which contain elements signed by unique set of numeric ID fields. For example:

let cities = [
    {cityID:1, cityName:'New York'},
    {cityID:2, cityName:'London'}
];

let streets = [
    {cityID:1, streetID:1, streetName: 'Wall Street'},
    {cityID:1, streetID:2, streetName: 'South Street'},
    {cityID:2, streetID:3, streetName: 'Downing Street'}
];

This lib can help you compare, link, merge, purge, filter, search, sort arrays of this kind.

How to install?

Simply open up a terminal, go to your project directory and run:

npm i --save array-merge-by-id

Try it before you buy it

You can try out all the examples (listed below) in your browser at RunKit.

Functions by category

Comparing, merging, linking elements of two arrays:

• mergeA - merges new/changed elements into an existing array
• compareA - compares elements of two arrays and returns an object containing common elements and differences
• linkA - extends each child array element with a reference to it's parent element
• eachPair - calls a callback method for each matched elements of provided arrays

Searching & filtering:

• filterByKeys - extract all the array elements which match the given key values (or are indicated by a comparer function)
• findFirstById - returns the first matched element of the given type
• indexOf - returns index of first matching element in the given array
• uniqueA - copies unique elements from source to a new array, which is then returned

Modifying, sorting, adding, removing elements of a single array:

• concat - does an in-place concatination of elements of the source array at the end of target array
• clear - removes all the elements of the given array
• purgeA - removes elements indicated by a hit list from the provided array
• overwrite - removes all the elements of target and replaces them with elements from source
• sortOn - sorts the given array based on the given key name array (or comparer function)

Other helper functions:

• compileC - compiles and returns a function which compares two data elements and detects which comes before which in an orderd list
• stringifyIDs - creates CSV containing values of all the keys ending with "ID". It's usefull for debugging

TypeScript typings

This lib was written in TypeScript, which is good news for people using TypeScript - all the typing information is included in the build (don't worry - it can still be used in vanilla JavaScript ... etc).

Functions

compareA(leftA, rightA, key_columns, config) ⇒ ArrayDiffResult

Compares elements of two arrays and returns an object containing common elements and differences (see comparisson results object).

Kind: global function
Returns: ArrayDiffResult - comparisson results object (see ArrayDiffResult)

| Param | Type | Description | | --- | --- | --- | | leftA | Array | first array be compared | | rightA | Array | second array be compared | | key_columns | CompareBy | definition on how elements of two arrays should be compared (see CompareBy) | | config | ArrayDiffConfig | (optional) additional config parameters (see ArrayDiffConfig) |

Example

let leftA = [
  {cityID:1, cityName:'New York', weather:"windy"},
  {cityID:2, cityName:'London',   weather:"raining"}
];

let rightA = [
  {cityID:2, cityName:'London', weather:"thunderstorm"},
  {cityID:3, cityName:'Moscow', weather:"snowing"}
];

let diff = compareA(leftA, rightA, ["cityID"]);

// statment below will print:
// {
//   leftDiff: [{cityID:1, cityName:'New York', weather:"windy"}],        <= New York doesn't exist in the `rightA`
//   leftCommon: [{cityID:2, cityName:'London', weather:"raining"}],      <= London does exist in `rightA`
//   rightDiff: [{cityID:3, cityName:'Moscow', weather:"snowing"}],       <= Moscow exists in `leftA`
//   rightCommon: [{cityID:2, cityName:'London', weather:"thunderstorm"}] <= London exists in `leftA`
// }
console.dir(diff);

filterByKeys(aSearch, key_columns, key_values, findFirstOnly) ⇒ Array

Extract all the array elements which match the given key values (or are indicated by a comparer function)

Kind: global function
Returns: Array - array of matched elements

| Param | Type | Description | | --- | --- | --- | | aSearch | Array | array to be searched | | key_columns | CompareBy | definition on how elements of two arrays should be compared (see CompareBy) | | key_values | Map | (optional) an object which should be matched with an element from aSearch array - it's optional because key_columns param can contain a function which doesn't need it | | findFirstOnly | boolean | (optional) should only the first matched element be returned (defaults to false) |

Example

let streets = [{cityID:22, streetID:1}, {cityID:44, streetID:2}, {cityID:22, streetID:3}];

let myCityStreets = filterByKeys(streets, ['cityID'], {cityID:22});

console.dir(myCityStreets); // will output [{cityID:22, streetID:1}, {cityID:22, streetID:3}]

findFirstById(aSearch, key_columns, key_values) ⇒ Object

Returns the first matched element of the given type

Kind: global function
Returns: Object - matched array element

| Param | Type | Description | | --- | --- | --- | | aSearch | Array | array to be searched | | key_columns | CompareBy | definition on how elements of two arrays should be compared (see CompareBy) | | key_values | Map | (optional) an object which should be matched with an element from aSearch array - it's optional because key_columns param can contain a function which doesn't need it |

Example

let streets = [{cityID:22, streetID:1}, {cityID:44, streetID:2}, {cityID:22, streetID:3}];

let myStreet = findFirstById(streets, ['cityID','streetID'], {cityID:44, streetID:2});

console.dir(myStreet); // will output {cityID:44, streetID:2}

indexOf(aSearch, key_columns, key_values) ⇒ number

Returns index of first matching element in the given array

Kind: global function
Returns: number - element index

| Param | Type | Description | | --- | --- | --- | | aSearch | Array | array to be searched | | key_columns | CompareBy | definition on how elements of two arrays should be compared (see CompareBy) | | key_values | Map | (optional) an object which should be matched with an element from aSearch array - it's optional because key_columns param can contain a function which doesn't need it |

Example

let streets = [{cityID:22, streetID:1}, {cityID:44, streetID:2}, {cityID:22, streetID:3}];

let streetIndex = findFirstById(streets, ['cityID','streetID'], {cityID:44, streetID:2});

console.log(streetIndex); // will print 1

linkA(parentA, childA, key_columns, linkName, config) ⇒ ArrayLinkResult

Extends each child array element with a reference to it's parent element (in parentA array). Optionally it can extend parane element with a map of it's children.

Kind: global function
Returns: ArrayLinkResult - linking results object(see ArrayLinkResult)

| Param | Type | Default | Description | | --- | --- | --- | --- | | parentA | Array | | array containting parent elements | | childA | Array | | array containing child elements | | key_columns | CompareBy | | definition on how elements of two arrays should be compared (see CompareBy) | | linkName | string | "parent" | (optional) name of the property which should be assigned a reference to parent element (defaults to 'parent') | | config | ArrayDiffConfig | | (optional) additional config parameters (see ArrayDiffConfig) |

Example

let cities = [ {cityID:22, cityName:'New York'}, {cityID:44, cityName:'London'} ];
let streets = [{cityID:22, streetID:1}, {cityID:44, streetID:2}, {cityID:22, streetID:3}];

let result = linkA(cities, streets, ['cityID'], 'city')

console.dir(result); // will output [ {cityID:22,streetID:1,city:<reference to New York>},
                     //               {cityID:44,streetID:2,city:<reference to London>},
                     //               {cityID:22,streetID:3,city:<reference to New York>}]

mergeA(currData, newData, key_columns, config) ⇒ ArrayDiffResult

Merges new/changed elements into an existing array

Kind: global function
Returns: ArrayDiffResult - comparisson results object (see ArrayDiffResult)

| Param | Type | Description | | --- | --- | --- | | currData | Array | an array of "current" data elements | | newData | Array | an array of changes and new data elements | | key_columns | CompareBy | definition on how elements of two arrays should be compared (see CompareBy) | | config | ArrayDiffConfig | (optional) additional config parameters (see ArrayDiffConfig) |

Example

let currData = [
  {cityID:1, cityName:'New York'},
  {cityID:2, cityName:'Londonnnnn'}
];

let newData = [
  {cityID:2, cityName:'London'}, // London is fixed
  {cityID:3, cityName:'Rome' }   // Rome is added
];

// function which applies changes to an existing element
let mergeFn = (element, changes) => { element.cityName = changes.cityName; };

let result = mergeA(currData, newData, ['cityID'], { callbackFn: mergeFn });

console.dir(currData); // will print [ {cityID:1, cityName:'New York'},
                       //              {cityID:2, cityName:'London'},
                       //              {cityID:3, cityName:'Rome' } ];

purgeA(aTarget, aHitList, key_columns, config) ⇒ Array

Removes elements indicated by a hit list from the provided array

Kind: global function
Returns: Array - an array of removed elements

| Param | Type | Description | | --- | --- | --- | | aTarget | Array | array to be purged | | aHitList | Array | hit list - indicates which element from aTarget should be removed | | key_columns | CompareBy | definition on how elements of two arrays should be compared (see CompareBy) | | config | ArrayPurgeConfig | (optional) additional config parameters (see ArrayPurgeConfig) |

Example

let targetA = [
  {cityID:1, cityName:'New York'},
  {cityID:2, cityName:'London'},
  {cityID:3, cityName:'Rome' }
];

let hitList = [ { cityID: 1 }, { cityID: 3 } ];

let elFreq = [];

let result = purgeA(targetA, hitList, ['cityID'] { elFreq: elFreq });

console.dir(targetA); // will print [ {cityID:1, cityName:'New York'} ];
console.dir(elFreq); // will print [1, 2] - it means that the first array
                     // element was found once, while the second twice

uniqueA(source, key_columns, config) ⇒ Array

Copies unique elements from source to a new array, which is then returned

Kind: global function
Returns: Array - array of unique elements

| Param | Type | Description | | --- | --- | --- | | source | Array | source array - which may contain duplicates | | key_columns | CompareBy | definition on how elements of two arrays should be compared (see CompareBy) | | config | ArrayUniqueConfig | (optional) additional config parameters (see ArrayUniqueConfig) |

Example

let source = [ {cityID:1, cityName:'New York'}, {cityID:2, cityName:'London'}, {cityID:2, cityName:'London'} ];

let result = uniqueA(source, ["cityID"]);

console.dir(result); // will print [ {cityID:1, cityName:'New York'}, {cityID:2, cityName:'London'} ]

compileC(key_columns) ⇒ function

Compiles and returns a function which compares two data elements and detects which comes before which in an orderd list. The compiled function expects the compared values to be numeric

Kind: global function
Returns: function - compiled function

| Param | Type | Description | | --- | --- | --- | | key_columns | CompareBy | definition on how elements of two arrays should be compared (see CompareBy) |

Example

// function for an ASCENDING order
let ascFn = compileC('cityID','streetID');

// will return 0
ascFn({cityID:1, streetID:1}, {cityID:1, streetID:1});

// will return 1
ascFn({cityID:1, streetID:1}, {cityID:1, streetID:2});

// will return -1
ascFn({cityID:2, streetID:1}, {cityID:1, streetID:2});

// function for an DESCENDING order
let descFn = compileC('cityID:desc','streetID:desc');

// will return 0
descFn({cityID:1, streetID:1}, {cityID:1, streetID:1});

// will return -1
descFn({cityID:1, streetID:1}, {cityID:1, streetID:2});

// will return 1
descFn({cityID:2, streetID:1}, {cityID:1, streetID:2});

eachPair(leftA, rightA, key_columns, callbackFn, config) ⇒ ArrayDiffResult

Calls a callback method for each matched elements of provided arrays

Kind: global function
Returns: ArrayDiffResult - comparisson results object

| Param | Type | Description | | --- | --- | --- | | leftA | Array | first array of elements | | rightA | Array | second array of elements | | key_columns | CompareBy | definition on how elements of two arrays should be compared (see CompareBy) | | callbackFn | function | function to be called for each of mathced element pairs | | config | ArrayDiffConfig | (optional) additional config parameters (see ArrayDiffConfig) |

Example

let cities = [ {cityID:22, name:'New York'}, {cityID:44, name:'London'} ];

let streets = [
  {cityID:22, streetID:1, name:'Elm Street'},
  {cityID:22, streetID:3, name:'Wall St'},
  {cityID:44, streetID:2, name:'Downing St'}
];

let callbackFn = (city, street) => {
  console.log(street.name + ' ' + city.name)
}

eachPair(cities, streets, ["cityID"], callbackFn);

// The following will be printed to console:
// Elm Street in New York
// Wall St in New York
// Downing St in London

sortOn(source, key_columns) ⇒ Array

Sorts the given array based on the given key name array (or comparer function)

Kind: global function
Returns: Array - array passed via source param

| Param | Type | Description | | --- | --- | --- | | source | Array | array to be sorted | | key_columns | CompareBy | definition on how elements of two arrays should be compared (see CompareBy) |

Example

let source = [{cityID:2},{cityID:3},{cityID:1}];

// sorting in ASCENDING order
sortOn(source, ["cityID"]);

console.log(source); // will print [{cityID:1},{cityID:2},{cityID:3}];

// sorting in DESCENDING order
sortOn(source, ["cityID:desc"]);

let streets = [
   {cityID:22, streetID:1, name:'Elm Street'},
   {cityID:44, streetID:2, name:'Downing St'},
   {cityID:22, streetID:3, name:'Wall St'}
];

// using multiple keys
sortOn(source, ["cityID", "streetID:desc"]);

clear(target) ⇒ Array

Removes all the elements of the given array

Kind: global function
Returns: Array - array passed via target param

| Param | Type | Description | | --- | --- | --- | | target | Array | array to be cleared |

Example

let source = [{cityID:2},{cityID:3},{cityID:1}];

clear(source);

console.dir(source); // will print []

overwrite(target, source) ⇒ Array

removes all the elements of target and replaces them with elements from source

Kind: global function
Returns: Array - array passed via target param

| Param | Type | Description | | --- | --- | --- | | target | Array | array to be overwritten | | source | Array | array who's elements are to be places in target |

Example

let target = [{cityID:100},{cityID:200}];
let source = [{cityID:2},{cityID:3},{cityID:1}];

overwrite(target, source);

console.dir(target); // will print [{cityID:2},{cityID:3},{cityID:1}]

concat(target, source) ⇒ Array

Does an in-place concatination of elements of the source array at the end of target array

Kind: global function
Returns: Array - array passed via target param

| Param | Type | Description | | --- | --- | --- | | target | Array | array which should receive new elements | | source | Array | array of elements which should be added at the end of the target array |

Example

let target = [{cityID:100},{cityID:200}];
let source = [{cityID:2},{cityID:3},{cityID:1}];

concat(target, source);

console.dir(target); // will print [{cityID:100},{cityID:200},{cityID:2},{cityID:3},{cityID:1}]

stringifyIDs(data) ⇒ string

Creates CSV containing values of all the keys ending with "ID". It's usefull for debugging

Kind: global function
Returns: string - CSV of key values

| Param | Type | Description | | --- | --- | --- | | data | Object | object or array to be converted to CSV |

Example

let obj = { cityID: 11, streetID: 22, streetName: 'Elm Street' };

let result = stringifyIDs(obj);

console.log(result); // will print "cityID:11,streetID:22"

Param types

CompareBy

Functions which compare array elements need to be instructed how two elements can be compared. This can be done in two ways:

• by passing an array of ID property names, which should be compared to determin the relation of the two objects
• by passing a comparer function, which receives tow elements and returns a numeric value indicating the relation of the objects

If an array of property names is passed, a comparer function will be compiled automatically (via compileC) function.

The following snippet shows how comparisson can be defined via a comparer function:

let leftA = [
  {cityID:1, cityName:'New York', weather:"windy"},
  {cityID:2, cityName:'London',   weather:"raining"}
];

let rightA = [
  {cityID:2, cityName:'London', weather:"thunderstorm"},
  {cityID:3, cityName:'Moscow', weather:"snowing"}
];

const comparerFn = (leftEl, rightEl) => {
    if(leftEl.cityID===rightEl.cityID) {
        return(0);
    }
    if(leftEl.cityID<rightEl.cityID) {
        return(1);
    }
    return(0);
}

let diff = compareA(leftA, rightA, comparerFn);

The previous example can be re-written to define comparrison via property name array instead of comparer function:

let leftA = [
  {cityID:1, cityName:'New York', weather:"windy"},
  {cityID:2, cityName:'London',   weather:"raining"}
];

let rightA = [
  {cityID:2, cityName:'London', weather:"thunderstorm"},
  {cityID:3, cityName:'Moscow', weather:"snowing"}
];

let diff = compareA(leftA, rightA, ["cityID"]);

Descending order

If a ID name array is passed as key_columns params, the compiled function will compare elements in ascending order. We can change this behaviour by appending :desc to a ID name ... like so: ['cityID:desc','streetID:desc']

See an example given for the sortOn function.

ArrayDiffConfig

We can modify the way the functions work, by providing an config object. All the options in the config object are optional.

Here's a list of available options:

• sortLeftBy - how should the leftA array be sorted
• sortRightBy - how should the rightA be sorted
• skipSort - set it to true if arrays are not to be sorted
• unique - set it to true if are all the array elements unique - it speeds up the algorithm
• linkName - property assigned to the rightA elements, which should be pointing to the matching element in the leftA
• mapName - property assigned to the leftA elements, containig array of all the matched elemens from the rightA
• callbackFn - a callback function, which should be called for each of the matched element pairs

sortLeftBy and sortRightBy

Description: defines how the arrays passed to the function should be sorted

Defaults to: value passed as key_columns param

Expected value: we can pass a function or an array of ID param names (see CompareBy)

In order to be more efficient, functions which rely on comparing array elements will sort both of the given arrays.

By default the functions use key_columns parameter to sort the arrays.

This can be overriden by specifying a dedicated sorting order for each of the two arrays:

• config.sortLeftBy = defines how the left array should be sorted (passed as the first param)
• config.sortRightBy = defines how the right array should be sorted (passed as the second param)

Sorting of an array can be disabled by assigning null to corresponding sort config param:

{
    sortLeftBy: null // don't sort the left array
}

ArrayPurgeConfig

This config si very similar to ArrayDiffConfig param type. The following params are the same as in ArrayDiffConfig:

• sortLeftBy
• sortRightBy
• skipSort

The following params are unique to this param type:

• sortBy = how should both arrays be sorted
• mapRemoved - flag indicating should removed elements be mapped and returned
• matchMulti - can an element from the hit list array be matched with multiple elements from the target array (defaults to false)

ArrayUniqueConfig

• skipSort - set it to true if arrays are not to be sorted
• elFreq - output param - an array in which element frequency is to be recorded (see the example given in the uniqueA method description)

Return Types

ArrayDiffResult

ArrayDiffResult contains results of comparing two arrays. It has the following structure:

{
    // an array of elements from left array (`leftA` param), which have not
    // been matched with any of the elements of the right array (`rightA` param)
    leftDiff:Array,
    // an array of elements from left array (`leftA` param),
    // which have been matched with at leas one element of the right array (`rightA` param)
    leftCommon:Array,
    // an array of elements from right array (`rightA` param), which have not
    // been matched with any of the elements of the left array (`leftA` param)
    rightCommon:Array,
    // an array of elements from right array (`rightA` param),
    // which have been matched with at leas one element of the left array (`leftA` param)
    rightDiff:Array
}

ArrayLinkResult

ArrayLinkResult contains results of linking two arrays. It has the following structure:

{
    // parent elements with no matched child elements
    childless:Array,
    // parent elements with matching children
    parents:Array,
    // child elements with matching parent
    children:Array,
    // child elements without a matching parent
    orphans:Array
}

License

MIT License, http://www.opensource.org/licenses/MIT