json-merger
v2.0.0
Published
Merge JSON (or YAML) files and objects with indicators like $import $remove $replace $merge etc
Downloads
97,299
Maintainers
Readme
json-merger
Merge JSON (or YAML) files and objects with operations like $import $remove $replace $merge and more.
Table of Contents:
API
.mergeFile(file: string, config?: Config)
javascript
var jsonMerger = require("json-merger");
var result = jsonMerger.mergeFile("a.json");
a.json:
{
"$merge": {
"source": {
"$import": "b.json"
},
"with": {
"prop1": {
"$replace": {
"prop1a": "this will replace b.json's property prop1"
}
},
"prop2": {
"prop2a": "this will merge with b.json's property prop2"
}
}
}
}
b.json:
{
"prop1": {
"prop1b": "will be replaced"
},
"prop2": {
"prop2b": "will be merged"
}
}
result
{
"prop1": {
"prop1a": "this will replace b.json's property prop1"
},
"prop2": {
"prop2a": "this will merge with b.json's property prop2",
"prop2b": "will be merged"
}
}
.mergeFiles(files: string[], config?: Config)
javascript
var jsonMerger = require("json-merger");
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json:
{
"a": "some value"
}
b.json:
{
"b": "some other value"
}
result
{
"a": "some value",
"b": "some other value"
}
.mergeObject(object: object, config?: Config)
javascript
var jsonMerger = require("json-merger");
var object = {
a: {
aa: "some value",
},
b: {
$import: "b.json",
},
};
var result = jsonMerger.mergeObject(object);
b.json:
{
"bb": "some other value"
}
result
{
"a": {
"aa": "some value"
},
"b": {
"bb": "some other value"
}
}
.mergeObjects(objects: object[], config?: Config)
javascript
var jsonMerger = require("json-merger");
var object1 = {
a: [1, 1, 1, 1],
};
var object2 = {
a: [2, 2],
};
var result = jsonMerger.mergeObjects([object1, object2]);
result
{
"a": [2, 2, 1, 1]
}
Merger(config?: Config)
The actual Merger
class is also exported. The other exports are just shortcut methods.
Using one Merger
instance has some performance advantages because it will cache previously loaded and processed files.
javascript
var Merger = require("json-merger").Merger;
var merger = new Merger();
// the first call will load and process "a.json"
var result1 = merger.mergeFile("a.json");
// the second call will return the cached result
var result2 = merger.mergeFile("a.json");
// clear the caches
merger.clearCaches();
Config
interface Config {
cwd?: string;
enableExpressionOperation?: boolean;
errorOnFileNotFound?: boolean;
errorOnRefNotFound?: boolean;
operationPrefix?: string;
params?: object;
stringify?: boolean | "pretty";
defaultArrayMergeOperation: "combine" | "replace" | "concat";
}
cwd: string
The current working directory when importing files. Defaults to process.cwd().
enableExpressionOperation: boolean
Set this property to true
to enable the $expression operation.
IMPORTANT: Do not use it to run untrusted code because it uses the node:vm module.
errorOnFileNotFound: boolean
Set this property to false
to disable throwing errors when an imported file does not exist.
errorOnRefNotFound: boolean
Set this property to false
to disable throwing errors when an JSON pointer or JSON path does not exist.
operationPrefix: string
Use this property to override the prefix to indicate a property is an operation like $import.
The default prefix is $
but it is possible to change this to for example@
to use keywords like@import
.
params: object
Object that will be available in $expression
operations as $params
variable.
stringify: boolean | "pretty"
Set this property to true
to stringify the JSON result. Set the property to "pretty"
if the output should be pretty printed.
defaultArrayMergeOperation: "combine" | "replace" | "concat"
Set this property to override default merge operation.
Default value is set to "combine"
. Possible values are:
Operations
$import
Use $import
to import other JSON or YAML files.
Files imported with $import
are processed before the result is returned.
{
"$import": "a.json"
}
JSON reference syntax is supported. The following example will import the first array item from the someArray
property in a.json
.
{
"$import": "a.json#/someArray/0"
}
When defined as an array, $import
will process and merge the files in order before returning the result.
{
"$import": ["a.json", "b.yaml", "c.json"]
}
When importing a file it is also possible to provide a different $params
object.
Setting this property will override the Config.params
property.
{
"$import": {
"path": "a.json",
"params": {
"prop": "some value that will be available in a.json as $params.prop"
}
}
}
The object syntax is also supported in an array.
{
"$import": [
{
"path": "a.json",
"params": {
"prop": "value1"
}
},
{
"path": "a.json",
"params": {
"prop": "value2"
}
}
]
}
Use $include
to process a file in the current scope.
$merge
Use the $merge
operation to merge objects and arrays.
javascript
var result = jsonMerger.mergeFile("a.json");
a.json
{
"$merge": {
"source": {
"a": {
"aa": "some value"
}
},
"with": {
"a": {
"bb": "some other value"
}
}
}
}
result
{
"a": {
"aa": "some value",
"bb": "some other value"
}
}
Merging with other files
The $merge
operation is often used with the $import
operation to merge other files from within the JSON itself.
javascript
var result = jsonMerger.mergeFile("a.json");
a.json
{
"$merge": {
"source": {
"$import": "b.json"
},
"with": {
"a": {
"bb": "some other value"
}
}
}
}
b.json
{
"a": {
"aa": "some value"
}
}
result
{
"a": {
"aa": "some value",
"bb": "some other value"
}
}
$remove
Use the $remove
operation to remove properties and array items.
Remove object properties
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"prop1": {
"prop1a": "some value"
},
"prop2": {
"prop2a": "some other value"
}
}
b.json
{
"prop2": {
"$remove": true
}
}
result
{
"prop1": {
"prop1a": "some value"
}
}
Remove array items
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [1, 2, 3]
}
b.json
{
"someArray": [
{
"$remove": true
},
{
"$remove": true
}
]
}
result
{
"someArray": [3]
}
$replace
Use the $replace
operation to replace properties and array items.
Replace object properties
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"prop1": {
"prop1a": "some value"
},
"prop2": {
"prop2a": "some other value"
}
}
b.json
{
"prop2": {
"$replace": {
"prop2b": "replaced value"
}
}
}
result
{
"prop1": {
"prop1a": "some value"
},
"prop2": {
"prop2b": "replaced value"
}
}
Replace array items
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [
{
"a": 1
},
{
"b": 2
}
]
}
b.json
{
"someArray": [
{
"$replace": {
"c": 3
}
}
]
}
result
{
"someArray": [
{
"c": 3
},
{
"b": 2
}
]
}
$concat
Use the $concat
operation to concatenate two arrays.
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [1]
}
b.json
{
"someArray": {
"$concat": [2]
}
}
result
{
"someArray": [1, 2]
}
$combine
Use the $combine
operation to combine two arrays.
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [1, 2, 3]
}
b.json
{
"someArray": {
"$combine": [3, 3]
}
}
result
{
"someArray": [3, 3, 3]
}
$append
Use the $append
operation to append an item to an array.
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [1, 2, 3]
}
b.json
{
"someArray": [
{
"$append": 4
}
]
}
result
{
"someArray": [1, 2, 3, 4]
}
$prepend
Use the $prepend
operation to prepend an item to an array.
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [1, 2, 3]
}
b.json
{
"someArray": [
{
"$prepend": 4
}
]
}
result
{
"someArray": [4, 1, 2, 3]
}
$insert
Use the $insert
operation to insert an item to an array.
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [1, 2, 3]
}
b.json
{
"someArray": [
{
"$insert": {
"index": 1,
"value": 4
}
}
]
}
result
{
"someArray": [1, 4, 2, 3]
}
Insert as last item
Set $insert.index
to -
to insert an item at the end of the array.
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [1, 2, 3]
}
b.json
{
"someArray": [
{
"$insert": {
"index": "-",
"value": 4
}
}
]
}
result
{
"someArray": [1, 2, 3, 4]
}
Insert before the last item
A negative $insert.index
can be used, indicating an offset from the end of the array.
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [1, 2, 3]
}
b.json
{
"someArray": [
{
"$insert": {
"index": -1,
"value": 4
}
}
]
}
result
{
"someArray": [1, 2, 4, 3]
}
$match
Use the $match
operation to search for a specific array item and merge with that item.
Match by index
Use $match.index
to match an array item by index.
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [1, 2, 3]
}
b.json
{
"someArray": [
{
"$match": {
"index": 1,
"value": 4
}
}
]
}
result
{
"someArray": [1, 4, 3]
}
Match by JSON pointer
Use $match.path
to match an array item with a JSON pointer.
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [1, 2, 3]
}
b.json
{
"someArray": [
{
"$match": {
"path": "/1",
"value": 4
}
}
]
}
result
{
"someArray": [1, 4, 3]
}
Match by JSON path query
Use $match.query
to match an array item with a JSON path query.
The following example will search for an array item containing the value 2
and merge it with the value 4
.
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [1, 2, 3]
}
b.json
{
"someArray": [
{
"$match": {
"query": "$[?(@ == 2)]",
"value": 4
}
}
]
}
result
{
"someArray": [1, 4, 3]
}
$move
Use the $move
operation to move an array item.
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [1, 2, 3]
}
b.json
{
"someArray": [
{
"$move": 1
}
]
}
result
{
"someArray": [2, 1, 3]
}
Move a matched array item
Use the $match
operation in conjunction with the $move
operation to move a specific array item.
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [1, 2, 3]
}
b.json
{
"someArray": [
{
"$match": {
"index": 0,
"value": {
"$move": 1
}
}
}
]
}
result
{
"someArray": [2, 1, 3]
}
Move a matched array item to the end
Use -
as $move.index
value to move an array item to the end.
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [1, 2, 3]
}
b.json
{
"someArray": [
{
"$match": {
"index": 0,
"value": {
"$move": "-"
}
}
}
]
}
result
{
"someArray": [2, 3, 1]
}
Move and merge a matched array item
Use $move.value
to not only move the item but also merge it with a value.
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [
{
"a": 1
},
{
"a": 2
},
{
"a": 3
}
]
}
b.json
{
"someArray": [
{
"$match": {
"query": "$[?(@.a == 3)]",
"value": {
"$move": {
"index": 0,
"value": {
"b": 3
}
}
}
}
}
]
}
result
{
"someArray": [
{
"a": 3,
"b": 3
},
{
"a": 1
},
{
"a": 2
}
]
}
$select
Use the $select
operation to select one or multiple values.
Be careful not to create an endless loop by selecting a parent property.
Select by JSON pointer
More information about JSON pointers can be found in the JSON pointer specification.
javascript
var result = jsonMerger.mergeFile("a.json");
a.json
{
"prop": {
"$select": "/otherProp"
},
"otherProp": "Should be the value of prop"
}
result
{
"prop": "Should be the value of prop",
"otherProp": "Should be the value of prop"
}
Use $select.query
to select by JSON path query
More information about JSON path queries can be found in the JSON path documentation.
javascript
var result = jsonMerger.mergeFile("a.json");
a.json
{
"prop": {
"$select": {
"query": "$.someArray[*]"
}
},
"someArray": [1, 2, 3]
}
result
{
"prop": 1,
"someArray": [1, 2, 3]
}
Use $select.multiple
to select multiple values
javascript
var result = jsonMerger.mergeFile("a.json");
a.json
{
"prop": {
"$select": {
"query": "$.someArray[?(@ < 3)]",
"multiple": true
}
},
"someArray": [1, 2, 3]
}
result
{
"prop": [1, 2]
}
Use $select.from
to select from an object
javascript
var result = jsonMerger.mergeFile("a.json");
a.json
{
"prop": {
"$select": {
"from": {
"$import": "b.json"
},
"path": "/someArray/2"
}
}
}
b.json
{
"someArray": [1, 2, 3]
}
result
{
"prop": 3
}
$repeat
Use the $repeat
operation to repeat a value.
Repeat with $repeat.to
operation
{
"$repeat": {
"from": 1,
"to": 4,
"value": "repeat"
}
}
result
["repeat", "repeat", "repeat"]
Repeat with $repeat.through
The current value is available on the scope as $repeat.value
variable.
operation
{
"$repeat": {
"from": 1,
"through": 4,
"value": {
"$expression": "$repeat.value"
}
}
}
result
[1, 2, 3, 4]
Repeat with $repeat.step
operation
{
"$repeat": {
"from": 0,
"through": 10,
"step": 5,
"value": {
"$expression": "$repeat.value"
}
}
}
result
[0, 5, 10]
Repeat with $repeat.range
operation
{
"$repeat": {
"range": "0:-2, 10, 20:30:5",
"value": {
"$expression": "$repeat.value"
}
}
}
result
[0, -1, -2, 10, 20, 25, 30]
Repeat with $repeat.in
as array
operation
{
"$repeat": {
"in": ["a", "b"],
"value": {
"$expression": "$repeat.value"
}
}
}
result
["a", "b"]
Repeat with $repeat.in
as object
The current key is available on the scope as $repeat.key
variable.
operation
{
"$repeat": {
"in": {
"keyA": "valueA",
"keyB": "valueB"
},
"value": {
"$expression": "{key: $repeat.key, value: $repeat.value}"
}
}
}
result
[
{ "key": "keyA", "value": "valueA" },
{ "key": "keyB", "value": "valueB" }
]
Getting the current index
The current index is available on the scope as $repeat.index
variable.
operation
{
"$repeat": {
"range": "1:2",
"value": {
"$expression": "$repeat.index"
}
}
}
result
[0, 1]
Nested repeat
Use $parent
to get to the parent scope containing the parent $repeat
.
operation
{
"$repeat": {
"range": "0:1",
"value": {
"$repeat": {
"range": "0:1",
"value": {
"$expression": "$parent.$repeat.index + '.' + $repeat.index"
}
}
}
}
}
result
["0.0", "0.1", "1.0", "1.1"]
$include
Use $include
to load other JSON or YAML files and process them in the current scope.
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"someArray": [1, 2, 3]
}
b.json
{
"someArray": [
{
"$include": "remove.json"
}
]
}
remove.json
{
"$remove": true
}
result
{
"someArray": [2, 3]
}
$expression
Use the $expression
operation to calculate a value with the help of a JavaScript expression.
The expression has access to the standard built-in JavaScript objects, the current scope and optionally an $input
variable.
By default this operation is disabled because it allows executing untrusted code which could introduce a security risk.
It can be enabled by setting the enableExpressionOperation
option.
Calculate a value
javascript
var result = jsonMerger.mergeFile("a.json");
a.json
{
"prop": {
"$expression": "1 + 2"
}
}
result
{
"prop": 3
}
Calculate a value using $expression.input
javascript
var result = jsonMerger.mergeFile("b.json");
a.json
{
"add": 2
}
b.json
{
"prop": {
"$expression": {
"expression": "1 + $input",
"input": {
"$import": "a.json#/add"
}
}
}
}
result
{
"prop": 3
}
Calculate a value using the scope $targetProperty
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
{
"prop": 1
}
b.json
{
"prop": {
"$expression": "$targetProperty + 2"
}
}
result
{
"prop": 3
}
Calculate a value using the scope $params
javascript
var result = jsonMerger.mergeFile("a.json", {
params: {
add: 2,
},
});
a.json
{
"prop": {
"$expression": "1 + $params.add"
}
}
result
{
"prop": 3
}
Scopes
Scopes can be created while processing operation properties.
If for example a $merge.with
is being processed then the merger will create a new scope for the $merge.with
property.
Or if a $repeat.value
property is being processed a new scope is created for the $repeat.value
property.
A scope always has a $source
property but not necessarily a $target
property.
When we are merging object A with object B, then the $target
property in the scope of object A is undefined
because object A is not merged with anything.
It does have a $source
property referring to object A itself.
Object B on the other hand has the processed object A as $target
because object B is being merged with object A.
The $source
property in the scope of object B refers to object B.
If object B had defined a $merge
operation, then the merger would create a new scope for the $merge.source
property and a new scope for the $merge.with
property.
The $target
within the $merge.source
scope would be undefined
because $merge.source
is not merged with anything.
The $target
within the $merge.with
scope is the processed $merge.source
because $merge.with
is being merged with $merge.source
.
The result of the $merge
operation will eventually be merged with object A.
When in the $merge.source
scope it is possible to get to the root (object B) scope using the $root
property or to a parent scope using the $parent
property.
interface Scope {
$params?: any; // $params properties in current scope
$parent?: Scope; // reference to parent scope
$repeat?: ScopeRepeat; // $repeat properties in current scope
$root: Scope; // reference to root scope
$source: any; // reference to the source object
$target?: any; // reference to the target object
}
Example
javascript
var result = jsonMerger.mergeFiles(["a.json", "b.json"]);
a.json
// this is the root scope
{
"prop1": {
"$expression": "$target" // refers to undefined because a.json has no target
},
"prop2": {
"$expression": "$targetProperty" // refers to undefined because a.json has no target
},
"prop3": {
"$expression": "$source" // refers to the unprocessed a.json
},
"prop4": {
"$expression": "$sourceProperty" // refers to the unprocessed a.json#/prop4
},
"prop5": {
"$expression": "$root.$target" // refers to undefined because a.json has no target
},
"prop6": {
"$expression": "$root.$source" // refers to the unprocessed a.json
},
"prop7": {
"$expression": "$parent" // refers to undefined because the current scope has no parent scope
},
}
b.json
// this is the root scope
{
"prop1": {
"$expression": "$target" // refers to the processed a.json
},
"prop2": {
"$expression": "$targetProperty" // refers to the processed a.json#/prop2
},
"prop3": {
"$expression": "$source" // refers to the unprocessed b.json
},
"prop4": {
"$expression": "$sourceProperty" // refers to the unprocessed b.json#/prop4
},
"prop5": {
"$merge": {
"source": { // $merge.source creates a new scope
"prop1": {
"$expression": "$target" // refers to undefined because b.json#/prop5/$merge/source has no target
},
"prop2": {
"$expression": "$targetProperty" // refers to undefined because b.json#/prop5/$merge/source has no target
},
"prop3": {
"$expression": "$source" // refers to the unprocessed b.json#/prop5/$merge/source
},
"prop4": {
"$expression": "$sourceProperty" // refers to the unprocessed b.json#/prop5/$merge/source/prop4
}
"prop5": {
"$expression": "$root.$target" // refers to the processed a.json
}
"prop6": {
"$expression": "$root.$source" // refers to the unprocessed b.json
},
"prop7": {
"$expression": "$parent.$target" // refers to the processed a.json
}
"prop8": {
"$expression": "$parent.$source" // refers to the unprocessed b.json
}
},
"with": { // $merge.with creates a new scope
"prop1": {
"$expression": "$target" // refers to the processed b.json#/prop5/$merge/source
},
"prop2": {
"$expression": "$targetProperty" // refers to the processed b.json#/prop5/$merge/source/prop2
},
"prop3": {
"$expression": "$source" // refers to the unprocessed b.json#/prop5/$merge/with
},
"prop4": {
"$expression": "$sourceProperty" // refers to the unprocessed b.json#/prop5/$merge/with/prop4
},
"prop5": {
"$expression": "$root.$target" // refers to the processed a.json
},
"prop6": {
"$expression": "$root.source" // refers to the unprocessed b.json
},
"prop7": {
"$expression": "$parent.$target" // refers to the processed a.json
},
"prop8": {
"$expression": "$parent.source" // refers to the unprocessed b.json
}
}
}
},
"prop6": {
"$repeat": {
"range": "0:1",
"value": { // $repeat.value creates a new scope with a $repeat property on it
"$repeat": {
"range": "0:1",
"value": { // $repeat.value creates a new scope with a $repeat property on it
"$expression": "'This is item ' + $parent.$repeat.index + '.' + $repeat.index"
}
}
}
}
}
}
Command line interface json-merger
You can use json-merger
as a command line tool:
Usage: json-merger [options] <file ...>
Options:
-V, --version output the version number
-p, --pretty pretty-print the output json
-o, --output [file] the output file. Defaults to stdout
-op, --operation-prefix [prefix] the operation prefix. Defaults to $
-am, --default-array-merge-operation [operation] the default array merge operation. Defaults to combine
--enable-expression-operation [value] enables expressions. Do not use it to run untrusted code because it uses the node:vm module. Defaults to false
--error-on-file-not-found [value] throw an error if a file is not found. Defaults to true
--error-on-ref-not-found [value] throw an error if a JSON pointer or JSON path is not found. Defaults to true
-h, --help output usage information
Usage:
json-merger a.json > result.json
json-merger --output result.json a.json
json-merger --output result.json --pretty a.json
Install json-merger
globally to be able to use the command line interface.
npm install -g json-merger
Changelog
2.0.0
- Changed emitted JavaScript target to ES6.
- The
$expression
operation is using thenode:vm
package instead of the deprecatedvm2
package. - The
$expression
operation is disabled by default because it allows executing untrusted code. - Added the
enableExpressionOperation
configuration option to enable the$expression
operation. - Added the
--enable-expression-operation
CLI option to enable the$expression
operation.
Roadmap
- Add configurable file resolvers to import files from different sources.
- Add configurable (de)serializers to import and export different file formats.