nxus-data-manager

Import file contents as arrays of objects. Includes parsers and exporters for:

• CSV / TSV
• JSON
• GeoJSON / ArcJSON

Provides convenience integration with nxus-storage to import and save to a storage model.

Installation

npm install nxus-data-manager --save

Options

All responses accept an options argument, in addition to any parser-specific options you can indicate:

• mapping: object of {field: newField} name mappings
• identityFields: for importing to models, array of fields to be used for createOrUpdate query
• truncate: for importing to models, true/false for deleting existing collection data before import. Ignored if identityFields is provided.
• strict: defaults to true. Only import columns/data that matches the attribute names for the model. Set to false to import everything.

Events

You can modify the records during import with the following specific events:

• records.type: e.g. dataManager.after('records.csv', (results) => {})
• record.type: e.g. dataManager.after('record.csv', (one) => {})
• models.identity: e.g. dataMangaer.after('models.user', (results) => {})
• model.identity: e.g. dataManager.after('model.user', (user) => {})

record_ events occur after parsing and name mapping model_ events occur after record events and before models are created/updated.

API

parser

Provide a parser for a particular type (file extension)

Parameters

• type string The type (e.g. 'html') this renderer should handle
• handler function Function to receive (content, options) and return parsed array of result objects

exporter

Provide an exporter for a particular type (file extension)

Parameters

• type string The type (e.g. 'html') this exporter creates
• handler function Function to receive (content, options) and return formatted output content

export

Request formattted output based on type

Parameters

• type string The type (e.g. 'html') of the output content
• records [object] The records to export
• opts object Options for the exporter context

Returns Promise String of formatted output

import

Request parsed results based on type

Parameters

• type string The type (e.g. 'html') of the content
• content string The contents to parse
• opts object Options for the parser context

Returns Promise Array of parsed result objects

importFile

Request parsed results from a file path

Parameters

• filename string The filename to read and parse
• opts object Options for the parser context

Returns Promise Array of parsed result objects

importToModel

Import string contents to a model

Parameters

• model string The identity of the model to populate
• type string The type (e.g. 'html') of the content
• content string The contents to parse
• opts object Options for the parser context

Returns Promise Array of instances

importFileToModel

Import file contents to a model

Parameters

• model string The identity of the model to populate
• filename string The filename to read and parse
• opts object Options for the parser context

Returns Promise Array of instances

fixture

Import a data file as fixture data. Can specify environment option to only load for e.g. test

Parameters

• modelId string The identity of the model to import
• path string The path to a file
• options object Options to pass to data-loader.importFile

API

src/index.js:32-32

Import file contents as arrays of objects

Installation

npm install nxus-data-loader --save

Options

All responses accept an options argument, in addition to any parser-specific options you can indicate:

• mapping: object of {field: newField} name mappings
• identityFields: for importing to models, array of fields to be used for createOrUpdate query
• truncate: for importing to models, true/false for deleting existing collection data before import. Ignored if identityFields is provided.
• strict: defaults to true. Only import columns/data that matches the attribute names for the model. Set to false to import everything.

Events

You can modify the records during import with the following specific events:

• records.type: e.g. app.get('data-loader').after('records.csv', (results) => {})
• record.type: e.g. app.get('data-loader).after('record.csv', (one) => {})
• models.identity: e.g. app.get('data-loader').after('models.user', (results) => {})
• model.identity: e.g. app.get('data-loader).after('model.user', (user) => {})

record_ events occur after parsing and name mapping model_ events occur after record events and before models are created/updated.

API

parser

src/index.js:87-91

Provide a parser for a particular type (file extension)

Parameters

• type string The type (e.g. 'html') this renderer should handle
• handler function Function to receive (content, options) and return parsed array of result objects

exporter

src/index.js:97-99

Provide an exporter for a particular type (file extension)

Parameters

• type string The type (e.g. 'html') this exporter creates
• handler function Function to receive (content, options) and return formatted output content

export

src/index.js:113-117

Request formattted output based on type

Parameters

• type string The type (e.g. 'html') of the output content
• records [object] The records to export
• opts object Options for the exporter context

Returns Promise String of formatted output

import

src/index.js:126-133

Request parsed results based on type

Parameters

• type string The type (e.g. 'html') of the content
• content string The contents to parse
• opts object Options for the parser context

Returns Promise Array of parsed result objects

importFile

src/index.js:141-150

Request parsed results from a file path

Parameters

• filename string The filename to read and parse
• opts object Options for the parser context

Returns Promise Array of parsed result objects

importToModel

src/index.js:160-164

Import string contents to a model

Parameters

• model string The identity of the model to populate
• type string The type (e.g. 'html') of the content
• content string The contents to parse
• opts object Options for the parser context

Returns Promise Array of instances

importFileToModel

src/index.js:173-177

Import file contents to a model

Parameters

• model string The identity of the model to populate
• filename string The filename to read and parse
• opts object Options for the parser context

Returns Promise Array of instances

_storeResultsWithModel

src/index.js:245-250

Simple do-storage function to help mock up storage for tests, other applications.

Parameters

• model [type] [description]
• values [type] [description]
• uniqueCriteria [type] [description]

Returns [type] [description]

export

src/JSONExporter.js:14-21

Stringify an array of results into JSON. Assumes top-level is array, unless opts.key is provided to wrap results in an object.

Parameters

• records array
• options object
• opts

parse

src/JSONParser.js:14-20

Parse JSON into an array of results. Assumes top-level is array, unless opts.key is provided to pick a top-level key from parsed object as results.

Parameters

• contents string
• options object
• opts