@nielsen-oss/yap-csv
v1.0.0
Published
Node.js package that manages CSV to JSON conversion supporting complex field mapping
Downloads
7
Readme
yap-csv
The purpose of this module is to support converting a csv (with delimiter support) to an array of json objects. Using a custom map that can handle both simple fields renames and more complicated fields that are a result of calculations
Usage
Add to project
yarn add yap-csv
Structure
The module exposes two variables a Parser
and ConverterConsts
.
One is used for the actual parsing process and one provide consts helpers.
Simple Usage
const {Parser} = require('yap-csv')
// parserPromise is a Promise
const parserPromise = Parser.parseSingle(FILE_NAME, FILE_LOCATION, DELIMITER, FIELDS_MAP, EXTRA_FIELDS_MAP)
parserPromise
.then((formattedData) => {
// Do what you want with formattedData
})
.catch((e) => {
console.error(e.stack)
})
Stream Usage
const {Parser} = require('yap-csv')
const formattedData = []
// parserStream is a stream
const parserStream = Parser.parseSingleStream(FILE_NAME, FILE_LOCATION, DELIMITER, FIELDS_MAP, EXTRA_FIELDS_MAP)
parserStream.on('error', (e) => {
console.error(e.stack)
})
parserStream.on('data', (chunk) => {
formattedData.push(chunk)
})
parserStream.on('end', () => {
// Do what you want with formattedData
})
Fields Specification
FILE_NAME
- The name of the file being parsedFILE_LOCATION
- The location of the file being parsedDELIMITER
- The delimiter of the textFIELDS_MAP
- A conversion map of the fieldsEXTRA_FIELDS_MAP
- A conversion map for new fields that you would like to add
FIELDS_MAP
The following is an example of a map
const convertConsts = require('yap-csv').ConverterConsts
const FIELDS_MAP = {
'FIELD1': 'simple_name_change',
'FIELD2': {
name: 'my_custom_number',
type: convertConsts.NUMBER_TYPE
},
'FIELD3': {
name: 'functional_change',
type: (dataObj) => {
return dataObj['FIELD3'].toLowerCase()
}
}
}
As you can see using the fields map you can:
- Preform simple field name changes
- Convert to the helper types that are provided in this module
- Preform conversions using a specified function that receives the original data object (before conversion)
EXTRA_FIELDS_MAP
The following is an example of a map of extra fields
const EXTRA_FIELDS_MAP = {
complicatedField: (convertedDataObj) => {
return convertedDataObj['simple_name_change']
}
}
As you can see using the fields map you can add fields that are not in the original data.
Each field should be a function that receives that data after it passed the original formatting using the FIELDS_MAP
.
ConverterConsts
The exported consts are:
NUMBER_TYPE
NUMBER_DOUBLE_TYPE
According the the types we parse the results to a parseInt
or parseFloat