npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

staticdata

v0.0.7

Published

module to convert CSV file data to JSON objects and provide structured way to access the data.

Downloads

9

Readme

Staticdata Module

A module to convert CSV file data into JSON object and provides structured methods to access the converted data.

How to install

npm install

How to include it in my project

To add this package as a dependency of your application, add the following to your package.json:

{
	"staticdata": ">= 0.0.1"
}

To use this module in your application, add the code similar to the follwoing:

var sd = require('staticdata');
var configObject = {
	path: '/path/to/my/staticdata/directory/'
};
sd.setup(configObject, function (error) {
	if (error) {
		// oops...
	}
	// now staticdata module is ready
});

Configurations

The module supports both CSV and JSON formatted files as resource data files.

With the configuration object passed to .setup(), you can modify the behavior of staticdata module.

{
	path: '/path/to/your/data/directory/',
	delimiter: '<optional>', // supported delimiters are: , ^ | ; \t
	autoUpdate: <optional> // true/false
	index: {
		// optional object to index converted data. for .getOneByIndex. .getManByIndex, and .getAllByIndexName
	},
	maxOpenFile: <optional>, // an integer to indicate how many files are allowed to stay open while converting the data. Default is 100.
}

path [string]

A path to a directory that contains all static data files (.json or .csv).

delimiter [string]

A string representation of delimiter used to parse CSV files.

The default is ,.

autoUpdate [boolean]

If set true, staticdata module will automatically update any file changes.

The default is false.

index [object]

Defines indexes to apply per static data file.

In order to use staticdata.getOneByIndex, staticdata.getManyByIndex, staticdata.getAllByIndexName, and staticdata.inflate, you must apply index(es).

Example:

var staticdata = require('staticdata');
var config = {
	path: '/my/staticdata/files/',
	index: {
		'animals.csv': [
			'id',
			'name'
		]
	}
};
// we are applying indexes to a column `id` and `name`
staticdata.setup(config, function (error) {
	if (error) {
		// error hmmm...
	}
	var animals = staticdata.create('animals');
	var dog = animals.getOneByIndex('id', '001');
	var cat = animals.getOneByIndex('name', 'Hello Kitty');
});

maxOpenFile [number]

Defines the maximum number of staticdata files to be opened while setting up.

The default is 100.

CSV

Supported CSV Delimiters

staticdata module supports the following delimiters in CSV file format:

  • Comma (,)
  • Tab (\t)
  • Semicolon (;)
  • Caret (^)
  • Vertical bar (|)

NOTE: Default delimiter used is ,.

Escaping Characters

If your CSV file contains the same characters as the delimiter, the characters must be escaped with a backslash (\).

Example with delimtier ,:

id,name,age
100,"Marley, Bob",33
101,"Harper, Ben",45

API

.create

Returns an instance of StaticData class.

You will be accessing the data from this instance.

StaticData create(String name)

Example:

/*
in order to create a static data object from a static data file called "example.csv", do the following:
*/
var staticdata = require('staticdata');
staticdata.setup(config, function (error) {
	if (error) {
		// hmmm
	}
	var example = staticdata.create('example');
});

Example With Subdirectory:

// to create a static data object from a static data file that is located in /test/another/myfile.csv, do the following:
var staitcdata = require('staticdata');
staticdata.setup(config, function (error) {
	if (error) {
		// hmmm
	}
	var myfile = staticdata.create('test/another/myfile');
});

.csvToObj

Parses CSV data into an object.

Object csvToObj(String csvData);

.arrayToCsv

Parses an array with objects into a CSV string.

String arrayToCsv(array arrayData)

Example:

var staticdata = require('staticdata');
var test = [
	{ id: 1, name: 'foo', value: 100 },
	{ id: 2, name: 'boo', value: 200 },
	{ id: 3, name: 'hello world', standard: true },
	{ id: 4, name: 'no more', standard: false, value: 0 }
];
staticdata.setup(config, function (error) {
	if (error) {
		// hmmm
	}
	var csv = staticdata.arrayToCsv(test);
	/*
	the result string:

	id,name,value,standard
	1,"foo",100,
	2,"boo",200,
	3,"hello world",,true
	4,"no more",0,false
	*/
});

.getBOM

Returns BOM string. Useful when creating a CSV file data for microsoft Excel etc...

Example:

var csvData = staticdata.arrayToCsv(arrayData);
var BOM = staticdata.getBOM();
// send the csv data with BOM to the client as a CSV file
response.send(BOM + csvData);

Auto-Update

If autoUpdate is set to true in the configurations object, staticdata module automatically updates internally cached staticdata on any existing file change.

The default is false.


StaticData Class

Methods

.getOne

Returns matched data by the CSV row index.

The second argument is an option. If given, the values of the given column only will be returned.

Object getOne(Integer index, [*Array propertiees])

.getMany

Returns a list of matched data by the CSV row indexes.

Array getMany(Array indexList, [*Array properties])

.getAll

Returns an array of all data

Array getAll([*Array properties])

.getOneByIndex

Returns a match data by the pre-indexed index key.

In order to make a use of this method, the data MUST be indexed.

For more details, please read configurations section above.

Object getOneByIndex(String indexName, String indexKey, [*Array properties])

Example:

var sd = require('staticdata');
var config = {
	path: '/path/to/my/data/',
	index: {
		'sub/example.csv': [
			'id'
		]
	}
};
sd.setup(config, function (error) {
	if (error) {
		// oops...
	}
	var example = sd.create('sub/example');
	var data = example.getOneByIndex('id', 'item001');
});

.getManyByIndex

Returns an object of matched data.

Object getManyByIndex(String indexName, Array indexKeyList, [*Array properties])

Example:

var sd = require('staticdata');
var config = {
	path: '/path/to/my/data/',
	index: {
		'sub/example.csv': [
			'id'
		]
	}
};
sd.setup(config, function (error) {
	if (error) {
		// oops...
	}
	var example = sd.create('sub/example');
	var data = example.getManyByIndex('id', ['item001', 'item002', 'item003']);
	console.log(data.item001, data.item002, data.item003);
});

.getAllByIndexName

Returns an indexed object of all data

Object getAllByIndexName(String indexName, [*Array properties])

####.inflate

Void inflate(StaticData parentStaticData, String parentKey, String childKey)

Combines 2 staticdata objects on parentKey and childKey.

parentKey is the column from source staticdata and childKey is the column from the staticdata given to the function.

NOTE: The child staticdata MUST be indexed by childKey.

Example:

/* CSV data of men.csv:
name,wife
"Bob",1
"Kevin",2
"Nathan",3

converted to:
[
	{ "name": "Bob", "wife": 1 },
	{ "name": "Kevin", "wife": 2 },
	{ "name": "Nathan", "wife": 3 },
]
CSV data of women.csv:
id,name
1,"Sandy"
2,"Olivia"
3,"Jess"

converted to:
[
	{ "id": 1, "name": "Sandy" },
	{ "id": 2, "name": "Olivia" },
	{ "id": 3, "name": "Jess" },
]
*/
// inflate the two files:
var sd = require('staticdata');
var config = {
	path: '/path/to/my/data/',
	index: {
		'women.csv': [
			'id'
		]
	}
};
sd.setup(config, function (error) {
	if (error) {
		// hmmmm
	}
	var men = sd.create('men');
	var women = sd.create('women');
	men.inflate(women, 'wife', 'id');
	/*
	Resulting data
	[
	    {
		"name": "Bob",
		"wife": {
		    "id": 1,
		    "Sandy"
		}
	    },
	    {
		"name": "Kevin",
		"wife": {
		    "id": 2,
		    "Olivia"
		}
	    },
	    {
		"name": "Nathan",
		"wife": {
		    "id": 3,
		    "Jess"
		}
	    }
	]
	*/
});