@valentech/sializer
v0.3.7
Published
[![Codacy Badge](https://app.codacy.com/project/badge/Grade/61f392c96c49481ba4c7d3f109db2fdc)](https://www.codacy.com/gh/pouya-eghbali/sia/dashboard?utm_source=github.com&utm_medium=referral&utm_content=pouya-eghbali/sia&utm_campaign=Badge_Grade) [![code
Downloads
47
Readme
What is new
This is a fork from the original Sia. Sia enables binary de-/serialization of arbitrary javascript data.
- Fixed a bug with Map serialization
- typed Arrays support added
- Objects of primitive Types (Boolean/Number/String) support added
- Object null prototype support added
- BigInt support added
- Circular datastructures support added (Map/Set/Array/Object can be self referencing)
- Autogrowing buffers (this degraded performance, but Sia is still faster than JSON)
- Replaced utfz-lib with @valentech/utfz-lib
- Handles RegExp and Date internally now with more space efficiency
In short: Sia can now handle all Javascript datatypes and won't overflow if the input is large.
Sia
Sia - Binary serialisation and deserialisation with built-in compression. You can consider Sia a strongly typed, statically typed domain specific binary language for constructing data. Sia preserves data types and supports custom ones.
Please note the Sia specification and implementation isn't final yet. As a core part of Clio programming language, Sia evolves with Clio. It is made to make fast RPC calls possible.
Why
I needed a fast schema-less serialization library that preserves type info and is able to code/decode custom types. I couldn't find one. At first I wanted to go with a JSON with types solution but it didn't work out, so I created my own.
Performance
This repository contains a pure JS implementation of Sia, on our test data Sia is 66% to 1250% faster than JSON and serialized data (including type information for all entries) is 10% to 30% smaller than JSON. Sia is faster and smaller than MessagePack and CBOR/CBOR-X. It is possible to use lz4 to compress Sia generated data even more and still be faster than JSON, MessagePack and CBOR-X.
Tests are run on a 2.4 GHz 8-Core Intel Core i9-9980HK CPU (5 GHz while running the benchmarks)
with 64 GB 2667 MHz DDR4 RAM. Node version 16.4.2, Mac OS 11.5.1. 100 loops each serialization library.
To run the benchmark suite you can run npm run benchmark
and to run the tests you can run npm run test
.
Specification
Read specs.md.
Install
To install the Node library and save it as a dependency, do:
npm i -S @valentech/sializer
// or
yarn add @valentech/sializer
Documentation
WIP
The Node Sia library exports 5 items:
const { sia, desia, Sia, DeSia, constructors } = require("sializer");
sia(data)
function serializes the given data using the default parameters.desia(buf)
function deserializes the given buffer using the default parameters.Sia(options)
class makes an instance of Sia serializer using the given options.DeSia(options)
class makes an instance of Sia deserializer using the given options.siaCompressed(data)
function serializes the given data using the default parameters + lz4 compression.desiaCompressed(buf)
function deserializes the given buffer using the default parameters + lz4 compression.SiaCompressed(options)
class makes an instance of Sia serializer using the given options + lz4 compression.DeSiaCompressed(options)
class makes an instance of Sia deserializer using the given options + lz4 compression.constructors
is an array of default constructors used both by Sia and DeSia.
The Sia
and DeSia
objects are the core components of the Sia library.
Basic usage
const { sia, desia } = require("sializer");
const buf = sia(data);
const result = desia(buf);
Sia class
const sia = new Sia({
size = 33554432, // Buffer size to use
constructors = builtinConstructors // An array of extra classes and types
});
const buf = sia.serialize(data);
Where size
is the maximum size of buffer to use, use a big size if you're expecting to
serialize huge objects. The constructors
option is an array of extra types and classes,
it includes instructions for serializing the custom types and classes.
DeSia class
const desia = new DeSia({
mapSize = 256 * 1000, // String map size
constructors = builtinConstructors, // An array of extra classes and types
});
const data = desia.deserialize(buf);
Where mapSize
is the minimum size of string map array to use, use a big size if you're
expecting to serialize huge objects. The constructors
option is an array of extra types
and classes, it includes instructions for deserializing the custom types and classes.
sia function
const buf = sia(data);
The sia
function is the Sia.serialize
method on an instance initialized with the default options.
desia function
const data = desia(buf);
The desia
function is the DeSia.deserialize
method on an instance initialized with the default options.
constructors
The constructors
option is an array of extra types and classes that Sia should support.
Here's an example of how to use it:
const { Sia, DeSia } = require("sializer");
const { constructors: builtins } = require("sializer");
const constructors = [
...builtins,
{
constructor: RegExp, // The custom class you want to support
code: 7, // A unique positive code point for this class, the smaller the better
args: (item) => [item.source, item.flags], // A function to serialize the instances of the class
build(source, flags) { // A function for restoring instances of the class
return new RegExp(source, flags);
},
},
];
const sia = new Sia({ constructors });
const desia = new DeSia({ constructors });
const regex = /[0-9]+/;
const buf = sia.serialize(regex); // serialize the data
const result = desia.deserialize(buf); // deserialize