nv-buf-serde
v0.0.9
Published
nv-buf-serde ======================= - rewrite v8/src/objects/value-serializer.cc to JS - to let you use nodejs'v8.serialize AND nodejs'v8.deserialize in browser, - similiar to structuredClone AND MessageChannel-transfer, but can be used to send/recv
Downloads
10
Readme
nv-buf-serde
rewrite v8/src/objects/value-serializer.cc to JS
to let you use nodejs'v8.serialize AND nodejs'v8.deserialize in browser,
similiar to structuredClone AND MessageChannel-transfer, but can be used to send/recv data to server-side
only work in 【BROWSER】 for transfering-with-remote-server:
coz for local-transfer structuredClone is best(json'speed is best,but only support simple data-type);
for 【server-side-remote-transfer】in nodejs you can use v8.serialize AND v8.deserialize directly;
this is 【SLOW】, coz it is wrote in pure-js.
only suited for 【SMALL-data such as complicated start-config】 with map/set/circular/array-buffer.... ,which only be transfered when app starting for one-time.
see UNSUPPORTED below for unsupported data-type
for most data-types,if data is small, its a little faster than v8.serialize/v8.deserilize in nodejs. coz it did NOT do any check/verify/validate
but for string , its 10X slower , coz it read the string char-by-char to code-point.
for not-small size array/object, its 5X slower.
see SIMPLE-PERF below
install
- npm install nv-buf-serde
usage
in browser
//copy the dist.js in ./DIST to your browser
const { ser,der } = v8serde;
for test in node
const { ser,der } = require("nv-buf-serde");example
ser(o:Any, version:Uint = 15) : ArrayBuffer; // version default is 15
// if server use node16 ,try 13.
// IF server nodejs ver < 16, this pkg maybe NOT work.
der(ab:ArrayBuffer) : <JSValue> // when decode, this pkg will NOT check version. const {deepStrictEqual} = require("assert");
var st = new Set([])
for(let e of [1,1.1,undefined,null,true,false,"abcd","aÿ我𝑒",12345678901234567890n,[],{}]) {st.add(e)}
var mp = new Map();
let dict = {a:100,b:-0}
let ary = [0,1,2,3]
for(let e of [[{},[]],[true,false],[dict,dict],[ary,ary]]) {mp.set(e,e)}
for(let e of [[]]) {mp.set(e,e)}
for(let e of [1,1.1,undefined,null,true,false,"abcd","aÿ我𝑒",12345678901234567890n]) {mp.set(e,e)}
var circular = [];
circular[0] = circular
circular[1] = {pr:circular[0]}
circular[2] = mp
st.add(mp);
mp.set("set",st);
/*
> circular
<ref *1> [
[Circular *1],
{ pr: [Circular *1] },
<ref *2> Map(15) {
[ {}, [] ] => [ {}, [] ],
[ true, false ] => [ true, false ],
[ [Object], [Object] ] => [ [Object], [Object] ],
[ [Array], [Array] ] => [ [Array], [Array] ],
[] => [],
1 => 1,
1.1 => 1.1,
undefined => undefined,
null => null,
true => true,
false => false,
'abcd' => 'abcd',
'aÿ我𝑒' => 'aÿ我𝑒',
12345678901234567890n => 12345678901234567890n,
'set' => Set(12) {
1,
1.1,
undefined,
null,
true,
false,
'abcd',
'aÿ我𝑒',
12345678901234567890n,
[],
{},
[Circular *2]
}
}
]
>
*/
> var dump = ser(circular);
/*
> dump
ArrayBuffer {
[Uint8Contents]: <ff 0f 41 03 5e 00 6f 22 02 70 72 5e 00 7b 01 3b 41 02 6f 7b 00 41 00 24 00 00 24 00 02 5e 03 41 02 54 46 24 00 02 5e 06 41 02 6f 22 01 61 49 c8 01 22 01 62 4e 00 00 00 00 00 00 00 80 7b 02 5e 08 24 00 02 5e 07 41 02 41 04 49 00 49 02 49 04 49 06 24 00 04 5e 0a 24 00 02 5e 09 41 00 24 00 00 5e 0b 49 ... 153 more bytes>,
byteLength: 253
}
>
*/
var dupe = der(dump);
assert.deepStrictEqual(circular,dupe)
client send to server :
// on client: ws.send(ser(obj)) ...
// on server: let obj = v8.deserialize(<encoded>)server send to client:
// on server: ws.send(v8.serialize(obj)) ...
// on client: let obj = der(<encoded>) TEST
cd ./TEST ./run.sh
f0 AND ff0 is v8.serialize/v8.deserialize
f1 AND ff1 is pure-js ser/der w-1bstr.js
{ rounds: 100000, f: [Function: f0], costed: 266.1992090046406 }
{ rounds: 100000, f: [Function: f1], costed: 1591.1110320091248 }
w-2bstr.js
{ rounds: 100000, f: [Function: f0], costed: 228.52056899666786 }
{ rounds: 100000, f: [Function: f1], costed: 660.8281089961529 }
w-bi.js
{ rounds: 1000000, f: [Function: f0], costed: 1849.7178589999676 }
{ rounds: 1000000, f: [Function: f1], costed: 1643.9760229885578 }
w-double.js
{ rounds: 1, f: [Function: ff0], costed: 2050.589771002531 }
{ rounds: 1, f: [Function: ff1], costed: 1856.7536469995975 }
w-int-not-smi.js
{ rounds: 1, f: [Function: ff0], costed: 2139.468289002776 }
{ rounds: 1, f: [Function: ff1], costed: 1751.2041779905558 }
w-odd-ball.js
{ rounds: 1000000, f: [Function: f0], costed: 1801.9907419979572 }
{ rounds: 1000000, f: [Function: f1], costed: 742.1094769984484 }
{ rounds: 1000000, f: [Function: f0], costed: 1820.1971789896488 }
{ rounds: 1000000, f: [Function: f1], costed: 748.5948320031166 }
{ rounds: 1000000, f: [Function: f0], costed: 1815.4723690003157 }
{ rounds: 1000000, f: [Function: f1], costed: 760.0095049887896 }
{ rounds: 1000000, f: [Function: f0], costed: 1806.1364599913359 }
{ rounds: 1000000, f: [Function: f1], costed: 730.0119130015373 }
w-packed-double.js
<Buffer ff 0f 41 03 4e 00 00 00 00 00 00 f0 bf 4e 00 00 00 00 00 00 00 80 4e 00 00 00 00 00 00 f0 3f 24 00 03>
<Buffer ff 0f 41 03 4e 00 00 00 00 00 00 f0 bf 4e 00 00 00 00 00 00 00 80 4e 00 00 00 00 00 00 f0 3f 24 00 03>
{ rounds: 1, f: [Function: ff0], costed: 2115.820453003049 }
{ rounds: 1, f: [Function: ff1], costed: 1731.7947219908237 }
w-packed-smi.js
<Buffer ff 0f 41 03 49 01 49 00 49 02 24 00 03>
<Buffer ff 0f 41 03 49 01 49 00 49 02 24 00 03>
{ rounds: 1, f: [Function: ff0], costed: 1007.4265009909868 }
{ rounds: 1, f: [Function: ff1], costed: 389.41917300224304 }
w-prim-wrap.js
<Buffer ff 0f 73 00 63 0a 61 00 ff 00 11 62 35 d8 52 dc> <Buffer ff 0f 73 00 63 0a 61 00 ff 00 11 62 35 d8 52 dc>
{ rounds: 100000, f: [Function: ff0], costed: 1601.2434429973364 }
{ rounds: 100000, f: [Function: ff1], costed: 638.7041679918766 }
w-smi.js
{ rounds: 1, f: [Function: ff0], costed: 2105.5412980020046 }
{ rounds: 1, f: [Function: ff1], costed: 950.825115993619 }
w-packed-with-attr.js
<Buffer ff 0f 41 0b 49 02 4e 9a 99 99 99 99 99 f1 3f 5f 30 54 46 22 04 61 62 63 64 00 63 0a 61 00 ff 00 11 62 35 d8 52 dc 5a 10 d2 0a 1f eb 8c a9 54 ab 41 00 ... 25 more bytes>
<Buffer ff 0f 41 0b 49 02 4e 9a 99 99 99 99 99 f1 3f 5f 30 54 46 22 04 61 62 63 64 00 63 0a 61 00 ff 00 11 62 35 d8 52 dc 5a 10 d2 0a 1f eb 8c a9 54 ab 41 00 ... 25 more bytes>
{ rounds: 1, f: [Function: ff0], costed: 1437.0307759940624 }
{ rounds: 1, f: [Function: ff1], costed: 740.9776969999075 }
w-ab-and-abvw.js
{ rounds: 1, f: [Function: ff0], costed: 3125.2701260000467 }
{ rounds: 1, f: [Function: ff1], costed: 1463.7843720018864 }
w-date.js
{ rounds: 100000, f: [Function: f0], costed: 378.10796700417995 }
{ rounds: 100000, f: [Function: f1], costed: 218.29990500211716 }
w-rgx.js
{ rounds: 100000, f: [Function: f0], costed: 360.65081399679184 }
{ rounds: 100000, f: [Function: f1], costed: 196.77194899320602 }
r-1bstr.js
OK
{ rounds: 1, f: [Function: ff0], costed: 622.7416110038757 }
{ rounds: 1, f: [Function: ff1], costed: 271.9350669980049 }
r-2bstr.js
OK
{ rounds: 1, f: [Function: ff0], costed: 136.93560498952866 }
{ rounds: 1, f: [Function: ff1], costed: 146.48304900527 }
r-bi.js
OK
{ rounds: 1, f: [Function: ff0], costed: 1521.1638139933348 }
{ rounds: 1, f: [Function: ff1], costed: 560.5311820060015 }
r-double.js
OK
{ rounds: 1, f: [Function: ff0], costed: 1761.6175539940596 }
{ rounds: 1, f: [Function: ff1], costed: 296.7176159918308 }
r-int-not-smi.js
OK
{ rounds: 1000000, f: [Function: ff0], costed: 2367.8616740107536 }
{ rounds: 1000000, f: [Function: ff1], costed: 451.3351549953222 }
r-odd-ball.js
OK
{ rounds: 1, f: [Function: ff0], costed: 1346.4026799947023 }
{ rounds: 1, f: [Function: ff1], costed: 240.1754889935255 }
r-packed-double.js
OK
{ rounds: 20, f: [Function: ff0], costed: 3246.5816289931536 }
{ rounds: 20, f: [Function: ff1], costed: 3989.6508100032806 }
r-packed-smi.js
OK
{ rounds: 1000000, f: [Function: ff0], costed: 1818.279228001833 }
{ rounds: 1000000, f: [Function: ff1], costed: 2230.154618009925 }
r-prim-wrap.js
OK
{ rounds: 1000000, f: [Function: ff0], costed: 7234.687916994095 }
{ rounds: 1000000, f: [Function: ff1], costed: 3035.835222005844 }
r-smi.js
OK
{ rounds: 1000000, f: [Function: ff0], costed: 6417.46975800395 }
{ rounds: 1000000, f: [Function: ff1], costed: 1456.2603430002928 }
r-packed-with-attr.js
OK
{ rounds: 20, f: [Function: ff0], costed: 1868.1501239985228 }
{ rounds: 20, f: [Function: ff1], costed: 7413.920781999826 }
r-ab-and-abvw.js
OK
{ rounds: 1000000, f: [Function: ff0], costed: 30922.769218996167 }
{ rounds: 1000000, f: [Function: ff1], costed: 22780.244821995497 }
r-date.js
OK
{ rounds: 1000000, f: [Function: ff0], costed: 2954.3600009977818 }
{ rounds: 1000000, f: [Function: ff1], costed: 1190.4288749992847 }
r-rgx.js
OK
{ rounds: 1000000, f: [Function: ff0], costed: 3780.0919840037823 }
{ rounds: 1000000, f: [Function: ff1], costed: 2295.3174780011177 }
r-mp-st-circular.js
OK
OK
OK
OK
OK
{ rounds: 2000, f: [Function: ff0], costed: 350.00537499785423 }
{ rounds: 2000, f: [Function: ff1], costed: 1388.989602997899 }
w-err.js
{ rounds: 1000000, f: [Function: f0], costed: 4555.216290995479 }
{ rounds: 1000000, f: [Function: f1], costed: 9781.851919993758 }
{ rounds: 1000000, f: [Function: f0], costed: 4957.54928599298 }
{ rounds: 1000000, f: [Function: f1], costed: 10075.762607008219 }
{ rounds: 1000000, f: [Function: f0], costed: 4710.144057005644 }
{ rounds: 1000000, f: [Function: f1], costed: 10048.256901994348 }
{ rounds: 1000000, f: [Function: f0], costed: 4756.645067989826 }
{ rounds: 1000000, f: [Function: f1], costed: 10162.962930992246 }
{ rounds: 1000000, f: [Function: f0], costed: 4752.813621997833 }
{ rounds: 1000000, f: [Function: f1], costed: 10042.401304006577 }
{ rounds: 1000000, f: [Function: f0], costed: 4700.770161986351 }
{ rounds: 1000000, f: [Function: f1], costed: 10009.027477994561 }
r-err.js
OK
{ rounds: 1, f: [Function: ff0], costed: 1471.0474539995193 }
{ rounds: 1, f: [Function: ff1], costed: 207.64260099828243 }METHODS
- too many, most are USELESS , just use ser/der AND refer to const is OK.
APIS
{
////-----------------------------------------------------------
fixed_cfg : _fixed_cfg, // v8flag compatible to nodejs
restrict : _restrict, // unsupported data-type readme
const : _const, // tag AND subtag(for array-buffer-view) definition
misc : _misc, // some int AND str util
zero_nid : _zero_nid, // v8 impl require ref-id from 0
ctx : _ctx, // context (for handle circular reference)
w : _w, // encode methods
r : _r, // decode methods
////----------------------------------------------------------------------
ser, // serialize USED on sender-side
der, // deserialize USED on recver-side
}RESTRICT
RESTRICT and UNSUPPORTED
const restrict = require("nv-buf-serde").restrict;
console.dir(restrict,{depth:null})
/*
{
////----- roughly support
kTheHole : `
the hole element will be replaced by undefined,coz in js-layer:
1. when serialize: the-hole-element can only be founded using .foreach, and that is expensive.
2. when deser : need use trick like { var a=[]; a[8] = 9;} to make the-hole, also expensive.
`,
kSparseArray : `
NOT support. only dense/packed_smi/packed_double.
hard to explain this, refer to ./TEST/hole-tst.js
this script will make sparse-array
`,
kSharedArrayBuffer : `
treated as normal ArrayBuffer,
although v8 has the code for this routing, but it failed when i test,
dont known how to find shared-array-buffer-id
`,
kArrayBufferView : `
0. array-buffer-view will be treated as kHostObject,for compatible with nodejes delegate(although its implement MAYBE wrong),
event if several array-buf-views refer to the same buffer, still be DEEP-COPIED:
var ab = new ArrayBuffer(4);
var u8a = new Uint8Array(ab);
var u16a = new Uint16Array(ab);
var ary = [ab,u8a,u16a];
var dupe_ary = der(ser(ary));
/*
1. dupe_ary[0]
2. dupe_ary[1].buffer
3. dupe_ary[2].buffer
[1. 2. 3.] are different buffer, this means:
the abview NOT support keeping the ref-of-<.buffer>-relations
*/
`,
JSMap : "extra attributes set On Map will be dropped, for compatible to current v8-impl",
JSSet : "extra attributes set On Set will be dropped, for compatible to current v8-impl",
RegExp : "extra attributes set On RegExp will be dropped, for compatible to current v8-impl",
Date : "extra attributes set On Date will be dropped, for compatible to current v8-impl",
PrimitiveWrapper : `
extra attributes set On PrimitiveWrapper will be dropped(treat-as-leaf), for compatible to current v8-impl;
not support <new BigInt>, in c++ layer ,it can be done. in js-layer ,impossible
`,
////------partly support:
"Error'message" : "Error'message treated as string",
"Error'stack" : "Error'stack treated as string",
////----- unsupported:
kVerifyObjectCount : "no verify",
throwDetachedArrayBufferError : "dont know how to get this state in js-layer",
kArrayBufferTransfer : "hard to do this in js-layer",
kSharedObject : 'hard to do this in js-layer',
kWasmModuleTransfer : 'hard to do this in js-layer',
kWasmMemoryTransfer : 'hard to do this in js-layer',
ITERATOR : "dont know how to get the iter cursor in js-layer, treated as a {}",
ATOMICS : "dont know how to do this, treated as a {}",
FPG : "function | lambda | promise| generator| all treated as a {}",
Proxy : "hard to do this in browser, so treated as the-target-be-proxied",
}
*/APPENDIX
this pkg JUST for test backword-serde in nvlang. NOT for production.
70% syntax in nvlang is same as JS :
(es3 + await + Destructuring-assignment + getter-setter-only-class(similar to pod struct)) + a little TS(just-used-as-comments)
the lefted 30% is a graph-dsl.LICENSE
- ISC