redstone-protocol

The protocol and parser the worker-client communication in redstone

Usage

import { encode, decode, createParser } from '@emiw/redstone-protocol';

// There are two sides to this module, the lower level encode/decode, and the higher level Parser class-ish.

// Encode/Decode

// Encoding
// IMPORTANT: `meta` must be JSON serializable, and data must be a buffer!
const encoded = encode({ state: 5, foo: 'bar' }, new Buffer('foo bar baz')); // encode(meta, data)
console.log(encoded); // "%7B%22state%22%3A5%2C%22foo%22%3A%22bar%22%7D:Zm9vIGJhciBiYXo=;"

// Decoding
const decoded = decode(encoded);
console.log(decoded); // "{ meta: { state: 5, foo: 'bar' }, data: <Buffer 66 6f 6f 20 62 61 72 20 62 61 7a> }"
console.log(decoded.data.toString('utf8'); // "foo bar baz"

// See below for more on the actual protocol.

// Parser

// The parser exists as a way to abstract away the process of storing chunks as the come in from a socket, extracting
// the packets, and parsing them.
const socket = getNetSocketSomehow();
const parser = createParser();
socket.on('data', parser.addChunk);

// WARNING: All of these events will be fired for every packet.
parser.on('packet', (packet) => {
  // packet = { meta: ..., data: ... }
});
parser.on('meta', (meta) => {
  // meta = ...
});
parser.on('data', (data) => {
  // data = Buffer(...)
});
socket.write(encode({ foo: 'bar' }));

// There are a few abstractions for dealing with sockets. To use them, just pass in a socket to `createParser`:
const parser = createParser(socket); // This automatically does `socket.on('data', parser.addChunk)`
parser.write(meta, data); // same as parser.write(encode(meta, data));

Protocol

The actual protocol has hastily implemented by @ariporad, and it could certainly be implemented better. But here's how it's implemented as of now:

There are three terms you need to know:

1. Packet - a single message which is sent over the network. It is made up of two sections.
2. Meta section - the first part of the packet. It contains a JSON serialized, URL encoded object (ie. encodeURIComponent(JSON.stringify(meta))).
3. Data section - Since this protocol was designed to pass through a large amount of data, this section is a base64 encoded Buffer.

Packets are in the format: meta:data;

Here's an example packet (encode({ foo: 'bar' }, new Buffer('foo')))

%7B%22foo%22%3A%22bar%22%7D:Zm9v;
^^^^^^^^^^ Meta ^^^^^^^^^^^ ^^^^ <- Data

Here's a "null" packet:

:;

For ideas/discussion about improving the protocol, see the wiki page.

License

MIT: emiw.mit-license.org.