When serializing data generated by OpenHPS, the framework will use a modified serialization strategy of TypedJSON to convert the data to JSON format. This serialized JSON data can be used for socket, MQTT and REST communication when transmitting data from client to server. In real-time situations, JSON messages might be too large to transmit from embedded devices to a server.

Protocol buffers are language-neutral and platform-neutral mechanisms for serializing structured data to lightweight buffers. Instead of encoding the structure of your data within the JSON data that is being serialized, a structured message type is defined beforehand that can be used by the serializer and deserializer. This structured message type will ensure that the data adheres to a specific structure while also prevent the need to encode this structure within the serialized data, lowering the size of the serialized data drastically.

This OpenHPS module includes the serialization and deserialization engine for Protocol Buffer Messages to OpenHPS classes and object instances. In addition, this module provides a CLI tool for generating protocol message types of all OpenHPS modules installed in your project. Combined, these two features ensure that you do not have to manually create message types and do not manually have to create a custom deserialization strategy for each class you might need to serialize.

Features

• Automatic protocol buffer message type generator
• Serialization and deserialization of all serializable OpenHPS classes

Known Limitations/Workarounds

• JavaScript only uses Number as a primitive. Protocol buffers use integers, floats, ... OpenHPS includes the NumberType enum option for SerializableMember decorators. When no number type is specified a number will be serialized to a string to ensure compatibility.
• Polymorphism and inheritance is not possible in protocol buffer v3. In order to ensure maintainable messages, any serializable object with known types will be treated as a google.protobuf.Any type with an included type_url that can be used for deserialization.
• Objects that use generic types will also make use of google.protobuf.Any. However, generic types that include primitives will be serialized using a primitive wrapper.

Usage

CLI Generator

This module can generate the protocol files (*.proto) automatically.

1. Install the module using npm install @openhps/protobuf@latest
2. Open the root directory of your project containing your package.json file
3. Execute the CLI command using openhps-protobuf

Parameters

-d <directory> Output directory
-v Verbose logging

Output Data

The output directory will include a subdirectory for each OpenHPS package.

Serialization and Deserialization

First the protocol buffer serializer has to be initialized with the messages.

import { ProtobufSerializer } from '@openhps/protobuf';

// Initialize the protocol buffer serializer with the protocol files
ProtobufSerializer.initialize("/home/openhps/protobuf/");

// Or

// Initialize the protocol buffer serialization by creating protocol messages in the ./tmp directory
ProtobufSerializer.initialize();

An example protocol buffer message type can be seen below for a DataObject. The absolute and relative positions are set to an Any type.

package openhps.core;
syntax = "proto3";
import "google/protobuf/any.proto";
message DataObject {
	string displayName = 1;
	int64 createdTimestamp = 2;
	string uid = 3;
	string parentUID = 4;
	google.protobuf.Any position = 5;
	repeated google.protobuf.Any relativePositions = 6;
}

package openhps.core;
syntax = "proto3";
import "Velocity.proto";
import "Orientation.proto";
import "google/protobuf/any.proto";
message Absolute3DPosition {
	string timestamp = 1;
	Velocity velocity = 2;
	Orientation orientation = 3;
	google.protobuf.Any unit = 4;
	string referenceSpaceUID = 5;
	google.protobuf.Any accuracy = 6;
	string probability = 7;
	double x = 8;
	double y = 9;
	double z = 10;
}

@openhps/socket

ModelBuilder.create()
	.addService(new SocketServer({
		srv: server,
		path: "/api/v1"
	}))
	.from(new SocketServerSource({
		uid: "source",
		// Override serializer and deserializer with protocol buffer
		serialize: (obj) => ProtobufSerializer.serialize(obj),
		deserialize: (obj) => ProtobufSerializer.deserialize(obj)
	}))
	.to()
	.build();

@openhps/mqtt

ModelBuilder.create()
	.addService(new MQTTServer({
		port: 1443,
	}))
	.from(new MQTTSourceNode({
		uid: "source",
		// Override frame serializer (not the options)
		serialize: (obj, options) => ({
			frame: ProtobufSerializer.serialize(obj),
			options
		}),
		deserialize: (obj) => ProtobufSerializer.deserialize(obj.frame)
	}))
	.to()
	.build();

Getting Started

If you have npm installed, start using @openhps/protobuf with the following command.

npm install @openhps/protobuf --save

Contributors

The framework is open source and is mainly developed by PhD Student Maxim Van de Wynckel as part of his research towards Hybrid Positioning and Implicit Human-Computer Interaction under the supervision of Prof. Dr. Beat Signer.

Contributing

Use of OpenHPS, contributions and feedback is highly appreciated. Please read our contributing guidelines for more information.

License

Copyright (C) 2019-2024 Maxim Van de Wynckel & Vrije Universiteit Brussel

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.