npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@tum-far/ubii-msg-formats

v0.16.6

Published

Hi. This is the repository for Ubii message formats. It contains everything about **what** ubii nodes and devices say to each other. If you are interested in how they communicate with each other, please see - https://zeromq.org/, https://zguide.zeromq.org

Downloads

39

Readme

ubii-msg-formats

Hi. This is the repository for Ubii message formats. It contains everything about what ubii nodes and devices say to each other. If you are interested in how they communicate with each other, please see

  • https://zeromq.org/, https://zguide.zeromq.org/docs/chapter3/#The-REQ-to-REP-Combination, https://zguide.zeromq.org/docs/chapter3/#The-DEALER-to-ROUTER-Combination
  • https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods, https://developer.mozilla.org/en-US/docs/Web/API/WebSocket

Table of Contents

Compilation

  • required: python >=3.5
  • required: 'protoc' compiler =3.11.1. If compiler is not installed/available in system path, get the release matching your OS from https://github.com/protocolbuffers/protobuf/releases/tag/v3.11.1 and copy the executable (/bin/protoc) to ubii-msg-formats/external/bin.
  • run 'node scripts/full-compile.js' to compile all parts (if python3 executable not automatically found path needs to be edited manually in script)
  • to compile just specific parts:
    • run 'python scripts/compile.py --opt [language]' where [language] is one of py/python, j/java, js/javascript, cs/csharp, all
    • run 'node scripts/generate-protobuf-static.js'
  • to build the custom python package (using proto-plus wrappers instead of default python types) install a python build backend (e.g. build), then simply build the package from the repository root (e.g. with installed build backend, run python -m build in the top level directory containing the pyproject.toml file)

Contribution

  • don't commit /dist for feature branches. you can test compilation, just don't commit the compiled files. You may want to use $GIT_DIR/info/exclude functionality to avoid unintentional commits.

CLIs

  • Run npm test to process all standard tests.
  • If snapshot assertions fail you can run npm run-script update-snapshots to process all standard tests and update all snapshots if the changes are intentional.

Ubii Messages

Note: The messages are based on google's Protobuf. Check out the Protobuf section below for more information on how to handle these types of messages. You can find some more information on the proto files itself describing the structure of the messages as well as sections on how to work with proto messages in relevant envrionments such as nodeJS.

There are three first order messages:

  • topicData
  • serviceRequest
  • serviceReply

All ubii related sub messages sent over the network are wrapped up in one of the first order messages. A first order message has submessages that contains further data that should be processed. Typically this is sepcified by a type oneof field.

Check out the oneof section below for how you can get the specified submessage by accessing the oneof properties.

The proto file of the first order messages imports all submessages. Thus the proto instance or a translator instance of a first order message can encode and decode all related message buffers.

TopicData

This message is used to communicate topic data from the client to the server and from the server to the client. It is used in a router-dealer / websocket communication pattern context.

Beside general information this message contains a topic data pair. The TopicData can contain the following data structures: https://github.com/SandroWeber/ubii-msg-formats/blob/develop/src/proto/topicData/topicDataRecord/topicDataRecord.proto

You can send a TopicData message to a master node in order to publish topic data pairs to a master node. In return, master nodes will send TopicData messages to registered devices to inform them about changes of their subscribed topics.

ServiceRequest

This message is used to formulate service request. It is sent from the client to the server. The message is used in a request-reply / HTTP(S) request communication pattern context.

The ServiceRequest can have the following submessages specifing the type of the reply: https://github.com/SandroWeber/ubii-msg-formats/blob/develop/src/proto/services/serviceRequest.proto

ServiceReply

This message is used by the server to formulate service replies to previous service requests from clients. The message is used in a request-reply / HTTP(S) request communication pattern context.

The ServiceReply can have the following submessages specifing the type of the reply: https://github.com/SandroWeber/ubii-msg-formats/blob/develop/src/proto/services/serviceReply.proto

Protobuf

The messages are based on google's Protobuf. The following section contains some topics that are super relevant for this project.

See the Protobuf Documentation and the used protobuf.js Github repository of a JS implementaion for more details.

Loading Proto Files

The proto files can be loaded synchronously or asynchronously. The asynchronous method should be prefered. However it is only available in a nodeJS environment.

Proto Files

Required and Validation

According to Protocol Buffers documentation, the required field rule is not recommended:

Required Is Forever You should be very careful about marking fields as required. If at some point you wish to stop writing or sending a required field, it will be problematic to change the field to an optional field – old readers will consider messages without this field to be incomplete and may reject or drop them unintentionally. You should consider writing application-specific custom validation routines for your buffers instead. Some engineers at Google have come to the conclusion that using required does more harm than good; they prefer to use only optional and repeated. However, this view is not universal. --Protobuf Documentation

OneOf

The use of oneofs in JS is very poorly documented. Therefore we have collected some snippets how you can use them in JS (and how they are used in this project). If you have some improvements, please update the appropriate code snippets and update this section.

  • Specify oneof as documented in the protobuf documentation. Example:
oneof avatar {
    string image_url = 1;
    bytes image_data = 2;
}
  • You can then acces the currently set oneof type at runtime via message['oneof specifier']. Example:
let currentOneofType = message.avatar;

The message['oneof specifier'] is a property of the proto Message prototype. It is not visible if the message is converted to a plain js object or toString() is used to convert it to a string.

  • You can then access the explicit value via message[message['oneof specifier']]. Example:
let currentOneofValue = message[message.avatar];

Usage of Proto Files in JS

This repository provides messageTranslator classes. These classes provide all relevant functionalities to work with the proto messages.

  • The main aspect is on a validated translating between the three proto message states: buffer <-> message <-> payload

Flatbuffers

https://google.github.io/flatbuffers/index.html

Compilation

Release version used to compile schema files: v1.11.0

Testing

This module uses the AVA test runner.

You can add new tests to the test folder. Entry point for the test runner is test.js within the test folder. See the AVA Documentation for more details on how to create new test cases for AVA.

Tips

There are extensions availabe for Visual Studio Code that support Proto3 files (Sytntax highlighting, syntax validation, ...). Just search for ext:proto in the marketplace.