Pareto Anywhere

Pareto Anywhere is the open source IoT middleware that makes sense of who/what is where/how in an application/technology/vendor-agnostic way.

Pareto Anywhere provides a single standard stream of real-time data, in the form of developer-friendly JSON, regardless of the underlying radio-identifiable devices & technologies.

pareto-anywhere is a lightweight Node.js package that can run on resource-constrained edge devices as well as on powerful cloud servers and anything in between (i.e. Anywhere). The bundled web apps, APIs and integrations offer a friendly and comprehensive introduction to context-aware physical spaces. And, thanks to a modular, open architecture, pareto-anywhere can easily be customised for scale and/or functionality.

Learn more at www.reelyactive.com/pareto/anywhere/

Getting Started

Follow our step-by-step tutorials to get started with Pareto Anywhere on your platform of choice:

• Run Pareto Anywhere on a personal computer
• Run Pareto Anywhere on a Raspberry Pi

Learn "owl" about the raddec, dynamb and spatem JSON data output:

• reelyActive Developer's Cheatsheet

Quick(est) start

Install (or update) the pareto-anywhere package globally with the command:

npm install -g pareto-anywhere

Run at any time with the command:

pareto-anywhere

Browse to localhost:3001 to see Pareto Anywhere running, and to explore the web apps and APIs.

This installation affords limited configuration through environment variables. For full configuration capability, instead follow the (Recommended) Quick start below.

(Recommended) Quick start

Clone this repository and its submodules, and install dependencies with the following commands:

git clone --recurse-submodules https://github.com/reelyactive/pareto-anywhere.git
cd pareto-anywhere
npm install

Then from the pareto-anywhere folder, run at any time with the command:

npm start

Browse to localhost:3001 to see Pareto Anywhere running, and to explore the web apps and APIs.

This installation affords full configuration through custom startup scripts and environment variables. Consult our Create a Pareto Anywhere startup script tutorial to learn more, and our Run Pareto Anywhere as a Windows service tutorial for running in production on Windows machines.

Updating

An existing installation can be updated by running the following commands from the root of the cloned repository:

git fetch
git pull origin master --recurse-submodules
npm install

What is pareto-anywhere?

pareto-anywhere is a convenient collection of various open source software modules by reelyActive which, together, enable context-aware physical spaces.

These open source software modules are hosted each in their own individual repository as follows:

| Software module | Role | |:----------------------------------------------------------|:-----| | barnowl | Interface for gateways, APs and readers | | barnacles | Efficient data aggregator/distributor | | barterer | Real-time location & sensor data API | | chickadee | Contextual associations store and API | | chimps | Spatial dynamics processor | | advlib | Library for wireless advertising packet decoding | | sniffypedia | Index for implicit URI assocation | | json-silo | Digital twins store |

pareto-anywhere is designed to serve as a friendly launch point for developers and end-users alike. Create a Pareto Anywhere startup script to add and customise modules, or, if necessary, edit the lib/paretoanywhere.js file directly, to remove and customise modules.

Input: Ambient Data

pareto-anywhere listens for ambient data from multiple sources, using the following open source software modules:

| Software module | Listens for | |:--------------------------------------------------------------|:------------| | barnowl | UDP raddec packets on port 50001 | | barnowl-aruba | WebSocket on /aruba | | barnowl-minew | HTTP POST on /minew | | barnowl-reel | UDP reel packets on port 50000 | | barnowl-huawei | UDP Huawei packets on port 50010 | | barnowl-impinj | HTTP POST on /impinj | | barnowl-rfcontrols | STOMP over WebSocket | | barnowl-csl | HTTP POST on /csl | | barnowl-axis | MQTT on localhost |

Create a Pareto Anywhere startup script to add and customise input modules, or forward raddec data as UDP packets simply by running the npm run forwarder script of the corresponding barnowl-x module, such as barnowl-hci.

Output: Hyperlocal Context

pareto-anywhere outputs hyperlocal context which may be queried via the /context API, or streamed as raddec, dynamb and spatem JSON data. These JSON data structures are explained in the reelyActive Developers Cheatsheet.

The pareto-anywhere APIs are served by the following open source software modules:

| Software module | API | |:------------------------------------------------------|:-------------| | barterer | /devices | | chickadee | /context & /associations & /features | | json-silo | /stories |

Output logfiles

To capture raddec and dynamb data into .csv files, from the root folder of this repository, start Pareto Anywhere with the command:

npm run logfile

Files named raddec-yymmdd-hhmmss.csv and dynamb-yymmdd-hhmmss.csv will appear in the root folder when data is processed, rotating periodically, and which can be opened in a spreadsheet for analysis.

Config Files

The /config folder, or a distinct folder specified by the PARETO_ANYWHERE_CONFIG_PATH environment variable, accepts the following run-time configuration files:

certificate.pem & key.pem

On startup, pareto-anywhere attempts to initiate a HTTPS server using these files, if present. If not, a HTTP server is instantiated instead.

Note that ambient data sources which relay data to pareto-anywhere should observe the corresponding protocol (http:// vs. https:// and ws:// vs. wss://) in their target configuration.

enocean.json

EnOcean Alliance devices require a mapping between their device identifier and their EnOcean Equipment Profile (EEP) in order for their payload to be decoded by advlib-esp and its sub-libraries. The /config/enocean.json file has the following format:

{
  "05174fc4/7": { "eepType": "A5-04-03" },
  "0591ee96/7": { "eepType": "D5-00-01" },
  "04141559/7": { "eepType": "D2-14-41" },
  "002eea1f/7": { "eepType": "F6-02-02" }
}

Note that the 32-bit device identifier is specified in lowercase hexadecimal and suffixed with "/7" which represents the EURID-32 idType, while the eepType is specified in UPPERCASE hexadecimal.

Native integrations

Pareto Anywhere integrates with just about any data store, stream processor or application either via existing APIs or the addition of a connector module. Additionally, the following integrations are natively supported by pareto-anywhere.

InfluxDB v2

From the root folder of this repository, start Pareto Anywhere with the command:

npm run influxdb2

If prompted, install barnacles-influxdb2 with the command npm install barnacles-influxdb2. pareto-anywhere will automatically write dynamb data to a local InfluxDB v2 database, if installed and running.

The local InfluxDB v2 instance is expected at http://localhost:8086 with org "reelyActive" and bucket "pareto-anywhere". Set INFLUXDB_TOKEN as an environment variable.

Elastic Stack

From the root folder of this repository, start Pareto Anywhere with the command:

npm run elasticsearch

If prompted, install barnacles-elasticsearch with the command npm install barnacles-elasticsearch. pareto-anywhere will automatically write to a local Elasticsearch database, if installed and running, so that the raddec and dynamb data may in turn be analysed in Kibana.

• pareto-anywhere v1.8 and above expect an Elasticsearch 8.x instance, offering nominal backwards-compatiblility with 7.x
• pareto-anywhere v1.7 and below require an Elasticsearch 7.x instance

The local Elasticsearch instance is expected at http://localhost:9200 and, if a Kibana instance is also running, the data will be available for analysis at http://localhost:5601.

To instead have pareto-anywhere store data in a remote Elasticsearch database, first set the environment variable ELASTICSEARCH_NODE to the corresponding URL, including the corresponding username and password, for example:

https://username:[email protected]:9243

Node-RED

The @reelyactive/node-red-pareto-anywhere Flow includes a pareto-anywhere-socketio Node that will automatically connect to pareto-anywhere's Socket.IO API running at localhost:3001, and will stream the raddec, dynamb and spatem JSON data to application logic in real-time.

Build and run with Docker

For users who prefer deployment using containers, first build the Docker image from the root folder of this repository:

docker build -t pareto-anywhere:core .

Then start the container:

docker run -d \
-p 3001:3001/tcp \
-p 50000:5000/udp \
-p 50001:50001/udp \
--restart unless-stopped \
--name pareto-anywhere \
pareto-anywhere:core

Browse to localhost:3001 to see Pareto Anywhere running and browse the APIs.

Run Pareto Anywhere, Elastisearch and Kibana with Docker-Compose

For users who prefer deployment using containers, Deploy Elasticsearch, Kibana and Pareto Anywhere from the root folder of this repository:

docker-compose up -d

Browse to localhost:3001 to see Pareto Anywhere running and browse the APIs. Browse to localhost:5601 to see Kibana dashboard running. Browse to localhost:9200 to see Elasticsearch running and browse the APIs.

Developer Notes

The pareto-anywhere codebase includes Pareto Apps as a git submodule (in the web/apps folder). To update to the latest Pareto Apps run:

git submodule update --remote

If developing Pareto Apps from within this repository, changes can be pushed to the Pareto Apps origin (pareto-anywhere-develop branch) by the usual means, from the web/apps folder:

cd web/apps
git checkout pareto-anywhere-develop
...update files...
git add [...]
git commit -m "Description of update"
git push origin pareto-anywhere-develop

Pareto Anywhere variants

Pareto Anywhere for Azure is adapted to run efficiently in the cloud as a stateless Azure Function. Learn more here.

Pareto Anywhere for the Web is adapted to run in-browser, leveraging the experimental Web Bluetooth Scanning feature, introduced to the Chrome browser in late 2019. Try it here.

Project History

In 2019, Pareto Anywhere was created to supersede hlc-server, our open source Hyperlocal Context Server, which was itself created in 2014.

pareto-anywhere v1.5.0 adopts chickadee v1.4.0 which migrates to ESMapDB from NeDB. If upgrading from a previous version, any stored associations will need to be recreated.

pareto-anywhere v1.13.0 removes explicit database dependencies (i.e. Elasticsearch), instead allowing the database to be specified at runtime (ex: npm run elasticsearch or npm run influxdb2).

Learn more about the Pareto by reelyActive collection at www.reelyactive.com/pareto

Contributing

Discover how to contribute to this open source project which upholds a standard code of conduct.

Security

Consult our security policy for best practices using this open source software and to report vulnerabilities.

License

MIT License

Copyright (c) 2020-2024 reelyActive

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.