Google Cloud IoT integration with balena

While the balena platform allows you to build, deploy, and manage fleets of devices, it has no interaction with the fleet's application at the data layer. This is where Google's IoT Core offering comes in handy.

Google IoT Core is a service designed to ingest data from connected devices and build rich applications that integrate with the other big data services of Google Cloud Platform. For more information visit this link.

We can harness the synergy between both technology stacks to create a very powerful end to end solution that includes everything a fleet owner might need (from infrastructure to big data needs).

This project shows you how to easily integrate your devices running balenaOS with Google IoT Core.

Installation

We've split the installation instructions into 4 different steps:

1. Create and configure Google Cloud Platform (GCP) project
2. Create balenaCloud application
3. Provision your device
4. Configure environment variables

1. Create and configure Google Cloud Platform (GCP) project

Before you begin

Before you begin you will need to create a GCP account and a billing account. The following links will help you get started with that:

• Create a Google account
• Create your billing account

Note that you don't need a credit card to get started if you opt for the free trial.

GCP setup script

The process of setting up the required GCP products is a bit involved. For your convenience we created a script that automates all of the steps needed. You can review the source code of this script here. If you are not comfortable running the script, you can check out this step by step guide, it will walk you through manually configuring all that is needed.

These are all the steps that the script performs for you:

• Request user login to GCP account
• Select an existing project or create a new one
• Link project to a billing account
• Enable required API's: compute, pubsub and cloudiotcore
• Select project region
• Create PubSub telemetry and state topics
• Create PubSub subscription for testing
• Create IAM service account
• Add roles and create keys for the service account

The script is meant to be run only once on your development machine or any non edge, non cloud device. It will interactively guide you through all the steps required.

Before running the script, make sure you have Google's SDK command-line tool installed by running gcloud --version on your terminal (if you need to, here is how to install gcloud)

To run the script:

# If you have npm installed
npm run gcp-setup

# If you don't have npm
./scripts/gcp-setup.sh

When the script execution is completed, note down the value of the following variables: GOOGLE_IOT_PROJECT, GOOGLE_IOT_REGION, GOOGLE_IOT_REGISTRY, GOOGLE_IOT_SERVICE_ACCOUNT_TOKEN. We will need to add them to our devices later.

The output of the script will look like this:

*** Export env variables ***
Setup completed, add the following env variables to your target:
GOOGLE_IOT_PROJECT=balena-gcp
GOOGLE_IOT_REGION=us-central1
GOOGLE_IOT_REGISTRY=balena-registry
GOOGLE_IOT_SERVICE_ACCOUNT_TOKEN=<YOUR_SERVICE_ACCOUNT_KEY_JSON_FORMATED>

2. Create balenaCloud application

If this is your first time using balena, we recommend going through the getting started tutorial.

You'll need to:

• Sign up for or login to the balenaCloud dashboard
• Create an application, selecting the correct device type for your hardware device (Raspberry Pi, Intel NUC, etc)
• Add a device to the application, enabling you to download the OS
• Flash the downloaded OS to your SD card with balenaEtcher
• Power up the device and check it's online in the dashboard

3. Provision your device

You're now ready to provision your balena device and push your application's code. From the project's directory run:

balena push <appName>

where <appName> is the name you gave your balenaCloud application in the previous step.

4. Configure environment variables

Once the app gets deployed to your device you'll see errors on the logs and possibly the main service will restart itself. This is because we have not configured the required application environment variables yet.

You need to create the following application environment variables and assign them the values you got from the script on step 1:

• GOOGLE_IOT_PROJECT
• GOOGLE_IOT_REGION
• GOOGLE_IOT_REGISTRY
• GOOGLE_IOT_SERVICE_ACCOUNT_TOKEN

To do so, you have two alternatives:

• Add them using the balenaCloud dashboard (more information here)
• Add them using the balena-cli by running balena env add <ENV_VAR_NAME> <ENV_VAR_VALUE> --application <appName>

Either way, once you add all environment variables, the main service will restart. Once it's started up again it'll automatically register itself with Google Cloud IoT as a device, and start pushing telemetry data to the pubsub channel you've created.

Verifying the data pipeline

Here is a quick way of verifying the data pipeline works in both ways. If you are having trouble you can reach us on our forums or open an issue on this repository.

Device to IoT Core

You can verify the telemetry data is going all the way to PubSub by tapping into the testing subscription that we created on step 1.

To do this:

• Visit the PubSub subscriptions page
• Click on your subscription, then View Messages and finally PULL

It might take a minute or so for the data to show up on the subscription.

IoT Core to device

To verify that config messages sent by IoT Core are reaching the device we can send a test message:

• Visit the IoT Core page
• Click on your registry, then go to devices and click on your device
• Click on the UPDATE CONFIG button and send a test message

You should see the message printed on your device's log.

Next steps

This project provides an easy way of getting started with the balenaOS + IoT Core combo. For more complex applications you can build your own using this sample app as the basis, or the Google samples for C, Java, NodeJS and Python available at https://cloud.google.com/iot/docs/samples/mqtt-samples

An overview of Google's cloud services that can be used to ingest, transform, and run analytics on the data is available at: https://cloud.google.com/solutions/iot-overview