@oceanstream/cli

About

OceanStream is an integrated cloud data platform used for hydroacoustic data collection and analysis in real-time. It is built on Azure IoT Edge infrastructure.

OceanStream CLI

Install from NPM with: (Add -g to install globally):

npm i @oceanstream/cli

Usage

Run the tool without any command-line arguments to see a top-level commands description.

npx oceanstream

or shorter: npx os

Output will be:

Usage: oceanstream [options] [command]

Operations on OceanStream Edge modules

Options:
  --version / -V                       output the version number
  --help / -h                          display help for command

Commands:
  create                               Initialize a new OceanStream project
  delete                               Delete an OceanStream project
  app                                  Operations on Azure applications
  env                                  Manage project environments with different settings and configurations
  device                               Operations on OceanStream edge devices
  module                               Manage OceanStream Edge modules for specific tasks on devices
  set                                  Set current environment, container registry, and device
  help                                 display help for command

You can also see help on any of top-level commands, for example: oceanstream app help.

Concepts

The CLI fullfills the following roles:

• manage the IoT Edge related infrastructure in Azure which is used by the OceanStream platform
• emulate IoT Edge devices using Azure VMs or local Docker containers
• build IoT Edge modules and deploy them on IoT Edge devices

Application is a representation of an Azure App registration. Consists of tenantId, clientId, clientSecret.

Environment is a predefined set of Azure infrastructure entities, which allows to operate on such set in a more convenient way. Currently environment consists of appName (reference to a specific application), subscriptionId, location, resourceGroupName, iotHubName.

Device is an Azure IoT Edge device.

Module is an Azure IoT Edge device-compatible module which essential wraps a Docker container.

Azure setup

First of all, you'll need to create an Azure app registration and assign it proper permissions. This is the only step which requires interacting with Azure web console (TODO: simplify using CLI commands). In Azure, create a new app registration. Create a new client secret for an app registration (Certificates & secrets section), save the Client Secret ID somewhere safe. Also make a note of Application (client) ID and Directory (tenant) ID (can be seen in an Overview section). If you don't have an active subscription, you also need to create one. Go to the subscription, open Access control (IAM) section, go to Roles and create a custom role named IoT Contributor using the following permissions:

{
  "actions": [
    "*",
    "Microsoft.Storage/storageAccounts/blobServices/containers/delete",
    "Microsoft.Storage/storageAccounts/blobServices/containers/read",
    "Microsoft.Storage/storageAccounts/blobServices/containers/write",
    "Microsoft.Storage/storageAccounts/blobServices/generateUserDelegationKey/action"
  ],
  "notActions": [
    "Microsoft.Authorization/*/Delete",
    "Microsoft.Authorization/*/Write",
    "Microsoft.Authorization/elevateAccess/Action",
    "Microsoft.Blueprint/blueprintAssignments/write",
    "Microsoft.Blueprint/blueprintAssignments/delete",
    "Microsoft.Compute/galleries/share/action"
  ],
  "dataActions": [
    "Microsoft.Devices/IotHubs/*",
    "Microsoft.Devices/IotHubs/*/read",
    "Microsoft.Devices/IotHubs/fileUpload/notifications/action",
    "Microsoft.Devices/IotHubs/devices/*",
    "Microsoft.Devices/IotHubs/twins/*",
    "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/delete",
    "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read",
    "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/write",
    "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/move/action",
    "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/add/action"
  ],
  "notDataActions": []
}

Assign this role to the previously created app registration.

Initial project setup

To create a new project, use the oceanstream create command. If you're already in a directory with a package.json file, the tool will use the existing folder and the project name from the package.json file. If you're not in a directory with a package.json file, you'll be prompted to enter a project name, and a new sub-directory with the provided name will be created in the current directory.

Environment

Now, create a new environment by running oceanstream env add. You will need to enter the environment name, select an application to use with an environment, select Azure Subscription, pick the location for the resources, create new or select existing Azure resource group, create new or select existing Azure IoT Hub (to reduce the costs, if it's the only IoT Hub you have, upon creation you can select F1 tier and capacity of 1). Note that if IoT Hub with such name exists anywhere globally, you will need to pick a different name. To see the list of created environments you can use oceanstream env ls.

IoT Device

To create an IoT Device, just run oceanstream device add. Select an environment and enter the device name. To create a device emulator in the cloud, use oceanstream device emulate. Select an environment, pick the device you want to emulate, create new or select an existing security group (should allow SSH connection), create new or select an existing virtual network, pick the names for the subnet / public IP / network interface, enter the VM name, VM administrator name and the administrator password. After the VM has been created, the SSH command to connect to it will be displayed in the console. The VM might not be available right way as it usually requires a few minutes to be fully provisioned.

Modules

To create an IoT Edge module, use oceanstream module create. Pick the name of the module and select the language to use. To build the module, use oceanstream module build. You will need to select an environment and for each of the following resource select existing or create new: container registry (to store image in), storage account (to upload module sources to), storage container (to store module sources in). After that, you'll need to select which module you want to build, pick the OS & architecture to build an image for and enter the image tag.

Deploy modules on IoT Device

To apply the modules on the IoT Device, you need to create the manifest first. In a manifests subdirectory of a project, create a <manifestName>.yaml file with the following contents (just an example):

modules:
  jsmodule:
    tag: 'v1'
  csmodule:
    tag: 'v2'

Here the jsmodule and csmodule are the names of the modules and tag is a tag to use when building the Docker image.

Now, execute the oceanstream device apply. Select the environment, IoT device to apply modules to, a container registry to pull modules from and the device manifest.

Installation

To install the CLI, follow these steps:

1. Navigate to the project folder if you already have it, or download the project.
2. Install npm packages by running the command: npm install
3. Create a .env file and specify the environment variable: NODE_ENV=development
4. Execute the command: npm link

After completing these steps, try running oceanstream help to see a list of available commands and ensure the installation was successful.

Commands

Project

create

Initialize a new IoT Edge project.

oceanstream create

| Parameter | Alias | Description | | :-------- | :---- | :----------- | | --name | -n | Project name |

Example:

If you want to create a new IoT Edge project in a new sub-directory, use the following command:

oceanstream create -n my-iot-project

Or

oceanstream create

You will be prompted to enter the project name:

Enter the project name: my-iot-project

After successful execution, you will receive the following output:

Creating the project my-iot-project

Now you can see your project folder in the directory from which the create command was executed.

oceanstream create

You will be prompted to enter the project name:

Enter the project name: my-iot-project

After successful execution, you will receive the following output:

Creating the project my-iot-project

Now you can see your project folder in the directory from which the create command was executed.

If you want to create a new IoT Edge project in the current directory using the project name from the package.json file, use the following command:

oceanstream create

After successful execution, you will receive the following output:

Creating the project <project-name-from-package.json>

Now your current directory has been initialized as an IoT Edge project.

delete

Delete an existing IoT Edge project.

oceanstream delete

This command will list all available IoT Edge projects in the current working directory and prompt you to select one to delete.

Example:

To delete an existing IoT Edge project, use the following command:

oceanstream delete

You will be prompted to select a project from the list of available projects:

? Select a project to delete: (Use arrow keys)
❯ my-iot-project
  another-iot-project

After selecting the project and confirming the deletion, you will receive the following output:

Deleting the project my-iot-project

The selected project folder will be removed from the current working directory.

You can also delete the IoT Edge project located in the current directory.

Example:

oceanstream delete

You will be prompted to select a project to delete. Choose "Current Directory":

? Select a project to delete: Current Directory

After confirming the deletion, you will receive the following output:

Deleting the project Current Directory
Project Current Directory deleted.

The current directory, which is the selected project folder, will be removed along with all its contents.

Note: Be cautious when using this command, as it will permanently delete the chosen project directory and all its contents. Make sure you have backed up any important data before proceeding.

App

The app command is used for managing Azure application credentials. To use this command, you need to navigate to your project's directory:

cd <project_name>

Replace <project_name> with the name of your project folder.

add

Add new Azure application credentials.

oceanstream app add

Parameters:

• Application name: name of the Azure application
• Directory (tenant) ID: the tenant ID for your Azure Active Directory
• Application (client) ID: the client ID for your Azure application
• Client secret: the client secret for your Azure application

Example:

cd my-iot-project

then

oceanstream app add

This command will prompt you for the required information:

? Application name name-of-app
? Directory (tenant) ID my-id
? Application (client) ID my-client
? Client secret [hidden] enter your secret here

Fill in the details as prompted.

list

List Azure application credentials.

oceanstream app list

or

oceanstream app ls

Example:

cd my-iot-project
oceanstream app ls

Output:

name-of-app
  Directory (tenant) ID: my-id
  Application (client) ID: my-client

remove

Remove an Azure application from the configuration file.

oceanstream app remove

or

oceanstream app rm

Example:

cd my-iot-project
oceanstream app remove

Output:

? Select application to remove from the configuration file (Use arrow keys)
❯ name-of-app

This command will prompt you to select the Azure application you want to remove from the configuration file.

Env

The env command is used for managing environments in your project. To use this command, you need to navigate to your project's directory:

cd <project_name>

Replace <project_name> with the name of your project folder.

add

Add a new environment.

oceanstream env add

Prompts:

Environment name

• Select the application to use with the environment
• Azure Subscription ID
• Select the Azure location to create resources within the environment
• Resource group
• IoT Hub name
• Container registry (will be used to store module container images)
• Storage account (will be used to store module sources)
• Storage container (will be used to store module sources)

Example:

oceanstream env add

Output:

? Environment name my-env
? Select the application to use with the environment my-app
✔ Connected to Azure API
? Azure Subscription ID my-sub
? Select the Azure location to create resources within the environment westeurope
? Resource group my-resource-group
? IoT Hub my-env-iot-hub
? Container registry (will be used to store module container images) myenviotcr
? Storage account (will be used to store module sources) myenviotsa
? Storage container (will be used to store module sources) Create new
? Storage container name my-env-iot-sc
✔ Storage container my-env-iot-sc has been created
Environment my-env has been added

After executing the oceanstream env add command, you will be prompted to provide the necessary information to set up a new environment, including the name of the environment, the Azure application to use, the subscription ID, the location to create resources within the environment, and other configuration options. Once you have provided all the required information, the environment will be created and added to the configuration file. You can then use the oceanstream env list command to see a list of all the environments currently configured for your project.

list

The env list command is used to display a list of all environments that have been added to your project. To use this command, navigate to your project's directory:

cd <project_name>

Then, run the env list command:

oceanstream env ls

This will display a list of all the environments that have been added to your project, along with their associated configuration details, such as the application used with the environment, the subscription ID, resource group, IoT hub, container registry, and storage account.

For example:

my-env
  Application: my-app
  Subscription ID: sub-id
  Resource group: my-resource-group
  IoT Hub: my-env-iot-hub
  Container registry: myenviotcr
  Storage account: myenviotsa
  Storage container: my-env-iot-sc

This can be useful for quickly reviewing the configuration details of each environment and ensuring that they are correct.

remove

To remove an environment, you can use the rm or remove command. This command will remove the environment from the configuration file.

For example:

oceanstream env rm

After running this command, you will be prompted to select the environment to remove from the configuration file.

Please note that removing an environment will not delete the resources created in Azure. If you want to delete the Azure resources associated with an environment, you will need to do so manually using the Azure portal or Azure CLI.

Device

The device command is used to manage Azure IoT Edge devices. Before using this command, make sure you navigate to your project's directory:

cd <project_name>

Replace <project_name> with the name of your project folder.

add

Add a new (or existing) Azure IoT Edge device.

| Parameter | Alias | Description | | :-------------- | :---- | :------------------------------------------- | | --environment | -e | Select the environment to create a device in | | --device-name | -n | Name of the IoT Edge device |

Example:

If you want to add a new Azure IoT Edge device to a specific environment and provide the device name with the command, use the following command:

oceanstream device add -e my-environment -n my-iot-device

If you want to add a new Azure IoT Edge device and provide the environment and device name interactively, use the following command without the --environment and --device-name options:

oceanstream device add

You will be prompted to select the environment to create a device in, and enter a name for the device. Once the device is successfully created, you will see a message confirming that the IoT Edge device has been created:

 IoT Edge Device <device-name> has been created

emulate

Emulate an IoT Edge device.

Options: | Option | Alias | Description | |---------------------------|-------|-----------------------------------------------| | --environment | -e | Select the environment to emulate | | --strategy | -s | Select the strategy to emulate: [Cloud, Local]| | --device | -n | Name of the IoT Edge device | | --security-group | -g | Name of the security group | | --virtual-network-name | -v | Name of the virtual network | | --public-ip | -p | Name of the public IP | | --domain-name | -d | Domain name | | --network-interface-name | -i | Name of the network interface | | --virtual-machine | -m | Name of the virtual machine | | --username | -u | Name of the admin | | --password | -w | Password of the username |

Example:

To emulate a device with specific settings, run the following command:

oceanstream device emulate -e my-environment -n my-iot-device -s Cloud|Local -g my-security-group -v my-virtual-network -p my-public-ip -d my-domain-name -i my-network-interface -m my-virtual-machine -u my-admin-username -w my-admin-password

To emulate a device interactively select options, run the following command without providing additional options:

oceanstream device emulate

To emulate a device, run the device emulate command and select the environment where you want to emulate the device. The command will connect to the Azure API and prompt you to select the device you want to emulate. Then, you will be prompted to choose an emulation strategy. You can choose to emulate the device locally or in the cloud using an Azure virtual machine.

If you choose to emulate the device in the cloud, the command will prompt you to select a security group to enable SSH access to the virtual machine and a virtual network to use for the virtual machine. Then, you will be prompted to enter an administrator username and password for the virtual machine. Once the virtual machine is created, you will be provided with instructions on how to connect to it using SSH.

apply

The device apply command is used to apply module configurations to an Azure IoT Edge device. Before using this command, you should already have created an IoT Edge device and modules to apply to it.

Options:

| Option | Alias | Description | | ------------- | ----- | --------------------------------------------- | | --environment | -e | Select the environment to apply the config to | | --device | -n | Name of the IoT Edge device | | --registry | -r | Select the registry to apply | | --manifest | -m | Select the manifest to apply |

First, create a new manifest file in the manifests directory of your project:

cd my-iot-project
mkdir manifests
cd manifests
touch example-manifest.yaml

Then, open the example-manifest.yaml file and add the content(just example):

modules:
  tmt:
    tag: "tag-name"

Save and close the file.

Finally, run the oceanstream device apply command to apply the modules configuration to the IoT device:

oceanstream device apply [options]

Here's an example of using the oceanstream device apply command along with the contents of a manifest file:

First, create a new manifest file in the manifests directory of your project:

Copy code
cd my-iot-project
mkdir manifests
cd manifests
touch example-manifest.yaml

Then, open the example-manifest.yaml file and add the following content:

Copy code

modules:
  tmt:
    tag: 'v1'

Save and close the file.

Finally, run the oceanstream device apply command to apply the modules configuration to the IoT device:

oceanstream device apply

You will be prompted to select the environment to deploy the modules in and to choose the IoT device from the IoT Hub to apply the modules configuration to. You will also need to select the container registry to pull modules from and the device manifest to apply. Once you have made these selections, the modules from the manifest will be applied to the IoT device.

Module

The module command is used for managing Azure IoT Edge modules. You can perform various operations with this command such as adding, listing, removing, and building modules.

add

To add a new IoT Edge module, use the following command:

Options:

| Option | Alias | Description | | ------------- | ----- | ------------------------------------------------ | | --module-name | -n | Name of the IoT Edge module | | --language | -l | Language of the IoT Edge module. [Cs, Js, Py, C] | | --source | -s | Source of the IoT Edge module | | --path | -p | Path of the IoT Edge module. For local modules | | --username | -u | Github username. For Github modules | | --repo | -r | Github repo name. For Github modules | | --token | -t | Github token. For Github modules |

oceanstream module add [options]

or

oceanstream module add

Then you will be prompted to provide the following information:

The name of the module: the name of the new module you want to create.

Module's programming language: the programming language used to write the module code.

Module's source: the source of the module, which can be one of the following options:

New module in a project directory: a new module will be created in your current project directory. Existing module in a local filesystem directory: the module will be imported from a local directory. Existing module in a GitHub repository: the module will be imported from a GitHub repository. After providing the required information, the new module will be created and added to your project.

For example, if you enter the command oceanstream module add, and then provide the name "mtm", select the programming language "C#", and choose to create a new module in a project directory, the output will be:

Created new module mtm

This means that a new module named "mtm" has been created in your project directory.

list

List all the IoT Edge modules in the project.

oceanstream module list

or

oceanstream module ls

remove

Remove an IoT Edge module from the project.

Options:

| Option | Alias | Description | | ------ | ----- | --------------------------- | | --name | -n | Name of the IoT Edge module |

oceanstream module remove

or

oceanstream module rm

During the execution of the command, if the --name option is not provided, you will be asked to select the module to remove from the project. After the selection or if the --name option is provided, the specified module will be removed from the project.

build

The module build command is used for building an IoT Edge module for a specific environment.

Options:

| Option | Alias | Description | | -------------- | ----- | ------------------------------------ | | --environment | -e | Select the environment to build in | | --module | -m | Select the module to build | | --os | -o | Select the OS to build for | | --architecture | -a | Select the architecture to build for | | --tag | -t | Select the tag to build |

oceanstream module build

Once you're in the project directory, you can run the module build command to build a module for a specific environment. If options are not provided, this command will prompt you to select the environment, module, operating system, architecture, and image tag for the module you want to build.

After you've answered all of the prompts or provided options, the command will build the module and create a container with the specified tag.

Set

Command for use current configuration

set

Set the current environment, registry, and device for the project.

oceanstream set

Options:

| Option | Alias | Description | | :----------- | :---- | :-------------------------- | | --env | -e | Set the current environment | | --registry | -r | Set the current registry | | --device | -d | Set the current device |

Example:

To set the current environment, registry, and device, use the following command:

oceanstream set --env production --registry my-registry --device my-device

After successful execution, you will receive the following output:

Environment production is set as current
Registry my-registry is set as current
Device my-device is set as current
[WARNING]: The specified environment, registry, and device will be automatically applied for future actions. If not specified, the current settings will be used. To change them, simply provide the new values again.

To set only the current environment, use the following command:

oceanstream set --env production

After successful execution, you will receive the following output:

Environment production is set as current
[WARNING]: The specified environment, registry, and device will be automatically applied for future actions. If not specified, the current settings will be used. To change them, simply provide the new values again.

Similarly, you can set only the current registry or device by using the --registry or --device option respectively.

Unit tests

Unit tests for the code base were written in Nightwatch using the @nightwatch/apitesting plugin. More info on API testing with Nightwatch.

Run tests with:

npm test

Licence

MIT