Weather service implementation

This service implements basic dlake services to demonstrate both service call and service UI injection.

Native process

When using native process execution, take care not to use already in use port. In order to avoid that, you can start your process using a startup shell script as following:

#!/bin/sh

export IIOS_SERVER_PORT=20013
export IIOS_NAMESPACE=ignitialio

node index.js

Redis

Ignitial.io services are based on Redis for service discovery and PUB/SUB RPC emulation.

You need to start a Redis server to proceed. For example:

docker run -d --name redis -p 6379:6379 redis

Docker

Configuration file must be based on ENV variables in order to easily configure Docker container execution.

Build

docker build --rm --force-rm -t ignitial/dlake .

Run

docker run -d -p 20013:20013 --name dlake --link redis:redis -e REDIS_HOST="redis" -e IIOS_SERVER_PORT=20013 ignitial/dlake

Kill

If Docker image does not contain PM2 and node app is not started using pm2-node, then take care when stopiing Docker container to send TERM signal to the service. Indeed, TERM signal will allow the service to clean up and unregister from Redis discovery dictionary.

docker exec dlake pkill -TERM node

Usage

DLake sevice aims to provide data services to any other IIO service or application.

Main concepts

Notice:
We are mainly targetting NoSQL databases (currently only MongoDB is implemented), but same ceoncepts could be implemented for a relational database as well.

For any MongoDB collection we need to implement a datum, which is simply a wrapping class providing at least basic CRUD operations (augmented with some sugar). These operations are already implemented in the Item class, which is in fact using engine specifics wrapping that within a IIO standardized API. For MongoDB, implementation can be found in lib/db/item-mongo.js file.

Basic operations

Basic operation available are:

• find(args, userId): equivalent to collection.find. Arguments are passed through an args object that is either a query (in this case a MongoDB query), or an object containing a query and an option field:

  mydatum.find({ _id: myID }, userId).then(docs => {
    // do something with docs
  }).catch(err => {})  
  
  // OR
  
  mydatum.find({ 
    query: { _id: myID }, 
    options: { projection: { fieldOne: 1 }}
  }, userId).then(docs => {
    // do something with docs
  }).catch(err => {})  

As you can see, one additional parameter to args is userId which is used to implement access control.

• findAndSort(args, userId): similar to previous, excepting that is paginated and sorted. args parameter needs then to contain sort, page, and pageSize fields to define corresponding options.
• findPaginated(args, userId): similar, but only focused on pagination:

  mydatum.find({ 
    query: { _id: myID }, 
    page: 1,
    pageSize: 30,
    options: { projection: { fieldOne: 1 }}
  }, userId).then(docs => {
    // do something with docs
  }).catch(err => {})    

• get(args, userId): returns one single item from a collection.

  mydatum.get({ _id: myID }, userId).then(doc => {
    // do something with doc
  }).catch(err => {})    
  
  // OR
  
  mydatum.get({ 
    query: { _id: myID }, 
    options: { projection: { fieldOne: 1 }}
  }, userId).then(doc => {
    // do something with doc
  }).catch(err => {})  

Notice:
_id can be a string that will be converted automatically to ObjecID when using Mongo.

• put(args, userId): inserts or updates an element defined by args object. If args provides an _id field, item is supposed to be updated (if corresponding item is found), unless is inserted. Updates are possible only if datum options appendOnly is not set to true.

  // update
  mydatum.put({ 
    _id: myID,
    fieldOne: '...',
    // ... 
  }, userId).then(() => {
    console.log('done')
  }).catch(err => {})    
  
  // OR insert
  mydatum.put({ 
    fieldOne: '...',
    // ..
  }, userId).then(result => {
    console.log(result._id)
  }).catch(err => {})  

As seen above, in the case of an insert, you can get back the item _id field for further use.

• putsert(args, userId): similar, excepting that is using updateOne with insert option set (specific to Mongo)
• del(args, userId): deletes one item defined by args query.

Notice:
All the methods are returning a Promise.

Default datums (collections)

There are several collections defined by default:

• users: users list for user management
• roles: user roles for access control
• connections: logs users connections for reporting
• activities: logs users activity for reporting..
• notifications: users application notifications

Deploy

In order to deploy DLake service you can either use a derivated class creating your own dedicated service, or deploy it as a Docker container providing to it your additional datum definitions.

Indeed, configuration file (config.index.js) provides a field that allows to define the directory where to look for datum definition files:

  {
    /* datum definition pathes */
    datum: {
      paths: [ './lib/datum', '/opt/dlake/datum' ]
    },
    // ...
  }

By default there is /opt/dlake/datum which corresponds to service deployment directory when build for docker (see Dockerfile). Then you can declare for example a volume link in order to target your definition directory.

Observing the configuration file, you can see as well any other environment variable that you can use for service tuning. An example of deployment command can be found below:

#!/bin/sh

docker run -d --name dlake-ottoman \
  -p 20091:20091 \
  -e DLAKE_NAME=dlake-ottoman \
  -e REDIS_HOST=redis \
  -e MONGODB_URI=mongodb://mongo:27017 \
  -e MONGODB_DBNAME=ottoman \
  -e IIOS_NAMESPACE=ottoman \
  -e IIOS_SERVER_PORT=20091 \
  -v ${pwd}/datum:/opt/dlake/datum \
  --link mongo:mongo \
  --link redis:redis \
  ignitial/dlake

When deploying in production, you can use Docker volume containers to share datum defintions between a source and the dlake service.

docker volume create --name datum-defs

then

#!/bin/sh

docker run -d --name dlake-ottoman \
  -p 20091:20091 \
  -e DLAKE_NAME=dlake-ottoman \
  -e REDIS_HOST=redis \
  -e MONGODB_URI=mongodb://mongo:27017 \
  -e MONGODB_DBNAME=ottoman \
  -e IIOS_NAMESPACE=ottoman \
  -e IIOS_SERVER_PORT=20091 \
  -v datum-defs:/opt/dlake/datum \
  --link mongo:mongo \
  --link redis:redis \
  ignitial/dlake