City of Things Client Library

The City of Things client library will help you to connect to the City of Things backend. It uses the Web Linking RFC 5988 and interprets the Link header for you. This way you can let the library follow the next/prev/last pages for you.

New major release v5

DataLayer API is added and the old code has been restructured and optimized for minimal code duplication.

New features added:

• DataLayer API
  • cityIds are unique identifiers for a city with a certain detail.
  • Ask for cityIds with client.getCityIds()
  • Use cityId to get DataLayer of a certain metric.
  • These DataLayers are pre-calculated statistics that can be served immediatly.
• Stastics API is still in place, but should be considered for smaller areas that need more detail.
  • They are calculated on the fly, so will not be served as fast as the DataLayer API.
  • Available in unit and batch

Table of contents

• Usage & authentication
• Object format
• CotClient API
• DataClient API
• LayerClient API
• Util namespace
• How to
• About Observables

Usage & authentication

Installation

npm install cot-lib --save

Typescript

This library is written in Typescript before being transpiled to Javascript, hence the type information is embedded. (Credentials are optional)

import { CotClient, LocationsClient, SourcesClient, Util } from 'cot-lib';

let credentials = {username: "user", password: "pass"};
let client = new CotClient('http://my-cot-backend.eu', credentials);
let hash = Util.Geohash.encode(31.123456,15.321654);

Javascript

This library can also be used as a regular javascript library. (Credentials are optional)

var CotClient = require('cot-lib').CotClient;
var Util = require('cot-lib').Util;

var credentials = {username: "user", password: "pass"};
var client = new CotClient('http://my-cot-backend.eu', credentials);
var hash = Util.Geohash.encode(31.123456,15.321654);

or

var cot = require('cot-lib');

var credentials = {username: "user", password: "pass"};
var client = new cot.CotClient('http://my-cot-backend.eu', credentials);
var hash = cot.Util.Geohash.encode(31.123456,15.321654);

Creating a CotClient

You start by creating a CotClient object. A CotClient object allows you to request the basic API functions. It needs a reference to the base url of the API.

Basic Auth credentials can be given as an extra optional argument.

import { CotClient } from 'cot-lib';
let credentials = {username: "user", password: "pass"};
let cotClient = new CotClient('http://my-cot-backend.eu', credentials);

Object format

The API uses a couple of object formats that it returns.

Type

A Type object describes sources of the same type. It has an id to uniquely identify it. It bundles different metrics that are measured by this Type.

{
    "id": "bpost.car",
    "metrics": ["dyamand.state.temperature", "dyamand.state.light"]
}
  

Source

A Source represents a sensor, actuator or data producer of some sort. It has an id to uniquely identify it. It has an optional name which is a human readable label for this Source. At last it can have a typeRefs field which contains a set of typeId strings that define the Types this Source belongs to OR/AND (more commonly) it has a metrics field immediatly referring to the metrics available for this source. (A Type simply bundles some metrics together, but is more of a 'managing feauture', since there are no API entrypoints for typeIds)

{
    "id": "Car1",
    "name": "Bpost Mockup Car 1",
    "typeRefs": [
      "bpost.car"
    ],
    "metrics": [
        "airquality.pm10"
    ]
}

Metric

A Metric represents a metric measurement type. For example airquality.pm10. It is a simple string to identify a single type of measurement. It also has a granularity field that determines the default size of returned TemperalPage brackets for data of this Metric. There is also an optional description field that explains the measurement. Finally there is an optional type field that contains the runtime type of the value (eg. FloatNumber, IntNumber).

{
    "id": "co2.ppm",
    "granularity": "HOURLY",
    "description": "The co2 levels measured as parts per million.",
    "type": "FloatNumber"
}

TemporalPage

A TemporalPage object is a bucket (like a page in a pageing mechanism, but spread by time instead of amount). This object contains the parsed Link header values, so they can easily be accessed.

{
    "link": {TemporalPageLink},
    "data": {Data | NamedData | BatchedData}
}

TemporalPageLink

A TemporalPageLink object contains the parsed Link header values relevant for the City of Things client to request follow up requests. It parses the Link header according to the RFC 5988 specification. (note that next, prev and last fields are optional and not always present).

{
    "self": "/sources/Car1/events/20161231/16?from=1483201200",
    "next": "/sources/Car1/events/20161231/17",
    "prev": "/sources/Car1/events/20161231/16?to=1483201200",
    "last": "/sources/Car1/events/20161231/20?to=1483215300",
}

Data

A Data object is a container for the actual data. It contains two keys: columns and values. The index of the columns entries correspond to the index of the entries of each row in the (nested) values-array.

{
    "columns": ["timestamp","co2","temperature"],
    "values": [
        [1483201200, 430, 297.15],
        [1483201202, 432, 297.13],
        [1483201212, 433, 300.01],
        [1483201301, 427, 301.14],
        [1483201302, 419, 298.06],
        [1483201331, 418, 299.48],
        [1483201340, 424, 299.82]
    ]
}

NamedData

This extends a Data object by adding a name field, which is a regular string field.

{
    "name": "Car1" or ["Car1", "Car2"],
    "columns": ["timestamp","co2","temperature"],
    "values": [
        [1483201200, 430, 297.15],
        [1483201202, 432, 297.13],
        [1483201212, 433, 300.01],
        [1483201301, 427, 301.14],
        [1483201302, 419, 298.06],
        [1483201331, 418, 299.48],
        [1483201340, 424, 299.82]
    ]
}

BatchedData

This extends a Data object by adding a bins field. The bins, are the amount of rows the (nested) values-array contains. BatchedData are used when requesting stats. The bins can for instance signify the 24 hours in a day.

{
    "columns": ["mean", "min", "max", "stddev", "count"],
    "values": [
        [16.5, 16, 18, 2, 156],
        [13.5, 12, 21, 2.5, 151],
        [16.15, 14, 20, 4.1, 162],
        ...,
        [10.85, 8, 13, 1.2, 180],
        [14.125, 14, 15, 0.55, 166],
    ],
    "bins": 24
}

CotClient API

CotClient(host: string, credentials?: CotClientCredentials): CotClient

Constructor, call as new CotClient('http://my-cot.backend.eu', {username: "user", password: "pass"}). This will construct a new CotClient object to work with.

cotClient.getSources(): Observable<Source[]>

Returns an Observable containing an array of all Source objects currently available.

cotClient.getSource(sourceId: string, expanded: string = false): Observable<Source>

Returns an Observable containing the Source object with the given sourceId. The expanded argument is false by default, if set to true, the typeRefs or metrics fields will contain full Types or Metrics instead of just ids.

cotClient.getMetrics(): Observable<Metric[]>

Returns an Observable containing an array of all Metric objects currently available.

cotClient.getMetric(metricId: string): Observable<Metric>

Returns an Observable containing the Metric object with the given metricId.

cotClient.getCityIds(): Observable<string[]>

Returns an Observable containing an array of strings that represent city data layer ids. They uniquely identify a city and its level of layer detail (the size of the geohashes used for the datalayer statistical merging).

cotClient.withSources(...sourceIds: string[]): DataClient

Returns a DataClient that internally iterates over all given sourceIds when requesting data or stats. For methods see DataClient API

cotClient.withLocations(...geohashes: string[]): DataClient

Returns a DataClient that internally iterates over all given geohashes of a given type when requesting data or stats. For methods see DataClient API

cotClient.withCityId(cityId: string): LayerClient

Returns a LayerClient that can be used to request statistical data for a layer over a certain city and geospatial granularity defined by the layer id. For methods see LayerClient API

DataClient API

A dataclient is a common structure for getting data and statistics from the client, no matter if you target one or more sources, or one or more locations (of a given type).

| Operation | Operation chain 1 | Operation chain 2 | Return | Description | |-------------------|------------------------|-----------------------------|---------------------------|--------------------------------------------------------------------------------| | .getData(metric) | | | | Request data of a given metric | | | .latest() | | Observable<TemporalPage> | Request the latest available data. | | | .poll(period, fromTS?) | | Observable<TemporalPage> | Continuously poll the data | | | .range(fromTS, toTS) | | Observable<TemporalPage> | Request a range of data, bracket by bracket | | .getStats(metric) | | | | Request statistics | | | .getUnit() | | | Request statistics in units (day/hour) | | | | .getHour(hourTS) | Observable<TemporalPage> | Get statistics unit for a single hour | | | | .getHourRange(fromTS, toTS) | Observable<TemporalPage> | Get statistics units for a range of hours, one after the other | | | | .getHourRecap(fromTS, toTS) | Observable<Data> | Get a single statistics hour unit as a recap representing the requested range | | | | .getHourRecapOf(hourTS[]) | Observable<Data> | Get a single statistics hour unit as a recap representing the requested hours. | | | | .getDay(dayTS) | Observable<TemporalPage> | Get statistics unit for a single day | | | | .getDayRange(fromTS, toTS | Observable<TemporalPage> | Get statistics units for a range of days, one after the other | | | | .getDayRecap(fromTS, toTS) | Observable<Data> | Get a single statistics day unit as a recap representing the requested range | | | | .getDayRecapOf(dayTS[]) | Observable<Data> | Get a single statistics day unit as a recap representing the requested days. | | | .getBatch() | | | Request statistics in batch form (complete day filled with hour bins) | | | | .getDay(dayTS) | Observable<TemporalPage> | Get statistics batch for a single day | | | | .getDayRange(fromTS, toTS) | Observable<TemporalPage> | Get statistics batches for a range of days, one after the other | | | | .getDayRecap(fromTS, toTS) | Observable<BatchedData> | Get a single statistics day batch as a recap representing the requested range | | | | .getDayRecapOf(dayTS[]) | Observable<BatchedData> | Get a single statistics day batch as a recap representing the requested days. |

LayerClient API

A layerclient is a structure for getting data and statistics from a city, typically to generate a visual overview (eg. a heatmap).

| Operation | Operation chain 1 | Return | Description | |-------------------|-----------------------------|---------------------------|--------------------------------------------------------------------------------| | .getLayer(metric) | | | Request data of a given metric | | | .getHour(hourTS) | Observable<TemporalPage> | Get statistics for a single hour | | | .getHourRange(fromTS, toTS) | Observable<TemporalPage> | Get statistics for a range of hours, one after the other | | | .getHourRecap(fromTS, toTS) | Observable<NamedData> | Get statistics merged into one hour but representing the requested hour range | | | .getHourRecapOf(hourTS[]) | Observable<NamedData> | Get statistics merged into one hour but representing the requested hours | | | .getDay(dayTS) | Observable<TemporalPage> | Get statistics for a single day | | | .getDayRange(fromTS, toTS | Observable<TemporalPage> | Get statistics for a range of days, one after the other | | | .getDayRecap(fromTS, toTS) | Observable<NamedData> | Get statistics merged into one day but representing the requested day range | | | .getDayRecapOf(dayTS[]) | Observable<NamedData> | Get statistics merged into one day but representing the requested days |

Util namespace

The Util namespace contains some static methods that can be helpful when working with the City of Things cilent library. They are structured in exposed static classes.

Util.Geohash.encode(lat: number, lng: number, precision: number = 9): string

Encodes a latitude-longitude point into a geohash with the given precision.

Util.Geohash.decode(geohash: string): {"latitude": number, "longitude": number}

Decode a given geohash into a latitude-longitude point.

Util.Geohash.bounds(geohash: string): number[]

Decode a given goehash into a bounding box that matches it. Data is returned as a four-element array: [minlat, minlon, maxlat, maxlon].

Util.Geohash.isContained(geohash: string, minlat: number, maxlat: number, minlng: number, maxlng: number): boolean

Wether the given geohash (or its upperleftcorner) is contained within the given coordinates.

Util.Time.timestamp(year: number, month: number, day = 1, hours = 0, minutes = 0, seconds = 0, milliseconds = 0): number

Returns the unix (UTC) timestamp (in milliseconds!) for the given date parameters. The arguments are considered to be in UTC time.

Argument | Optional | Description ----------------|:---------:|------------------------------------- year |no | Four digit number month |no | The month in the year (1-12) day |yes | The day of the month (1-31) hours |yes | The hour of the day (0-23) minutes |yes | The minute of the day (0-59) seconds |yes | The seconds of the day (0-59) milliseconds |yes | The milliseconds of the day (0-999)

Util.Time.getDayStr(timestamp: number): string

Returns the string representation that can be used in the REST api urls. (eg. 20171205 for December 5, 2017) (timestamp argument is in milliseconds!)

Util.Time.getHourStr(timestamp: number): string

Returns the string representation that can be used in the REST api urls. (eg. 20171205/13 for December 5, 2017 at 1pm) (timestamp argument is in milliseconds!)

Util.Time.getWeekdaysTS(year: number, month: number): number[][]

Returns an array of 7 string arrays (mon-sun). Each array contains the UTC timestamps (in milliseconds!) for each instance of that weekday in the month.

How to

Follow this table for a few practical things to do and where to look for the information required.

I want to ... | API to use | Creation --------------------------------------------------------|---------------------------------------------------------------|---------------------------------------------------- display individual datapoints of a (smaller) location | DataClient#getData(metric) | Create with client.withLocations(hash1, hash2) show a graph of a sensor | DataClient#getData(metric) | Create with client.withSources(id1, id2) show statistics of a sensor of a day/hour/range | DataClient#getStats(metric) | Create with client.withSources(id1, id2) show statistics of a small location of a day/hour/range | DataClient#getStats(metric) | Create with client.withLocations(hash1, hash2) show statistics of whole city of a day/hour/range | LayerClient#getLayer(metric) | Create with client.withCityId(cityId) generate a heatmap of a city | LayerClient#getLayer(metric) | Create with client.withCityId(cityId)

About Observables

This API makes heavy use of ReactiveX Observables. They are a very modern way to do asynchronous communication. When calling subscribe(...) on an Observable, the Observable is activated and starts its work. It will asynchronously notify its different handlers. The first handler - typically called next (or onNext) - can be notified multiple times, everytime there is new data available, the second one handles errors and the last one is called when the Observable stream has completed its work.

Simple use of Observables

The simplest way to use them is as follows. Given a method getMyData() that returns an Observable<MyData> type. This is how you handle it:

Typescript

client.getMyData().subscribe(
    nextData => { 
        // handle nextData as its coming in, this handler will be called on every update with new data
    },
    error => {
        // do something when an error occurs
    },
    () => {
        // this handler will be called when the data streaming is complete and there is no more incoming data
    }
);

Javascript

client.getMyData().subscribe(
    function(nextData) { 
        // handle nextData as its coming in, this handler will be called on every update with new data
    },
    function(error) {
        // do something when an error occurs
    },
    function() {
        // this handler will be called when the data streaming is complete and there is no more incoming data
    }
);

There are many more things you can do with Observables, for a more comprehensive guide we refer to the ReactiveX introduction website and the RxJs github page with its excellent Readme.

What about Promises?

Beware: An Observable allows you to call the toPromise() method on it, after which it can be handled as a normal Promise with a then(...) method. Be aware though that to first argument callback of the then(...) method will only be called with the very last next update of the observable. So this will not always behave as you would expect.