@rddill/cloudant

v1.1.12

Published

3 years ago

common no sql access for local couchDB and cloud based Cloudant

Downloads

0High
0Medium
0Low

rddill

CouchDB Cloudant

Cloudant.js

Common routines to allow SQL-like access to either CouchDB or Cloudant. Requests run identically in both environments. The service requires a json object with credentials for either or both Cloudant and CouchDB.

V1.06 adds error processing for situations when the target DBM is not available.
V1.07 updates test script to support change in select2 return
V1.08 adds support for IBM IAM authentication. This introduces additional elements to the cloudant object in the json input and also adds an additional element at the end of the sequence (useIAM) with is either true or not. Since IAM is only (currently) applicable to Cloudant, there is no attempt to support it for CouchDB. If useIAM === true, then any values for cloudant username and password are ignored. Likewise, if useIAM !=== true, then any values for the cloudant IAM section are ignored.
V1.09 adds support for IAM token expiration. Tokens expire 60 minutes after creation. This will not refresh tokens once time has passed 50 minutes after creation. Token refresh happens on the next request after the 50 minute window has passed.
V1.0.10 changes the IAM refresh approach. While refresh was working in 1.0.9, the command requested (insert, update, etc.) failed. This inserts an adjustable wait time to let the system settle after refreshing the IAM token before redoing the original request. The default wait time is 2 seconds and is managed in an accessible field called authenticateTimeout This expects a value based on milliseconds, making the default value 2000
V1.0.11 Update to selectMulti. Incorrectly passing key value, resolved.
V1.0.12 Add support for _find
V1.1.0 Update IAM support to auto refresh after 3500 seconds. This is handled in the authenticate routine iff IAM is selected. If user/password selected, there is no need for token refresh and this is not invoked.
V1.1.1 Update createBackup service to also capture defined indices
V1.1.4 update Restore table to work with new backup service.
V1.1.5 fix bug in Backup when saving multiple indices.
V1.1.6 fix bug in Backup when saving large files.
V1.1.7 fix bug in Backup when saving large files, failing to resolve on final retrieve.
V1.1.8 remove console.log from backup routine.
V1.1.9 upgrade to nodejs v12.
V1.1.10 fix bug in restoreTable service
V1.1.11 update to node16
V1.1.12 use for node14

The format of the json object is:

{
  "cloudant": {
    "apikey": "your IAM api key",
    "host": "your IAM host name",
    "iam_apikey_description": "API Key description",
    "iam_apikey_name": "IAM APIKey name",
    "iam_role_crn": "IAM role",
    "iam_serviceid_crn": "IBM Service ID",
    "username": "your-cloudant-user-name",
    "password": "your-cloudant-password",
    "port": 443,
    "urlBase": "your-cloudant-url-without-http[s]://-prefix",
    "url": "your-cloudant-url"
  },
  "couchdb": {
    "url": "your-couchdb-local-url",
    "urlBase": "your-couchdb-local-url-without-http://-prefix",
    "username": "your-couchdb-username",
    "password": "your-couchdb-password"
  },
  "useCouchDB": true,
  "useIAM": true,
  "backupFolder": "/HTML/backups"
}

The item, useCouchDB requires either true or false as the supplied value. If useCouchDB is true then the cloudant credentials are ignored and valid credentials are required for your couchdb data base. if useCouchDB is false, then the couchdb credentials are ignored and valid credentials are required for your cloudant data base.

If you choose to use the backup and restore option, it is recommended that you provide a valid value for the backupFolder option. This will ensure that your backups are sent to the correct location.

This service supports the following capabilities:

getCredsFromFile = 'function (_file): ingests credentials from file at provided absolute path.';
getCredsFromJSON = 'function (JSON): ingest credentials from provided JSON object.';
authenticate = 'function (): uses credentials ingested via one of the setCreds services to authenticate to couchdb or cloudant server';
create = 'function (_name): create a new database';
drop = 'function (_name): drop a database';
insert = 'function (_name, _object): insert JSON _object into database _name';
update = 'function (_name, _oid, _object): update JSON object specified by object _oid in database _name with new object _object';
select = 'function (_name, _selector): select objects from database _name specified by selection criteria _selector';
select2 = 'function (_name, _keysw, _selector): select objects from database _name specified by selection criteria _selector using multiple keys as defined in keys';
_delete = 'function (_name, _oid): delete object specified by _oid in database _name';
listAllDatabases = 'function (): list all databases I can access';
getDocs = 'function (_name): list all documents in database _name';
capabilities = 'return this object with descriptors.';
createBackup = 'function (_name): create a backup of database _name.';
getBackups = 'function (): get a list of available time-stamped backups.';
restoreTable = 'function (_file): restore database _name.';

Each service uses standard JSDOC documentation, which is available here

Where authentication is required in the following examples, the complete authentication process is shown. When using these services in an on-going basis, providing credentials and then performing authentication is only required once; it does not have to be done each time a service request has been made. The cloudant module preserves the result of the last authentication action and uses that result in subsequent service execution.

Authentication Services:

getCredsFromFile

// in this example, credentials are held in a file called db.env.json which is in the root folder of the calling function.
// get the path node module
let path = require('path');
let db = require(rddill/cloudant)
// create a path to the local file (db.env.json)
let envFile = path.join(__dirname, 'db.env.json');
// ingest the credentials
let _fRes = db.getCredsFromFile(envFile);

on Success, returns JSON object with credentials for selected data base on Failure, returns JSON object with two elements:

errorMessage: text string
error: error object

getCredsFromJSON

// in this example, credentials are held in a file called db.env.json which is in the root folder of the calling function.
// get the path node module
let path = require('path');
let db = require(rddill/cloudant)
// create a path to the local file (db.env.json)
let envFile = path.join(__dirname, 'db.env.json');
let authJSON = JSON.parse(fs.readFileSync(envFile));
authJSON.backupFolder = process.cwd() + authJSON.backups;
// ingest the credentials
let _fRes = db.getCredsFromJSON(authJSON);

on Success, returns JSON object with credentials for selected data base on Failure, returns JSON object with two elements:

errorMessage: text string
error: error object

authenticate

// in this example, credentials are held in a file called db.env.json which is in the root folder of the calling function.
// get the path node module
let path = require('path');
let db = require(rddill/cloudant)
let envFile = path.join(__dirname, 'db.env.json');
let authJSON = JSON.parse(fs.readFileSync(envFile));
authJSON.backupFolder = process.cwd() + authJSON.backups;
// ingest the credentials
let _fRes = db.getCredsFromJSON(authJSON);
let _auth = db.authenticate();

on Success, returns response object from database authenticate action on Failure, returns JSON object with one of three error elements:

errorMessage: if authenticate credentials do not exist, then returns text string with message stating authenticate credentials are null
errorMessage: if body.error exists, then returns text string with body.error + body.reason
error: if body.error does not exist, then returns error object

Database Management Services:

create

// get the path node module
let path = require('path');
let db = require(rddill/cloudant)
let db1 = 'test-table1';
// create a path to the local file (db.env.json)
let envFile = path.join(__dirname, 'db.env.json');
let authJSON = JSON.parse(fs.readFileSync(envFile));
authJSON.backupFolder = process.cwd() + authJSON.backups;
// ingest the credentials
let _fRes = db.getCredsFromJSON(authJSON);
return db.authenticate()
  .then(_auth => {
    return db.create(db1)
      .then(_db => {
        console.log('create successful: ', _db_);
        // Your code to process once creation complete goes here
      })
      .catch(error => {
        console.log('create error: ', error.error);
      });
  });

on Success, returns JSON object with body object from create operation

{success: (complete response object)} on create Failure, returns JSON object:
{errorMessage: text string}
{error: Error object} on JSON Parse Failure, returns JSON object with one element:
{error: JSON error object}

drop

// get the path node module
let path = require('path');
let db = require(rddill/cloudant)
let db1 = 'test-table1';
// create a path to the local file (db.env.json)
let envFile = path.join(__dirname, 'db.env.json');
let authJSON = JSON.parse(fs.readFileSync(envFile));
authJSON.backupFolder = process.cwd() + authJSON.backups;
// ingest the credentials
let _fRes = db.getCredsFromJSON(authJSON);
  return db.authenticate()
    .then(_auth => {
      return db.drop(db1)
        .then(_db => {
          // insert your code here for successful completion
        })
        .catch(error => {
          console.log('drop error: ', error.error);
        });
    });

on Success, returns JSON object with body object from drop operation:

{success: { success: { ok: true } }} on create Failure, returns JSON object:
{errorMessage: text string} on JSON Parse Failure, returns JSON object with one element:
{error: JSON error object}

listAllDatabases

// get the path node module
let path = require('path');
let db = require(rddill/cloudant)
let db1 = 'test-table1';
// create a path to the local file (db.env.json)
let envFile = path.join(__dirname, 'db.env.json');
let authJSON = JSON.parse(fs.readFileSync(envFile));
authJSON.backupFolder = process.cwd() + authJSON.backups;
// ingest the credentials
let _fRes = db.getCredsFromJSON(authJSON);
return db.authenticate()
  .then(_auth => {
    return db.listAllDatabases()
      .then(_select => {
        // process the returned list of data bases
        // check the _select.success object for details
      })
      .catch(error => {
          console.log('listAllDatabases error: ', error.error);
        });

  )};

on Success, returns JSON object with following object from operation:

{success: {rows: [array of table names], total_rows: (# of rows in array)} on create Failure, returns JSON object:
{errorMessage: text string} on JSON Parse Failure, returns JSON object with one element:
{error: JSON error object}

Backup and Restore Services:

createBackup

// get the path node module
let path = require('path');
let db = require('rddill/cloudant');
let fs = require('fs');
let db1 = 'test-table1';
// create a path to the local file (db.env.json)
let envFile = path.join(__dirname, 'db.env.json');
let authJSON = JSON.parse(fs.readFileSync(envFile));
authJSON.backupFolder = process.cwd() + authJSON.backups;
// ingest the credentials
let _fRes = db.getCredsFromJSON(authJSON);
return db.authenticate()
  .then(_auth => {
    return db.createBackup(db1)
      .then(_select => {
        backupFile = _select.success.name;
        let _backup = _select.success
        // actually create the backup file on your local system...
        // the following example creates a backup in your defined backup folder location
        // separating the view content from the data content
        // This should be treated as a working EXAMPLE and modified accordingly for your own purposes.
        let fileName = authJSON.backupFolder + 'Backup_';
        fileName += (_backup.name == "") ? "allFiles" : backupFile;
        fileName +="_"+getTimeStamp()+".json";
        console.log("creating backup at: ", fileName);
        let rows = JSON.parse(_backup).row;
        let _views = '"views": ['; var viewNum = 0;
        let _str = "[";
        for (let each = 0; each < rows.length; each++)
        {(function(_idx, _array)
          {
            if(_array[_idx].doc._id != '_design/views')
            { if(_idx>0){_str+=", ";}
              _str+=JSON.stringify(_array[_idx].doc); 
            }
            else {
              if(viewNum>0){_views+=", ";} _views += '{ "_id": "_design/views", "views":' +JSON.stringify(_array[_idx].doc.views)+'}';
            }
          })(each, rows)}
        _str+="]"; _views += "]";
        fs.writeFileSync(fileName, '{"table" : "'+_backup.name+'", "date": "'+getTimeStamp()+'", "rows": '+_str+', '+_views+'}', 'utf8');
      })
      .catch(error => {
          console.log('create Backup File error: ', error.error);
        });
  getTimeStamp: function() { return (new Date(Date.now()).toISOString().replace(/:/g, '.')); }

getBackups

// get the path node module
let path = require('path');
let db = require(rddill/cloudant)
let db1 = 'test-table1';
// create a path to the local file (db.env.json)
let envFile = path.join(__dirname, 'db.env.json');
let authJSON = JSON.parse(fs.readFileSync(envFile));
authJSON.backupFolder = process.cwd() + authJSON.backups;
// ingest the credentials
let _fRes = db.getCredsFromJSON(authJSON);
return db.authenticate()
  .then(_auth => {
    return db.getBackups()
      .then(_select => {
        // process the returned list of data bases
        // check the _select.success object for details
      })
      .catch(error => {
          console.log('getBackups error: ', error.error);
        });

on Success, returns JSON object with following object from operation:

{success: files } on Failure, returns JSON object:
{error: error}

restoreTable

// get the path node module
let path = require('path');
let db = require(rddill/cloudant)
let db1 = 'test-table1';
let backupFileName = 'testBackupfile.json';
// create a path to the local file (db.env.json)
let envFile = path.join(__dirname, 'db.env.json');
let authJSON = JSON.parse(fs.readFileSync(envFile));
authJSON.backupFolder = process.cwd() + authJSON.backups;
// ingest the credentials
let _fRes = db.getCredsFromJSON(authJSON);
return db.authenticate()
  .then(_auth => {
    return db.restoreTable(backupFile)
      .then(_restore => {
        // perform appropriate post-processing and logging here
      })
      .catch(error => {
          console.log('restoreTable error: ', error.error);
        });

on Success, returns JSON object with following object from operation:

{success: {table: _table, records: records, views: views}}
The records object and the views object list each record and view inserted using either a success object (e.g. successfully inserted), or a failure object (unsuccessful insert):
- {success: {id: _object._id, success: _res}}
- {error: {id: _object.id, error: _err}} on table create Failure, returns JSON object:
{error: error}