mg-dbx-napi

v1.5.13

Published

a month ago

High speed Synchronous and Asynchronous access to InterSystems Cache/IRIS and YottaDB from Node.js or Bun.

Downloads

207

0High
0Medium
0Low

chrisemunt

nosql intersystems cache iris yottadb m mumps

mg-dbx-napi

High speed Synchronous and Asynchronous access to InterSystems Cache/IRIS and YottaDB from Node.js or Bun.

Chris Munt [email protected]
16 October 2024, MGateway Ltd http://www.mgateway.com

Verified to work with Node.js and the Bun JavaScript engine.
Two connectivity models to the InterSystems or YottaDB database are provided: High performance via the local database API or network based.
Release Notes can be found at the end of this document.

Contents

What Is mg-dbx-napi?

This package provides Node.js and Bun applications with extremely high-performance access to InterSystems database products and the YottaDB database.

Installing mg-dbx-napi

   npm install mg-dbx-napi

Using mg-dbx-napi

mg-dbx-napi is loaded into a Node.js or Bun.js script as follows:

    import {server, mglobal, mclass, mcursor} from 'mg-dbx-napi';

If you don't require all the classes, just import the ones you need, eg:

    import {server, mglobal} from 'mg-dbx-napi';

Notes:

mg-dbx-napi is compatible with both Node.js (version 20 and later) and Bun.js (version 1.0 and later).
Pre-compiled versions of the mg-dbx-napi.node add-on are included within the installed package, for the following platforms:
- Windows 64-bit
- Linux x64
- Linux ARM64

The two Linux versions have been compiled using version 2.31 of the GNU C Library, so they will work with the following Linux distributions and their later versions:

Ubuntu 20.04
Debian 11 (Bullseye)
Suse Linux 15-SP5
Redhat 9.2
CentOS v9
Raspberry Pi OS 2023-05-03/11

If you are using a different Linux distribution, check its C Library version using:

    ldd --version

mg-dbx-napi will automatically detect your operating system and architecture and load the appropriate add-on version and intermediate interface shim module.
If you want to use mg-dbx-napi on a platform other than those for which pre-compiled add-on versions are provided, please contact Chris Munt [email protected]

Supported Databases

mg-dbx-napi is designed for use with the following databases:

mg-dbx-napi can connect in two ways to these databases:

networked connection, where the Node.js/Bun.js system and database can be on separate servers
in-process API connection, where the Node.js/Bun.js system and database reside on the same physical server

Networked connections provide the most all-round platform compatibility (eg Node.js running on a Windows system can connect to YottaDB which only runs on Linux), and highest levels of security. In order to use a networked connection, you need to install our DB Superserver on the database system: see next section below.

The in-process API connection mode provides the very highest levels of performance, reaching near in-memory performance access to the respective database persistent storage. Note that in this mode, you need to be aware of the platform availability of the database products:

InterSystems' Cache and IRIS database versions are available for both Windows and Linux (x64 and ARM64)
YottaDB is only available on Linux (x64 and ARM64)

The in-process API connection does not require installation of our DB Superserver on the database system, unless:

you want to use SQL to access the data held on the database
you want to use the Merge command with the YottaDB database

Installing the M support routines (also known as the DB Superserver)

The M support routines are required for:

Network based access to databases (unless the experimental JavaScript based superserver is used).
Direct access to SQL (either via the API or via the network).

If none of the above apply you do not need to install these routines - proceed to Connecting to the database.

Two M routines need to be installed (%zmgsi and %zmgsis). These can be found in the Service Integration Gateway (mgsi) GitHub source code repository (https://github.com/chrisemunt/mgsi). Note that it is not necessary to install the whole Service Integration Gateway, just the two M routines held in that repository.

Installation for InterSystems Cache/IRIS

   do $system.OBJ.Load("/isc/zmgsi_isc.ro","ck")

Change to your development UCI and check the installation:

   do ^%zmgsi

   MGateway Ltd - Service Integration Gateway
   Version: 4.5; Revision 30 (10 November 2023)

Installation for YottaDB

The instructions given here assume a standard 'out of the box' installation of YottaDB (version 1.38) deployed in the following location:

   /usr/local/lib/yottadb/r138

The primary default location for routines:

   /root/.yottadb/r1.38_x86_64/r

Copy all the routines (i.e. all files with an 'm' extension) held in the GitHub /yottadb directory to:

   /root/.yottadb/r1.38_x86_64/r

Change directory to the following location and start a YottaDB command shell:

   cd /usr/local/lib/yottadb/r138
   ./ydb

Link all the zmgsi routines and check the installation:

   do ylink^%zmgsi

   do ^%zmgsi

   MGateway Ltd - Service Integration Gateway
   Version: 4.5; Revision 30 (10 November 2023)

Note that the version of zmgsi is successfully displayed.

Finally, add the following lines to the interface file (zmgsi.ci in the example used in the db.open() method).

   sqlemg: ydb_string_t * sqlemg^%zmgsis(I:ydb_string_t*, I:ydb_string_t *, I:ydb_string_t *)
   sqlrow: ydb_string_t * sqlrow^%zmgsis(I:ydb_string_t*, I:ydb_string_t *, I:ydb_string_t *)
   sqldel: ydb_string_t * sqldel^%zmgsis(I:ydb_string_t*, I:ydb_string_t *)
   ifc_zmgsis: ydb_string_t * ifc^%zmgsis(I:ydb_string_t*, I:ydb_string_t *, I:ydb_string_t*)

A copy of this file can be downloaded from the /unix directory of the mgsi GitHub repository here

Starting the DB Superserver (for network based connectivity only)

The default TCP server port for zmgsi is 7041. If you wish to use an alternative port then modify the following instructions accordingly.

For InterSystems DB servers the concurrent TCP service should be started in the %SYS Namespace.

Start the DB Superserver using the following command:

   do start^%zmgsi(0)

To use a server TCP port other than 7041, specify it in the start-up command (as opposed to using zero to indicate the default port of 7041).

For YottaDB, as an alternative to starting the DB Superserver from the command prompt, Superserver processes can be started via the xinetd daemon. Instructions for configuring this option can be found in the mgsi repository here

Connecting to the database

Example:

    import {server, mglobal} from 'mg-dbx-napi';
    let db = new server();
    db.open({
      type: "IRIS",
      path:"/opt/IRIS20181/mgr",
      username: "_SYSTEM",
      password: "SYS",
      namespace: "USER"
    });

    let person = new mglobal(db, "Person");
    let name = person.get(1);

    // ..do more stuff, then finally:

    db.close();

Most mg-dbx-napi methods are capable of operating either synchronously or asynchronously. For an operation to complete asynchronously, simply supply a suitable callback as the last argument in the call.

The first step is to include the mg-dbx-napi classes in your JavaScript project. For example:

   import {server, mglobal, mclass, mcursor} from 'mg-dbx-napi';

The server class will always be required. For classes mglobal, mclass and mcursor, just import the ones that you need.

Creating an instance of the server class:

   let db = new server();

Open a connection to the database

First create a JavaScript container object (defined as db in the examples) as described in the Connecting to the database section.

In the following examples, modify all paths (and any user names and passwords) to match those of your own installation.

InterSystems Cache

API based connectivity

Assuming Cache is installed under /opt/cache20181/

       var open = db.open({
           type: "Cache",
           path:"/opt/cache20181/mgr",
           username: "_SYSTEM",
           password: "SYS",
           namespace: "USER"
         });

Network based connectivity

Assuming Cache is accessed via localhost listening on TCP port 7041

       var open = db.open({
           type: "Cache",
           host: "localhost",
           tcp_port: 7041,
           username: "_SYSTEM",
           password: "SYS",
           namespace: "USER"
         });

InterSystems IRIS

API based connectivity

Assuming IRIS is installed under /opt/IRIS20181/

       var open = db.open({
           type: "IRIS",
           path:"/opt/IRIS20181/mgr",
           username: "_SYSTEM",
           password: "SYS",
           namespace: "USER"
         });

Network based connectivity

Assuming IRIS is accessed via localhost listening on TCP port 7041

       var open = db.open({
           type: "IRIS",
           host: "localhost",
           tcp_port: 7041,
           username: "_SYSTEM",
           password: "SYS",
           namespace: "USER"
         });

YottaDB

API based connectivity

Assuming an 'out of the box' YottaDB installation under /usr/local/lib/yottadb/r138.

       var open = db.open({
           type: "YottaDB",
           path: "/usr/local/lib/yottadb/r138",
           env_vars: {
             ydb_gbldir: '/root/.yottadb/r1.38_x86_64/g/yottadb.gld',
             ydb_routines: '/root/.yottadb/r1.38_x86_64/o*(/root/.yottadb/r1.38_x86_64/r /root/.yottadb/r) /usr/local/lib/yottadb/r138/libyottadbutil.so',
             ydb_ci: '/usr/local/lib/yottadb/r138/zmgsi.ci'
           }
         });

Note that the actual YottaDB environment variables that you should use will depend on your YottaDB configuration, and, in particular, the Globals and Routines that you may need to access from Node.js or Bun.js.

Network based connectivity

Assuming YottaDB is accessed via localhost listening on TCP port 7041

       var open = db.open({
           type: "YottaDB",
           host: "localhost",
           tcp_port: 7041,
         });

Additional (optional) properties for the open() method

timeout: The timeout (in seconds) to be applied to database operations invoked via network based connections. The default value is 10 seconds.

Return the version of mg-dbx-napi

   var result = db.version();

Example:

   console.log("\nmg-dbx-napi Version: " + db.version());

Returning (and optionally changing) the current directory (or Namespace)

   current_namespace = db.namespace([<new_namespace>]);

Example 1 (Get the current Namespace):

   var nspace = db.namespace();

Note this will return the current Namespace for InterSystems databases and the value of the current global directory for YottaDB (i.e. $ZG).

Example 2 (Change the current Namespace):

   var new_nspace = db.namespace("SAMPLES");

If the operation is successful this method will echo back the new Namespace name. If not successful, the method will return the name of the current (unchanged) Namespace.

Returning (and optionally changing) the current character set

UTF-8 is the default character encoding for mg-dbx-napi. The other option is the 8-bit ASCII character set (characters of the range ASCII 0 to ASCII 255). Native Unicode (as UTF-16) is supported for InterSystems DB Servers. The ASCII character set is a better option when exchanging single-byte binary data with the database.

   current_charset = db.charset([<new_charset>]);

Example 1 (Get the current character set):

   var charset = db.charset();

Example 2 (Change the current character set):

   var new_charset = db.charset('ascii');

Example 3 (Native Unicode support for InterSystems DB Servers):

   var new_charset = db.charset('utf-16');

If the operation is successful this method will echo back the new character set name. If not successful, the method will return the name of the current (unchanged) character set.
Currently supported character sets and encoding schemes: 'ascii', 'utf-8' and 'utf-16' for InterSystems DB Servers.

Setting (or resetting) the timeout for the connection

   new_timeout = db.settimeout(<new_timeout>);

Specify a new timeout value (in seconds) for the connection. If the operation is successful this method will return the new value for the timeout.

Example (Set the timeout to 30 seconds):

   var new_timeout = db.settimeout(30);

Get the error message associated with the previous database operation

   error_message = db.geterrormessage();

This method will return the error message (as a string) associated with the previous database operation. An empty string will be returned if the previous operation completed successfully.

Close database connection

   db.close();

Invocation of database commands

Database commands can either be invoked directly from a database object or via a mglobal container object. The latter approach does give slightly better performance - particularly in cases where the same global reference is repeatedly used.

mglobal container objects can be created as described in the Connecting to the database section.

Example (using a global named "Person"):

   let person = new mglobal(db, "Person");

Set a record

Using a global object:

   var result = mglobal.set(<key>, <data>);

Example:

   person.set(1, "John Smith");

Alternatively:

   var result = db.set(<global name>, <key>, <data>);

Example:

   db.set("Person", 1, "John Smith");

Get a record

Using a global object:

   var result = mglobal.get(<key>);

Example:

   var name = person.get(1);

Alternatively:

   var result = db.get(<global name>, <key>);

Example:

   var name = db.get("Person", 1);

Delete a record

Using a global object:

   var result = mglobal.delete(<key>);

Example:

   var result = person.delete(1);

Alternatively:

   var result = db.delete(<global name>, <key>);

Example:

   var result = db.delete("Person", 1);

Check whether a record is defined

Using a global object:

   var result = mglobal.defined(<key>);

Example:

   var result = person.defined(1);

Alternatively:

   var result = db.defined(<global name>, <key>);

Example:

   var result = db.defined("Person", 1);

Parse a set of records (in order)

Using a global object:

   var result = mglobal.next(<key>);

Example:

   var key = "";
   while ((key = person.next(key)) != "") {
      console.log("\nPerson: " + key + ' : ' + person.get(key));
   }

Alternatively:

   var result = db.next(<global name>, <key>);

Example:

   var key = "";
   while ((key = db.next("Person", key)) != "") {
      console.log("\nPerson: " + key + ' : ' + db.get("Person", key));
   }

Parse a set of records (in reverse order)

Using a global object:

   var result = mglobal.previous(<key>);

Example:

   var key = "";
   while ((key = person.previous(key)) != "") {
      console.log("\nPerson: " + key + ' : ' + person.get(key));
   }

Alternatively:

   var result = db.previous(<global name>, <key>);

Example:

   var key = "";
   while ((key = db.previous("Person", key)) != "") {
      console.log("\nPerson: " + key + ' : ' + db.get("Person", key));
   }

Increment the value of a global node

Using a global object:

   var result = mglobal.increment(<key>, <increment_value>);

Example (increment the value of the "counter" node by 1.5 and return the new value):

   var result = person.increment("counter", 1.5);

Alternatively:

   var result = db.increment(<global name>, <key>, <increment_value>);

Example (increment the value of the "counter" node by 1.5 and return the new value):

   var result = db.increment("Person", "counter", 1.5);

Lock a global node

Using a global object:

   var result = mglobal.lock(<key>, <timeout>);

Example (lock global node '1' with a timeout of 30 seconds):

   var result = person.lock(1, 30);

Alternatively:

   var result = db.lock(<global name>, <key>, <timeout>);

Example (lock global node '1' with a timeout of 30 seconds):

   var result = db.lock("Person", 1, 30);

Note: Specify the timeout value as '-1' for no timeout (i.e. wait until the global node becomes available to lock).

Unlock a (previously locked) global node

Using a global object:

   var result = mglobal.unlock(<key>);

Example (unlock global node '1'):

   var result = person.unlock(1);

Alternatively:

   var result = db.unlock(<global name>, <key>);

Example (unlock global node '1'):

   var result = db.unlock("Person", 1);

Merge (or copy) part of one global to another

Global objects must be used to invoke the Merge command.

Merge from global2 to global1:

   var result = <global name 1>.merge([<key1>,] <global name 2> [, <key2>]);

Example 1 (merge ^MyGlobal2 to ^MyGlobal1):

   global1 = new mglobal(db, 'MyGlobal1');
   global2 = new mglobal(db, 'MyGlobal2');
   global1.merge(global2);

Example 2 (merge ^MyGlobal2(0) to ^MyGlobal1(1)):

   global1 = new mglobal(db, 'MyGlobal1', 1);
   global2 = new mglobal(db, 'MyGlobal2', 0);
   global1.merge(global2);

Alternatively:

   global1 = new mglobal(db, 'MyGlobal1');
   global2 = new mglobal(db, 'MyGlobal2');
   global1.merge(1, global2, 0);

Create a set of child data nodes in the DB Server

Status: Beta release.

   var result = mglobal.setchildnodes([<partialkey...>,] <[childnodes]>, <{options}>);

Where:

childnodes: an array of child data nodes. Each element has the following form:
```
  {key: <keys value(s)>, data: <data value>}
```
options: A JSON object in which the following options can be defined:
- lock: if true, acquire a lock on the target DB data node before proceeding.
- locktimeout: timeout for the lock operation (in seconds).
- numerickeys: Treat the dataset as a numerically-keyed JavaScript array.

Example 1:

   global = new mglobal(db, 'MyGlobal');
   const nodes = [];
   nodes[0] = {key: 1, data: "record#1"};
   nodes[1] = {key: 2, data: "record#2"};
   nodes[2] = {key: 3, data: "record#3"};
   nodes[3] = {key: "a", data: "record#a"};
   result = global.setchildnodes(nodes, {});

This will create the following data nodes in the DB Server:

   ^MyGlobal(1)="record#1"
   ^MyGlobal(2)="record#2"
   ^MyGlobal(3)="record#3"
   ^MyGlobal("a")="record#a"

Example 2 (using partial keys):

   global = new mglobal(db, 'MyGlobal', "a");
   const nodes = [];
   nodes[0] = {key: 1, data: "record#1"};
   nodes[1] = {key: 2, data: "record#2"};
   nodes[2] = {key: 3, data: "record#3"};
   nodes[2] = {key: "a", data: "record#a"};
   result = global.setchildnodes("b", nodes, {});

This will create the following data nodes in the DB Server:

   ^MyGlobal("a","b",1)="record#1"
   ^MyGlobal("a","b",2)="record#2"
   ^MyGlobal("a","b",3)="record#3"
   ^MyGlobal("a","b","a")="record#a"

Example 3 (specifying multiple key values in an array):

   global = new mglobal(db, 'MyGlobal');
   const nodes = [];
   nodes[0] = {key: 1, data: "record#1"};
   nodes[1] = {key: [1, 2, "c"], data: "three keys"};
   result = global.setchildnodes(nodes, {});

This will create the following data nodes in the DB Server:

   ^MyGlobal(1)="record#1"
   ^MyGlobal(1,2,"c")="three keys"

Example 4 (using numeric keys):

   global = new mglobal(db, 'MyGlobal');
   const nodes = [];
   nodes[0] = "record#1";
   nodes[1] = "record#2";
   nodes[2] = "record#3};
   result = global.setchildnodes(nodes, {numerickeys: true});

This will create the following data nodes in the DB Server:

   ^MyGlobal(0)="record#1"
   ^MyGlobal(1)="record#2"
   ^MyGlobal(2)="record#3"

Retrieve a set of child data nodes from the DB Server

Status: Beta release.

   var result = mglobal.getchildnodes([<partialkey...>,], <{options}>);

Where:

options: A JSON object in which the following options can be defined:
- getdata: if true, return the child node data in addition to the key values (set to true or false, default: false).
- max: the maximum number of nodes to return. A default maximum of 100 records is automatically applied.
- start: start from this key value.
- end: finish at this key value.
- lock: if true, acquire a lock on the target DB data node before proceeding.
- locktimeout: timeout for the lock operation (in seconds).
- numerickeys: Treat the dataset as a numerically-keyed JavaScript array.

Child node data will re returned in an array, each element of which has the following form:

   {key: <key value>, data: <data value>}

If the getdata property is not set to true, a simple array of key values will be returned.

The following examples are based on the DB Server data set:

   ^MyGlobal("a","b",1)="record#1"
   ^MyGlobal("a","b",2)="record#2"
   ^MyGlobal("a","b",3)="record#3"
   ^MyGlobal("a","b",4,1)="record#4,1"
   ^MyGlobal("a","b","a")="record#a"

Example 1 (return all key values):

   global = new mglobal(db, 'MyGlobal');
   result =  global.getchildnodes("a", "b", {});

The following JavaScript array will be returned:

   result[0] = 1
   result[1] = 2
   result[2] = 3
   result[3] = 4
   result[4] = "a"

Example 2 (return all key values and the associated data):

   global = new mglobal(db, 'MyGlobal');
   result =  global.getchildnodes("a", "b", {getdata: true});

The following JavaScript array will be returned:

   result[0] = {key: 1, data: "record#1}
   result[1] = {key: 2, data: "record#2}
   result[2] = {key: 3, data: "record#3}
   result[3] = {key: 4}
   result[4] = {key: "a", data: "record#a}

Example 3 (specifying start and end key values):

   global = new mglobal(db, 'MyGlobal');
   result =  global.getchildnodes("a", "b", {getdata: true, start: 2, end: 3});

The following JavaScript array will be returned:

   result[1] = {key: 2, data: "record#2}
   result[2] = {key: 3, data: "record#3}

Example 4 (using numeric keys):

For the following dataset contained in the DB Server:

   ^MyGlobal(0)="record#1"
   ^MyGlobal(1)="record#2"
   ^MyGlobal(2)="record#3"
   ^MyGlobal(3,1)="record#3,1"

JavaScript:

   global = new mglobal(db, 'MyGlobal');
   result = global.getchildnodes({numerickeys: true});

This will return the following JavaScript array:

   result[0] = "record#1"
   result[1] = "record#2"
   result[2] = "record#3"
   result[3] = undefined

Note that JavaScript undefined is returned for nodes where there's no data defined at the current key-level, but data defined at a lower (i.e. descendant) key-level.

Data nodes are returned up to the point at which the key is undefined, or the value defined in the end option is reached (whichever comes first).

Cursor based data retrieval

The mcursor class.

This facility provides high-performance techniques for traversing records held in database globals.

Specifying the query

The first task is to specify the 'query' for the global traverse.

   query = new mcursor(db, {global: <global_name>, key: [<seed_key>]}[, {<options>}]);

The 'options' object can contain the following properties:

multilevel: A boolean value (default: multilevel: false). Set to 'true' to return all descendant nodes from the specified 'seed_key'.
getdata: A boolean value (default: getdata: false). Set to 'true' to return any data values associated with each global node returned.
format: Format for output (default: not specified). If the output consists of multiple data elements, the return value (by default) is a JavaScript object made up of a 'key' array and an associated 'data' value. Set to "url" to return such data as a single URL escaped string including all key values ('key[1->n]') and any associated 'data' value.

Example (return all keys and names from the 'Person' global):

   query = new mcursor(db, {global: "Person", key: [""]}, {multilevel: false, getdata: true});

Traversing the dataset

In key order:

   result = query.next();

In reverse key order:

   result = query.previous();

In all cases these methods will return 'null' when the end of the dataset is reached.

Example 1 (return all key values from the 'Person' global - returns a simple variable):

   query = new mcursor(db, {global: "Person", key: [""]});
   while ((result = query.next()) !== null) {
      console.log("result: " + result);
   }

Example 2 (return all key values and names from the 'Person' global - returns an object):

   query = new mcursor(db, {global: "Person", key: [""]}, {multilevel: false, getdata: true});
   while ((result = query.next()) !== null) {
      console.log("result: " + JSON.stringify(result, null, '\t'));
   }

Example 3 (return all key values and names from the 'Person' global - returns a string):

   query = new mcursor(db, {global: "Person", key: [""]}, {multilevel: false, getdata: true, format: "url"});
   while ((result = query.next()) !== null) {
      console.log("result: " + result);
   }

Example 4 (return all key values and names from the 'Person' global, including any descendant nodes):

   query = new mcursor(db, {global: "Person", key: [""]}, {multilevel: true, getdata: true});
   while ((result = query.next()) !== null) {
      console.log("result: " + JSON.stringify(result, null, '\t'));
   }

M programmers will recognise this last example as the M $Query() command.

Traversing the global directory (return a list of global names)

   query = new mcursor(db, {global: <seed_global_name>}, {globaldirectory: true});

Example (return all global names held in the current directory)

   query = new mcursor(db, {global: ""}, {globaldirectory: true});
   while ((result = query.next()) !== null) {
      console.log("result: " + result);
   }

Invocation of database functions

   result = db.function(<function>, <parameters>);

Example:

M routine called 'math':

   add(a, b) ; Add two numbers together
             quit (a+b)

JavaScript invocation:

  result = db.function("add^math", 2, 3);

Transaction Processing

M DB Servers implement Transaction Processing by means of the methods described in this section. When implementing transactions, care should be taken with JavaScript operations that are invoked asynchronously. All the Transaction Processing methods describe here can only be invoked synchronously.

Start a Transaction

   result = db.tstart(<parameters>);

At this time, this method does not take any arguments.
On successful completion this method will return zero, or an error code on failure.

Example:

   result = db.tstart();

Determine the Transaction Level

   result = db.tlevel(<parameters>);

At this time, this method does not take any arguments.
Transactions can be nested and this method will return the level of nesting. If no Transaction is active this method will return zero. Otherwise a positive integer will be returned to represent the current depth of Transaction nesting.

Example:

   tlevel = db.tlevel();

Commit a Transaction

   result = db.tcommit(<parameters>);

At this time, this method does not take any arguments.
On successful completion this method will return zero, or an error code on failure.

Example:

   result = db.tcommit();

Rollback a Transaction

   result = db.trollback(<parameters>);

At this time, this method does not take any arguments.
On successful completion this method will return zero, or an error code on failure.

Example:

   result = db.trollback();

Direct access to InterSystems classes (IRIS and Cache)

Invocation of a ClassMethod

   result = db.classmethod(<class_name>, <classmethod_name>, <parameters>);

Example (Encode a date to internal storage format):

   result = db.classmethod("%Library.Date", "DisplayToLogical", "10/10/2019");

Creating and manipulating instances of objects

The following simple class will be used to illustrate this facility.

   Class User.Person Extends %Persistent
   {
      Property Number As %Integer;
      Property Name As %String;
      Property DateOfBirth As %Date;
      Method Age(AtDate As %Integer) As %Integer
      {
         Quit (AtDate - ..DateOfBirth) \ 365.25
      }
   }

Create an entry for a new Person

   person = db.classmethod("User.Person", "%New");

Add Data:

   result = person.setproperty("Number", 1);
   result = person.setproperty("Name", "John Smith");
   result = person.setproperty("DateOfBirth", "12/8/1995");

Save the object record:

   result = person.method("%Save");

Retrieve an entry for an existing Person

Retrieve data for object %Id of 1.

   person = db.classmethod("User.Person", "%OpenId", 1);

Return properties:

   let number = person.getproperty("Number");
   let name = person.getproperty("Name");
   let dob = person.getproperty("DateOfBirth");

Calculate person's age at a particular date:

   let today = db.classmethod("%Library.Date", "DisplayToLogical", "10/10/2019");
   let age = person.method("Age", today);

Reusing an object container

Once created, it is possible to reuse containers holding previously instantiated objects using the reset() method. Using this technique helps to reduce memory usage in the JavaScript environment.

Example 1 Reset a container to hold a new instance:

   person.reset("User.Person", "%New");

Example 2 Reset a container to hold an existing instance (object %Id of 2):

   person.reset("User.Person", "%OpenId", 2);

Direct access to SQL: MGSQL and InterSystems SQL (IRIS and Cache)

mg-dbx-napi provides direct access to the Open Source MGSQL engine (https://github.com/chrisemunt/mgsql) and InterSystems SQL (IRIS and Cache).

Note: In order to use this facility the M support routines should be installed (%zmgsi and %zmgsis).

Specifying the SQL query

The first task is to specify the SQL query.

   query = new mcursor(db, {sql: <sql_statement>[, type: <sql_engine>]});

Or:

   query = db.sql({sql: <sql_statement>[, type: <sql_engine>]});

Example 1 (using MGSQL):

   query = db.sql({sql: "select * from person"});

Example 2 (using InterSystems SQL):

   query = db.sql({sql: "select * from SQLUser.person", type: "Cache"});

Execute an SQL query

   var result = <query>.execute();

The result of query execution is an object containing the return code and state and any associated error message. The familiar ODBC return and status codes are used.

Example 1 (successful execution):

   {
       "sqlcode": 0,
       "sqlstate": "00000",
       "columns": [
                     {
                        "name": "Number",
                        "type": "INTEGER"
                     },
                       "name": "Name",
                        "type": "VARCHAR"
                     },
                       "name": "DateOfBirth",
                        "type": "DATE"
                     }
                  ]
   }

Example 2 (unsuccessful execution):

   {
       "sqlcode": -1,
       "sqlstate": "HY000",
       "error": "no such table 'person'"
   }

Traversing the returned dataset (SQL 'select' queries)

In result-set order:

   result = query.next();

In reverse result-set order:

   result = query.previous();

In all cases these methods will return 'null' when the end of the dataset is reached.

Example:

   while ((row = query.next()) !== null) {
      console.log("row: " + JSON.stringify(result, null, '\t'));
   }

The output for each iteration is a row of the generated SQL result-set. For example:

   {
       "number": 1,
       "name": "John Smith",
   }

SQL cleanup

For 'select' queries that generate a result-set it is good practice to invoke the 'cleanup' method at the end to delete the result-set held in the database.

   var result = <query>.cleanup();

Reset an SQL container with a new SQL Query

   <query>.reset({sql: <sql_statement>[, type: <sql_engine>]);

JavaScript Superserver

Note: This facility is experimental at this time. Also, due to missing functionality in the Bun JavaScript engine, the JavaScript Superserver only works under Node.js.

mg-dbx-napi can connect to remote DB Servers over the network. Usually, this requires the use of a Superserver written in M-code installed within the DB Server. We now include a Superserver written purely in JavaScript (mgsi_node.mjs). The JavaScript Superserver resides on the same host as the DB Server.

mg-dbx-napi, running on the remote client, connects to this Superserver which then connects to the DB Server via its API. Using this facility removes the need for the M-based Superserver operating within the DB Server environment.

Required components:

On the client: mg_dbx_napi.mjs|ts and mg-dbx-napi.node
On the DB Server: mgsi_node.mjs and mg-dbx-napi.node

Starting the JavaScript Superserver:

   node mgsi_node.mjs <tcp_port>

Example (Superserver listening on TCP port 7777):

   node mgsi_node.mjs 7777

By default, the Superserver will only write initial start-up information to the console. If you wish to see evidence of all requests being processed, start the Superserver in 'verbose' mode:

   node mgsi_node.mjs 7777 verbose

On the client-side, the mg-dbx-napi open() method must be supplied with parameters for both API-based and network-based access. mg-dbx-napi will use the network-oriented parameters to first connect to the remote JavaScript Superserver which will then use the forwarded API-oriented parameters to connect to the DB Server.

Ordinarily, when both API-oriented and network-oriented parameters are supplied to the open() method, the API-oriented parameters take precedence. However, in this case we want the network-oriented parameters to take precedence in the first instance. To ensure that this happens, set the use parameter to 'tcp'.

Example (IRIS):

      var open = db.open({
           type: "YottaDB",
           use: "tcp",
           path:"/opt/IRIS20181/mgr",
           host: "localhost",
           tcp_port: 7777,
           username: "_SYSTEM",
           password: "SYS",
           namespace: "USER"
         });

Example (YottaDB):

       var open = db.open({
           type: "YottaDB",
           use: "tcp",
           path: "/usr/local/lib/yottadb/r138",
           host: "localhost",
           tcp_port: 7777,
           env_vars: {
             ydb_gbldir: '/root/.yottadb/r1.38_x86_64/g/yottadb.gld',
             ydb_routines: '/root/.yottadb/r1.38_x86_64/o*(/root/.yottadb/r1.38_x86_64/r /root/.yottadb/r) /usr/local/lib/yottadb/r138/libyottadbutil.so',
             ydb_ci: '/usr/local/lib/yottadb/r138/zmgsi.ci'
           }
         });

Background and History of This Package

Node.js was released in 2009 and is based on the Google V8 JavaScript engine. It has always been possible to extend the functionality of Node.js by creating add-on modules that work directly to the V8 C++ API. With Node.js version 8, the third iteration of a new C++ API was released - Node-API. This API is intrinsically part of Node.js and, as such, is independent of the underlying JavaScript implementation. A key design goal of this new API was that it should be Application Binary Interface (ABI) stable across versions of Node.js. In other words, it should not be necessary to recompile add-on modules based on Node-API when the underlying Node.js platform is upgraded to a new version. By contrast, Node.js add-ons based on the native V8 API need to be recompiled every time the underlying Node.js/V8 engine is upgraded.

2022 saw the release of a new server-oriented JavaScript engine in beta form - Bun. Bun is based on Apple's JavaScriptCore engine which is used in the Safari web browser. Bun promised better performance than Node.js - up to 3x faster has been claimed. In order to hit the ground running, Bun natively implements hundreds of Node.js APIs, including around 90% of the Node-API functions. Therefore, in theory, Bun should be able to use add-on modules developed for Node.js provided they are based on Node-API.

mg-dbx-napi is based on Node-API and is designed to work with both Node.js and Bun. It offers applications the same functionality as the earlier mg-dbx module for Node.js which is based on the native V8 API.

mg-dbx-napi provides us with the opportunity to explore a high-performance JavaScript engine (Bun) working with the high-performance databases provided by InterSystems and YottaDB but in a way that is compatible with our current mg-dbx add-on solution for Node.js.

Those familiar with mg-dbx will know that applications engage directly with the functionality provided by the mg-dbx.node add-on. With mg-dbx-napi.node, a JavaScript shim is supplied for both Node.js and Bun to provide a common application interface for the two JavaScript engines. mg-dbx-napi.node is itself an add-on written in C++ and manages the interface to the database and is common to both Node.js and Bun.

The Node.js shim is provided as a JavaScript module:

   MyNodeApplication.js -> mg_dbx_napi.js -> mg-dbx-napi.node

The Bun shim is provided as a TypeScript module:

   MyBunApplication.js -> mg_dbx_napi.ts -> mg-dbx-napi.node

However, the packaging included within this repository means that it will automatically detect and load the correct version of both the shim module and add-on module for your operating system and architecture.

License

http://www.mgateway.com
Email: [email protected]

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Release Notes

v1.0.1 (11 January 2023)

Initial Release

v1.0.2 (12 January 2023)

Remove the need to prefix global names with the '^' character for API-based connections to YottaDB.
Allow the environment variables required for binding to the YottaDB API to be specified as name/value pairs in a separate JSON object.

v1.0.2b (22 June 2023)

Documentation update.

v1.1.3 (25 August 2023)

Introduce support for invoking all database commands asynchronously.

v1.2.4 (10 November 2023)

Introduce support for the mcursor class

v1.3.5 (18 November 2023)

Introduce support for direct SQL access.
Correct a fault in the cursor operation to return a global directory listing from YottaDB.

v1.4.6 (8 December 2023)

Introduce native Unicode support for InterSystems DB Servers - as character set/encoding UTF-16.

v1.4.7 (4 February 2024)

Correct a regression in the processing of numeric values (regression introduced in v1.4.6).

v1.4.8 (12 March 2024)

Introduce a JavaScript based Superserver. This facility is currently only available for Node.js.

v1.4.9 (19 March 2024)

Remove the need for the M support routines (superserver code) to be installed in order to support the Merge command under YottaDB. If API-based connectivity to a local YottaDB database is selected then the Merge command will be executed via a series of YottaDB API calls.
Add a 'verbose' option to the JavaScript based Superserver.

v1.4.10 (17 May 2024)

Correct a fault in the InterSystems get and change namespace operations (db.namespace()).

v1.5.11 (7 October 2024)

Introduce support for the setchildnodes() and getchildnodes() methods.

v1.5.12 (9 October 2024)

Include an option to lock the target global for the setchildnodes() and getchildnodes() methods.
For cases where data doesn't exist for a DB node, the getchildnodes() method will not create a 'data' property.

v1.5.13 (16 October 2024)

Introduce the concept of numerickeys for the setchildnodes() and getchildnodes() methods.
- This is a facility for storing and retrieving (in or from the DB Server) data held in numerically-keyed JavaScript arrays.
Allow multiple key values to be specified (as a JavaScript array) in the setchildnodes() method.
Include a minor optimisation for the YottaDB implementation of the getchildnodes() method.

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

mg-dbx-napi

What Is mg-dbx-napi?

Installing mg-dbx-napi

Using mg-dbx-napi

Supported Databases

Installing the M support routines (also known as the DB Superserver)

Installation for InterSystems Cache/IRIS

Installation for YottaDB

Starting the DB Superserver (for network based connectivity only)

Connecting to the database

Open a connection to the database

InterSystems Cache

API based connectivity

Network based connectivity

InterSystems IRIS

API based connectivity

Network based connectivity

YottaDB

API based connectivity

Network based connectivity

Additional (optional) properties for the open() method

Return the version of mg-dbx-napi

Returning (and optionally changing) the current directory (or Namespace)

Returning (and optionally changing) the current character set

Setting (or resetting) the timeout for the connection

Get the error message associated with the previous database operation

Close database connection

Invocation of database commands

Set a record

Get a record

Delete a record

Check whether a record is defined

Parse a set of records (in order)

Parse a set of records (in reverse order)

Increment the value of a global node

Lock a global node

Unlock a (previously locked) global node

Merge (or copy) part of one global to another

Create a set of child data nodes in the DB Server

Retrieve a set of child data nodes from the DB Server

Cursor based data retrieval

Specifying the query

Traversing the dataset

Traversing the global directory (return a list of global names)

Invocation of database functions

Transaction Processing

Start a Transaction

Determine the Transaction Level

Commit a Transaction

Rollback a Transaction

Direct access to InterSystems classes (IRIS and Cache)

Invocation of a ClassMethod

Creating and manipulating instances of objects

Create an entry for a new Person

Retrieve an entry for an existing Person

Reusing an object container

Direct access to SQL: MGSQL and InterSystems SQL (IRIS and Cache)

Specifying the SQL query

Execute an SQL query

Traversing the returned dataset (SQL 'select' queries)

SQL cleanup

Reset an SQL container with a new SQL Query

JavaScript Superserver

Background and History of This Package

License

Release Notes

v1.0.1 (11 January 2023)

v1.0.2 (12 January 2023)

v1.0.2b (22 June 2023)

v1.1.3 (25 August 2023)

v1.2.4 (10 November 2023)

v1.3.5 (18 November 2023)

v1.4.6 (8 December 2023)

v1.4.7 (4 February 2024)