@totemstan/debe

v1.43.0

Published

2 years ago

built on totem - does everything but eat

Downloads

0High
0Medium
0Low

acmesds999

DEBE

Extends the TOTEM web service to provide the following endpoints:

DATASET.TYPE ? QUERY
NOTEBOOK.TYPE ? QUERY
AREA/PATH/FILE.TYPE ? QUERY
COMMAND.TYPE ? QUERY

to access DATASETs, NOTEBOOKs, FILEs and COMMANDs per Totem's API and TOTEM's skinning guide.

Use TYPE to convert a DATASET:

db | xml | csv | txt | flat | kml | html | json

inspect a DATASET:

tree | schema | nav | delta | view | blog

render a NOTEBOOK:

view | run | plugin | pivot | site | spivot | brief | gridbrief | pivbrief | runbrief

probe a NOTEBOOK:

exe | tou | md | status | suitors | usage | reset | EVENTS

manage a NOTEBOOK:

import | export | publish | addkey | subkey

license a NOTEBOOK:

js | py | m | me | jade | ...

DEBE also provides the following service COMMANDs:

agent | alert | ingest | riddle | task | ping

to distribute jobs, alert clients, ingest data, validate sessions, spread tasks, and test connections, and provides several FILE areas:

stores | uploads | shares | west | east | jades

for supervised/unsupervised file sharing.

Manage

npm install @totemstan/debe		# install
npm run start [ ? | $ | ...]	# Unit test
npm run verminor				# Roll minor version
npm run vermajor				# Roll major version
npm run redoc					# Regen documentation

npm run	startdbs				# Start required database servers
npm run setprot					# Configure for protected mode
npm run setdebug				# Configure for debugging mode
npm run setoper					# Configure for operational mode
npm run setprod					# Configure for production mode

npm run genio						# Generate totem-man.github.io from jades/totemblog
npm run raster --in=src --out=tar	# Rasterize source url into a target file

Usage

Acquire, optionally configure and start a DEBE server:

require("debe").config({
	key: value, 						// set key
	"key.key": value, 					// indexed set
	"key.key.": value					// indexed append
}, sql => {
	console.log( sql ? "look mom - Im running!" : "something evil is lurking" );
});

where configuration keys follow ENUMS deep copy conventions.

Program Reference

String

String
- ~linkify(ref) ⇐ Array
- ~mailify()
- ~align()
- ~trimGoogle()

String~linkify(ref) ⇐ Array

Returns a ref-joined list of links

Kind: inner method of String
Extends: Array

| Param | Type | | --- | --- | | ref | String |

String~mailify()

Kind: inner method of String

String~align()

Kind: inner method of String

String~trimGoogle()

Kind: inner method of String

Array

Array

Array~gridify(noheader)

Creates an html table from an array.

Kind: inner method of Array

| Param | Type | Description | | --- | --- | --- | | noheader | Boolean | switch to enable header processing |

Array~groupify(dot)

Groups each "x.y.z. ...." spec in the list.

Kind: inner method of Array

| Param | Type | Description | | --- | --- | --- | | dot | string | item seperator |

Array~blog(keys, ds, cb)

Blogs each string in the list.

Kind: inner method of Array
See: totem:blogify

| Param | Type | Description | | --- | --- | --- | | keys | List | list of keys to blog | | ds | String | Name of dataset being blogged | | cb | function | callback(recs) blogified version of records |

Array~merge(Recs, idx)

Merge changes when doing table deltas from their baseline versions.

Kind: inner method of Array

| Param | Type | Description | | --- | --- | --- | | Recs | Array | Source records to merge into this records | | idx | String | Key name to use for detecting record changes |

Array~schemaify(src)

Returns a schema of the array using the specified src path.

Kind: inner method of Array

| Param | Type | Description | | --- | --- | --- | | src | String | path to source |

Array~treeify(idx, kids, level, keys, wt)

Returns a tree = {name,weight,nodes} from records having been sorted on keys=[key,...]

Kind: inner method of Array

| Param | Type | Description | | --- | --- | --- | | idx | Number | starting index (0 on first call) | | kids | Number | number of leafs following starting index (this.length on first call) | | level | Number | current depth (0 on first call) | | keys | Array | pivot keys | | wt | String | key name that contains leaf weight (defaults to "size") |

Array~joinify(cb)

Joins a list with an optional callback cb(head,list) to join the current list with the current head.

Kind: inner method of Array

| Param | Type | | --- | --- | | cb | function |

Example

[	a: null,
		g1: [ b: null, c: null, g2: [ x: null ] ],
		g3: [ y: null ] ].joinify()

returning a string
	"a,g1(b,c,g2(x)),g3(y)"

Data

DEBE

Provides UI interfaces to the barebone TOTEM web service to support notebooks and other entities. This module documented in accordance with jsdoc.

Env Dependencies

HOSTNAME = name of host machine
REPO = http://DOMAIN:ACCOUNT
JIRA = http://DOMAIN
RAS = http://DOMAIN
BY = https://DOMAIN

Requires: module:totem, module:atomic, module:geohack, module:man, module:randpr, module:enums, module:reader, module:skin, module:blog, module:dogs, module:pipe, module:crypto, module:child_process, module:fs, module:stream, module:cluster, module:repl, module:i18n-abide, module:optimist, module:tokml, module:officegen
Author: ACMESDS
Example

// npm test D2
// Start challenge-protected server with additional byTable-routed entpoints.

config({
	riddles: 10,
	"byTable.": {
		wfs: function (req,res) {
			res("here i go again");

			Fetch(ENV.WFS_TEST, data => {
				Trace(data);
			});
		}
	}
}, sql => {
	Trace( "This bad boy in an encrypted service with a database and has an /wfs endpoint" );
});

Example

// npm test D3
// Start server using default config

config({
}, sql => {
	Trace( "Stateful network flow manger started" );
});

Example

// npm test D4
// Start server and prep file system

function readFile(sql, path, cb) {
	sql.beginBulk();
	readers.xls( "./config.stores/test.xls", rec => { 
		if (rec) 
			cb(rec,sql);

		else 
			sql.endBulk();
	});
}

config({
}, sql => {
	var recs = 0, now = new Date();
	readFile( sql, "./config.stores/test.xls", (rec,sql) => {
		if ( ++recs<5 ) {
			var 
				doc = (rec.doc || rec.Doc || rec.report || rec.Report || "")
						.replace( /\n/g, " ")
						.replace( /\&\#10;/g, " "),

				docs = doc				
						.match( /(.*)TEXT:(.*)COMMENTS:(.*)/ ) || [ "", "", doc, ""],

				text = "";

			docs[2].replace( /\.  /g, "\n").replace( /^[0-9 ]*\. \(.*\) (.*)/gm, (str,txt) => text += txt + ".  " );

			sql.query("INSERT INTO openv.docs SET ?", {
				Reported: rec.reported || rec.Reported || now,
				Name: rec.reportID || ("tbd-"+recs),
				Pipe: JSON.stringify( text )
			}, err => Trace("add", err) );
		}
	});
});

DEBE

DEBE~$libs

Kind: inner property of DEBE

~$libs

$libs.$site

Kind: static property of $libs

$libs.$notebooks

Kind: static property of $libs

$libs.$

See man

Kind: static property of $libs

$libs.$log

See debe

Kind: static property of $libs

$libs.$task

See debe

Kind: static property of $libs

$libs.$sql

See jsdb

Kind: static property of $libs

$libs.$neo

See jsdb

Kind: static property of $libs

$libs.$copy

See enums

Kind: static property of $libs

$libs.$each

See enums

Kind: static property of $libs

$libs.$fetch()

Kind: static method of $libs

$libs.$get()

Kind: static method of $libs

$libs.$help()

Kind: static method of $libs

DEBE~tableRoutes.

Route table to a database according to security requirements.

Kind: inner property of DEBE

DEBE~defaultDocs

Default doc for reserved notebook keys

Kind: inner property of DEBE

DEBE~licenseOnDownload

Kind: inner property of DEBE

DEBE~filters.

Filter dataset recs on specifed req-res thread

Kind: inner property of DEBE

~filters.

filters..xdoc

Kind: static property of filters.

| Param | Type | Description | | --- | --- | --- | | recs | Array | Records to filter | | req | Object | Totem session request | | res | function | Totem session response |

filters..xxls

Kind: static property of filters.

| Param | Type | Description | | --- | --- | --- | | recs | Array | Records to filter | | req | Object | Totem session request | | res | function | Totem session response |

filters..xpps

Kind: static property of filters.

| Param | Type | Description | | --- | --- | --- | | recs | Array | Records to filter | | req | Object | Totem session request | | res | function | Totem session response |

filters..xppt

Kind: static property of filters.

| Param | Type | Description | | --- | --- | --- | | recs | Array | Records to filter | | req | Object | Totem session request | | res | function | Totem session response |

filters..dbx(recs, req, res)

Kind: static method of filters.

| Param | Type | Description | | --- | --- | --- | | recs | Array | Records to filter | | req | Object | Totem session request | | res | function | Totem session response |

filters..db(recs, req, res)

Kind: static method of filters.

| Param | Type | Description | | --- | --- | --- | | recs | Array | Records to filter | | req | Object | Totem session request | | res | function | Totem session response |

filters..kml(recs, req, res)

Kind: static method of filters.

| Param | Type | Description | | --- | --- | --- | | recs | Array | Records to filter | | req | Object | Totem session request | | res | function | Totem session response |

filters..flat(recs, req, res)

Kind: static method of filters.

| Param | Type | Description | | --- | --- | --- | | recs | Array | Records to filter | | req | Object | Totem session request | | res | function | Totem session response |

filters..txt(recs, req, res)

Kind: static method of filters.

| Param | Type | Description | | --- | --- | --- | | recs | Array | Records to filter | | req | Object | Totem session request | | res | function | Totem session response |

filters..html(recs, req, res)

Kind: static method of filters.

| Param | Type | Description | | --- | --- | --- | | recs | Array | Records to filter | | req | Object | Totem session request | | res | function | Totem session response |

filters..tree(recs, req, res)

Kind: static method of filters.

| Param | Type | Description | | --- | --- | --- | | recs | Array | Records to filter | | req | Object | Totem session request | | res | function | Totem session response |

filters..schema(recs, req, res)

Kind: static method of filters.

| Param | Type | Description | | --- | --- | --- | | recs | Array | Records to filter | | req | Object | Totem session request | | res | function | Totem session response |

DEBE~byArea.

/AREA/FILE-endpoint routers

Kind: inner property of DEBE

byArea..root(req, res)

Default area navigator.

Kind: static method of byArea.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

DEBE~byTable.

/TABLE-endpoint routers

Kind: inner property of DEBE

~byTable.

byTable..uploads

Upload files to upload area

Kind: static property of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..stores

Upload files to stores area

Kind: static property of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..search(req, res)

Search for a file

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..searches(req, res)

Search of multiple files

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..words(req, res)

Word statistics

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..wms(req, res)

WMS

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..wfs(req, res)

WFS

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..tips(req, res)

Provide image tips.

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..follow(req, res)

Track web links.

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..proctor(req, res)

Proctor quizes.

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..likeus(req, res)

Update like-us stats

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..users(req, res)

Return list of clients that have used this service

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..graphs(req, res)

Retrieve requested neo4j graph.

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..notebooks(req, res)

Return published notebooks

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..ingest(req, res)

Endpoint to ingest a source into the sql database

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..decode(req, res)

Endpoint to return release information about requested license.

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..restart(req, res)

Endpoint to restart totem if authorized.

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..alert(req, res)

Endpoint to send notice to all clients

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..stop(req, res)

Endpoint to send emergency message to all clients then halt totem

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..devstatus(req, res)

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..milestones(req, res)

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..config(req, res)

Configure DEBE/TOTEM.

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..info(req, res)

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..DG(req, res)

Digital globe interface.

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..HYDRA(req, res)

Hydra interface.

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..NCL(req, res)

NCL interface.

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..ESS(req, res)

ESS interface.

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..MIDB(req, res)

MIDB interface.

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..matlab(req, res)

Matlab interface.

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

byTable..ESC(req, res)

ESC remedy interface.

Kind: static method of byTable.

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

DEBE~byType.

/TABLE.TYPE-endpoint routers

Kind: inner property of DEBE

DEBE~site.

Site skinning context

Kind: inner property of DEBE

DEBE~errors.

Error messages

Kind: inner property of DEBE

DEBE~paths.

Paths to things

Kind: inner property of DEBE

DEBE~probono : boolean

Enable to give-away plugin services

Kind: inner property of DEBE

DEBE~isSpawned : Boolean

Enabled when this is child server spawned by a master server

Kind: inner property of DEBE

DEBE~bySOAP : Object

Reserved for soap interfaces

Kind: inner property of DEBE

DEBE~linkInspect()

Inspect doc - kludge i/f to support nlp project

Kind: inner method of DEBE

DEBE~licenseCode()

License notebook engine code.

Kind: inner method of DEBE

DEBE~sendMail()

Kind: inner method of DEBE

DEBE~initialize(sql, init)

Initialize DEBE on startup.

Kind: inner method of DEBE

| Param | Type | Description | | --- | --- | --- | | sql | Object | MySQL connector | | init | function | callback(sql) when service init completed |

DEBE~SOAPsession(req, res, proxy)

Process an bySOAP session peer-to-peer request. Currently customized for Hydra-peer and could/should be revised to support more generic peer-to-peer bySOAP interfaces.

Kind: inner method of DEBE

| Param | Type | Description | | --- | --- | --- | | req | Object | HTTP request | | res | Object | HTTP response | | proxy | function | Name of APP proxy function to handle this session. |

DEBE~genDoc(recs, req, res)

Convert records to requested req.type office file.

Kind: inner method of DEBE

| Param | Type | Description | | --- | --- | --- | | recs | Array | list of records to be converted | | req | Object | Totem session request | | res | function | Totem session response |

DEBE~setAutorun()

Kind: inner method of DEBE

DEBE~exeAutorun()

Kind: inner method of DEBE

DEBE~getEngine()

Kind: inner method of DEBE

DEBE~fileUpload()

Kind: inner method of DEBE

DEBE~savePage()

Kind: inner method of DEBE

DEBE~statusPlugin()

Kind: inner method of DEBE

DEBE~matchPlugin()

Kind: inner method of DEBE

DEBE~docPlugin()

Kind: inner method of DEBE

DEBE~trackPlugin()

Kind: inner method of DEBE

DEBE~getPlugin()

Kind: inner method of DEBE

DEBE~simPlugin()

Kind: inner method of DEBE

DEBE~usersPlugin(req, res)

Endpoint to return users of a requested plugin/notebook/table.

Kind: inner method of DEBE

| Param | Type | Description | | --- | --- | --- | | req | Object | http request | | res | function | Totem session response callback |

DEBE~exportPlugin(req, res)

Endpoint to export requested plugin/notebook/table.

Kind: inner method of DEBE

| Param | Type | Description | | --- | --- | --- | | req | Object | http request | | res | function | Totem session response callback |

DEBE~importPlugin(req, res)

Endpoint to import requested plugin/notebook/table.

Kind: inner method of DEBE

| Param | Type | Description | | --- | --- | --- | | req | Object | http request | | res | function | Totem session response callback |

DEBE~exePlugin(req, res)

Endpoint to execute plugin req.table using usecase req.query.ID || req.query.Name.

Kind: inner method of DEBE

| Param | Type | Description | | --- | --- | --- | | req | Object | http request | | res | function | Totem session response callback |

DEBE~modifyPlugin(req, res)

Endpoint to add keys to requested plugin/notebook/table.

Kind: inner method of DEBE

| Param | Type | Description | | --- | --- | --- | | req | Object | http request | | res | function | Totem session response callback |

DEBE~retractPlugin(req, res)

Endpoint to remove keys from requested plugin/notebook/table given.

Kind: inner method of DEBE

| Param | Type | Description | | --- | --- | --- | | req | Object | http request | | res | function | Totem session response callback |

DEBE~helpPlugin(req, res)

Endpoint to return plugin/notebook/table usage info.

Kind: inner method of DEBE

| Param | Type | Description | | --- | --- | --- | | req | Object | http request | | res | function | Totem session response callback |

DEBE~runPlugin(req, res)

Endpoint to run a dataset-engine plugin named X = req.table using parameters Q = req.query or (if Q.id or Q.name specified) dataset X parameters derived from the matched
dataset (with json fields automatically parsed). On running the plugin's engine X, this method then responds on res(results). If Q.Save is present, the engine's results are also saved to the plugins dataset. If Q.Pipe is present, then responds with res(Q.Pipe), thus allowing the caller to place the request in its job queues. Otherwise, if Q.Pipe vacant, then responds with res(results). If a Q.agent is present, then the plugin is out-sourced to the requested agent, which is periodically polled for its results, then responds with res(results).

Kind: inner method of DEBE

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

DEBE~getCert(req, res)

Endpoint to create/return public-private certs of given url query

Kind: inner method of DEBE

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem session request | | res | function | Totem session response |

Contacting, Contributing, Following

Feel free to

submit and status TOTEM issues
contribute to TOTEM notebooks
revise TOTEM requirements
browse TOTEM holdings
or follow TOTEM milestones

License

MIT

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

DEBE

Manage

Usage

Program Reference

String

String~linkify(ref) ⇐ Array

String~mailify()

String~align()

String~trimGoogle()

Array

Array~gridify(noheader)

Array~groupify(dot)

Array~blog(keys, ds, cb)

Array~merge(Recs, idx)

Array~schemaify(src)

Array~treeify(idx, kids, level, keys, wt)

Array~joinify(cb)

Data

DEBE

Env Dependencies

DEBE~$libs

$libs.$site

$libs.$notebooks

$libs.$

$libs.$log

$libs.$task

$libs.$sql

$libs.$neo

$libs.$copy

$libs.$each

$libs.$fetch()

$libs.$get()

$libs.$help()

DEBE~tableRoutes.

DEBE~defaultDocs

DEBE~licenseOnDownload

DEBE~filters.

filters..xdoc

filters..xxls

filters..xpps

filters..xppt

filters..dbx(recs, req, res)

filters..db(recs, req, res)

filters..kml(recs, req, res)

filters..flat(recs, req, res)

filters..txt(recs, req, res)

filters..html(recs, req, res)

filters..tree(recs, req, res)

filters..schema(recs, req, res)

DEBE~byArea.

byArea..root(req, res)

DEBE~byTable.

byTable..uploads

byTable..stores

byTable..search(req, res)

byTable..searches(req, res)

byTable..words(req, res)

byTable..wms(req, res)

byTable..wfs(req, res)

byTable..tips(req, res)

byTable..follow(req, res)

byTable..proctor(req, res)

byTable..likeus(req, res)

byTable..users(req, res)

byTable..graphs(req, res)

byTable..notebooks(req, res)

byTable..ingest(req, res)

byTable..decode(req, res)

byTable..restart(req, res)

byTable..alert(req, res)

byTable..stop(req, res)

byTable..devstatus(req, res)

byTable..milestones(req, res)

byTable..config(req, res)