ENUMS

Provides methods to clock, fetch, enumerate, stream, get, and regulate data. Also makes connections to required services: mysql, neo4j, txmail, rxmail.

Manage

npm install @totemstan/enums	# install
npm run start [ ? | $ | ...]	# Unit test
npm run verminor				# Roll minor version
npm run vermajor				# Roll major version
npm run redoc					# Regen documentation

Usage

Acquire and optionally configure ENUMS as follows:

const { Copy, Each, Extend, Stream, Fetch, ... } = require("@totemstan/enums").config({
	key: value, 						// set key
	"key.key": value, 					// indexed set
	"key.key.": value					// indexed append
});

where configuration keys follow ENUMS deep copy conventions.

Program Reference

ENUMS

Provides methods to clock, fetch, enumerate, regulate, stream, fetch and get data. This module documented in accordance with jsdoc.

Env Dependencies

FETCH_PASS = password for local fetching cert
URL_LEXNEX = URL to lexis-nexis service w user/password credentials
URL_MYSQL = URL to mysql service w user/password credentials
URL_NEO4J = URL to neo4j service w user/password credentials
URL_TXMAIL = URL to smtp email service w user/password credentials
URL_RXMAIL = URL to imap email service w user/password credentials
URL_RSSFEED = URL to rss service w user/password credentials
URL_LEXNEX = URL to lexis-nexis service w user/password credentials

Requires: module:os, module:cluster, module:fs, module:http, module:https, module:vm, module:cp, module:crypto, module:stream, module:mysql, module:neo4j-driver, module:nodemailer, module:nodemailer-smtp-transport
Author: ACMESDS

• ENUMS
  • .mysqlOpts
  • .neo4jOpts
  • .rxmailOpts
  • .txmailOpts
  • .escapeId
  • .escape
  • .Log
  • .sites
  • .maxFiles
  • .maxRetry
  • .certs
  • .Start(host, cbs)
  • .Trace()
  • .config(opts)
  • .typeOf()
  • .getList()
  • .Copy(src, tar, deep) ⇒ Object
  • .Each(A, cb)
  • .Regulate(opts, taskcb(recs), [feedcb(step)]) ⇒ Clock
  • .Fetch(ref, [cb], [cb])

ENUMS.mysqlOpts

Kind: static property of ENUMS

ENUMS.neo4jOpts

Kind: static property of ENUMS

ENUMS.rxmailOpts

Kind: static property of ENUMS

ENUMS.txmailOpts

Kind: static property of ENUMS

ENUMS.escapeId

Kind: static property of ENUMS

ENUMS.escape

Kind: static property of ENUMS

ENUMS.Log

Kind: static property of ENUMS

ENUMS.sites

Fetch quick SITEREFs

https://www.programmableweb.com/search/military

ACLED
https://www.programmableweb.com/api/acled-rest-api-v25
ACCT [email protected] / ACLEDsnivel1
API https://developer.acleddata.com/rehd/cms/views/acled_api/documents/API-User-Guide.pdf
SITE https://developer.acleddata.com/
The Armed Conflict Location & Event Data Project (ACLED) is a real-time data and and crisis analysis and mapping project on political violence and protest across the world. ACLED's mission is to produce dis-aggregated, locally informed data and analysis globally in real time. An ACLED REST API enables users to retrieve data about actors, actor type, country, region and get data in JSON, XML, CSV or text. Filter data by date, area, pagination, etc.

Animetrics FIMS
https://www.programmableweb.com/api/animetrics-fims-cloud-rest-api
http://animetrics.com/fims-cloud
Aimed at the law enforcement, security and military intelligence industries, Animetrics' FaceR Identity Management Solution (FIMS) allows organizations to bring mobile security and video surveillance facial-biometrics applications into the field for use in real time. FIMS Cloud is used to centralize and access a user's cloud based photographic stores using facial recognition. FIMS Cloud organizes, searches and centralizes access to photographic stores using 1:many web service based verification engine. Access to the service is provided via a RESTful API. Public documentation is not available.

Navlost WXT Weather Tesseract
https://www.programmableweb.com/api/navlost-wxt-weather-tesseract
The WXT Weather Service provides atmospheric weather information through a REST architecture, HTTP requests, and JSON formats. It integrates METAR/TAF information, sun, and moon calculations, targeting aviation and energy applications. Advanced features include: -Upper atmosphere information (e.g., research, aviation, rocketry, military) -Automated, push-type notification of arbitrary weather-related events (alert service) -Calculation of arbitrary results derived from weather forecast information via a server-side scripting language. The default response type is application/json, although other formats may be supported. At the present time, there is partial support for comma-separated value (CSV) responses.
https://wxt.navlost.eu/api/v1/
https://wxt.navlost.eu/doc/api/

Kind: static property of ENUMS

ENUMS.maxFiles

Max files to Fetch when indexing a folder

Kind: static property of ENUMS

ENUMS.maxRetry

Fetch wget/curl maxRetry

Kind: static property of ENUMS

ENUMS.certs

Legacy Fetching certs

Kind: static property of ENUMS

ENUMS.Start(host, cbs)

Start the command prompter for the hosting module with callback to initializer.

Kind: static method of ENUMS

| Param | Type | Description | | --- | --- | --- | | host | String | Name of hosting module | | cbs | Array | List of initializer callbacks (ctx defines context to run command) |

ENUMS.Trace()

Trace log message and args.

Kind: static method of ENUMS

ENUMS.config(opts)

Configure enums

Kind: static method of ENUMS

| Param | Type | Description | | --- | --- | --- | | opts | Object | options |

ENUMS.typeOf()

Test an object x:

isString(x), isDate(x), isFunction(x), isArray(x), isObject(x)
isEmpty(x), isNumber(x), isKeyed(x), isBoolean(x), isBuffer(x)
isError(x)

Kind: static method of ENUMS

ENUMS.getList()

Kind: static method of ENUMS

ENUMS.Copy(src, tar, deep) ⇒ Object

Copy source hash src to target hash tar. If the copy is shallow (deep = false), a Copy({...}, {}) is equivalent to new Object({...}). In a deep copy, (e.g. deep = "."), src keys are treated as keys into the target thusly:

{	
	A: value,			// sets target[A] = value

	"A.B.C": value, 	// sets target[A][B][C] = value

	"A.B.C.": {			// appends X,Y to target[A][B][C]
		X:value, Y:value, ...
	},	

	OBJECT: [ 			// prototype OBJECT (Array,String,Date,Object) = method X,Y, ...
		function X() {}, 
		function Y() {}, 
	... ]

}

Kind: static method of ENUMS
Returns: Object - target hash

| Param | Type | Description | | --- | --- | --- | | src | Object | source hash | | tar | Object | target hash | | deep | String | copy key |

ENUMS.Each(A, cb)

Enumerate Object A over its keys with callback cb(key,val).

Kind: static method of ENUMS

| Param | Type | Description | | --- | --- | --- | | A | Object | source object | | cb | function | callback (key,val) |

ENUMS.Regulate(opts, taskcb(recs), [feedcb(step)]) ⇒ Clock

Regulate a task defined by options opts

every 	= "NUM sec||min||hr||... "
start	= DATE  
end		= DATE  
on		= NUM  
off		= NUM  
util	= NUM  
batch	= INT  
limit	= INT  

with callbacks to

taskcb( recs ) to process a `recs`-batch
feedcb( step ) to feed the queue via `step(recs)` 

When a feedcb is provided, the mandatory taskcb is placed into a buffered workflow that terminates when the recs-batch goes null.

If no feedcb is provided, the taskcb is periodically executed (i.e. in an unbuffered workflow) using null recs-batches.

The regulated task is monitored/managed by the supplied options (defaults)

task 	= notebook being regulated ("task")
name	= usecase being regulated ("case")
client	= task owner ("system")
watch	= QoS task watchdog timer [s]; 0 disables (60)

A nonzero QoS sets a tasking watchdog timer to manage the task. A credit deficient client is signalled by calling feedcb(null).

To establish the task as a proposal, set Sign0 = 1 in the taskDB: in so doing, if Sign1 , ... are not signed-off (eg not approved by a task oversight commitee) before the proposal's start time, the task will be killed.

Regulate uses the following DBs:

userDB = openv.profiles client credit/billing information
taskDB = openv.queues tasking/billing information
snapDB = openv.<task> holds the task snapshot context

Kind: static method of ENUMS
Returns: Clock - Clock built for regulation options

| Param | Type | Description | | --- | --- | --- | | opts | Object | Task regulation options hash | | taskcb(recs) | function | Process record batch recs | | [feedcb(step)] | function | Feed a record batch using step(recs) |

ENUMS.Fetch(ref, [cb], [cb])

GET (PUT || POST || DELETE) information from (to) a ref url

PROTOCOL://HOST/FILE ? QUERY & FLAGS
SITEREF

given a cb callback function (or a data Array || Object || null).

The ref url specifies a PROTOCOL

http(s) 	=	http (https) protocol
curl(s) 	=	curl (curls uses certs/fetch.pfx to authenticate)
wget(s)		=	wget (wgets uses certs/fetch.pfx to authenticate)
mask 		=	http access via rotated proxies
file		=	file or folder path
notebook	=	selected notebook record
lexnex 		=	Lexis-Nexis oauth access to documents

When a file path is "/"-terminated, a folder index is returned. File paths may also contain wild-* cards.

Use the FLAGS

_every 	= "NUM sec||min||hr||..."
_start	= DATE  
_end	= DATE  
_on		= NUM  
_off	= NUM  						
_util	= NUM  

to regulate the fetch in a job queue with callbacks to the cb task. Use the FLAGS

_watch	= NUM  
_task 	= "job task name"
_name	= "job case name"
_client = "job owner"

to monitor the task in the job queues. Use the FLAGS

_batch	= NUM
_limit	= NUM
_keys	= [...]
_comma	= "delim"
_newline= "delim"

to read a csv-file and feed record batches to the cb callback.

Kind: static method of ENUMS

| Param | Type | Description | | --- | --- | --- | | ref | String | source URL | | [cb] | string | array | function | null | callback or data | | [cb] | function | optional callback when first cb is data |

Example

Fetch( ref, text => {			// get request
})

Example

Fetch( ref, [ ... ], stat => { 	// post request with data hash list
})

Example

Fetch( ref, { ... }, stat => { 	// put request with data hash
})

Example

Fetch( ref, null, stat => {		// delete request 
})

Array

• Array
  • ~stream()
  • ~Extend()
  • ~serial(fetch, cb)
  • ~any(cb) ⇒
  • ~all(cb) ⇒
  • ~get(index, ctx) ⇒ Object

Array~stream()

Kind: inner method of Array

Array~Extend()

Kind: inner method of Array

Array~serial(fetch, cb)

Serialize an Array to the callback cb(rec,info) or cb(null,stack) at end given a sync/async fetcher( rec, res ).

Kind: inner method of Array

| Param | Type | Description | | --- | --- | --- | | fetch | function | Callback to fetch the data sent to the cb | | cb | function | Callback to process the fetched data. |

Array~any(cb) ⇒

Kind: inner method of Array
Returns: this

| Param | Type | Description | | --- | --- | --- | | cb | function | Callback(arg,idx) |

Array~all(cb) ⇒

Kind: inner method of Array
Returns: this

| Param | Type | Description | | --- | --- | --- | | cb | function | Callback(arg,idx) |

Array~get(index, ctx) ⇒ Object

Index an array using a indexor:

string of the form "to=from & to=eval & to & ... & !where=eval"
hash of the form {to: from, ...}
callback of the form (idx,array) => { ... }

The "!where" clause returns only records having a nonzero eval.

Kind: inner method of Array
Returns: Object - Indexed data

| Param | Type | Description | | --- | --- | --- | | index | String | Object | function | Indexer | | ctx | Object | Context of functions etc |

Example

[{x:1,y:2},{x:10,y:20}].get("u=x+1&v=sin(y)&!where=x>5",Math)
{ u: [ 11 ], v: [ 0.9129452507276277 ] }

Example

[{x:1,y:2},{x:10,y:20}].get("x")
{ x: [ 1, 10 ] }

Example

[{x:1,y:2},{x:10,y:20}].get("x&mydata=y")
{ mydata: [ 2, 20 ], x: [ 1, 10 ] }

Example

[{x:1,y:2},{x:10,y:20}].get("mydata=[x,y]")
{ mydata: [ [ 1, 2 ], [ 10, 20 ] ] }

Example

[{x:1,y:2},{x:10,y:20}].get("mydata=x+1")
{ mydata: [ 2, 11 ] }

Example

[{x:1,y:2},{x:10,y:20}].get("",{"!all":1})
{ x: [ 1, 10 ], y: [ 2, 20 ] }

Example

[{x:1,y:2},{x:10,y:20}].get("")
[ { x: 1, y: 2 }, { x: 10, y: 20 } ]

Example

[{x:1,y:2},{x:10,y:20}].get("u")
{ u: [ undefined, undefined ] }

Example

[[1,2,3],[10,20,30]].get("1&0")
{ '0': [ 1, 10 ], '1': [ 2, 20 ] }	

String

• String
  • ~replaceSync()
  • ~tag(el, at) ⇒ String
  • ~parseEval($)
  • ~parseJS(ctx)
  • ~parse$(query)
  • ~parseJSON(def)
  • ~parsePath(query, index, flags, where) ⇒ Array
  • ~chunkFile(path, opts, {Function))
  • ~parseFile(path, opts, cb)
  • ~streamFile(path, opts, cb)
  • ~trace(msg, req, res)
  • ~serial()
  • ~get()

String~replaceSync()

Kind: inner method of String

String~tag(el, at) ⇒ String

Tag url with specified attributes.

Kind: inner method of String
Returns: String - tagged results

| Param | Type | Description | | --- | --- | --- | | el | String | tag html element or one of "?&/:=" | | at | String | tag attributes = {key: val, ...} |

String~parseEval($)

Parse "$.KEY" || "$[INDEX]" expressions given $ hash.

Kind: inner method of String

| Param | Type | Description | | --- | --- | --- | | $ | Object | source hash |

String~parseJS(ctx)

Run JS against string in specified context.

Kind: inner method of String

| Param | Type | Description | | --- | --- | --- | | ctx | Object | context hash |

String~parse$(query)

Return an EMAC "...${...}..." string using supplied context.

Kind: inner method of String

| Param | Type | Description | | --- | --- | --- | | query | Object | context hash |

String~parseJSON(def)

Parse string into json or set to default value/callback if invalid json.

Kind: inner method of String

| Param | Type | Description | | --- | --- | --- | | def | function | Object | default object or callback that returns default |

String~parsePath(query, index, flags, where) ⇒ Array

Parse a "PATH?PARM&PARM&..." url into the specified query, index, flags, or keys hash as directed by the PARM = ASKEY := REL || REL || _FLAG = VALUE where REL = X OP X || X, X = KEY || KEY$[IDX] || KEY$.KEY and returns [path,file,type].

Kind: inner method of String
Returns: Array - [path,table,type,area,url]

| Param | Type | Description | | --- | --- | --- | | query | Object | hash of query keys | | index | Object | hash of sql-ized indexing keys | | flags | Object | hash of flag keys | | where | Object | hash of sql-ized conditional keys |

String~chunkFile(path, opts, {Function))

Chunk stream at path by splitting into newline-terminated records. Callback cb(record) until the limit is reached (until eof when !limit) with cb(null) at end.

Kind: inner method of String

| Param | Type | Description | | --- | --- | --- | | path | String | source file | | opts | Object | {newline,limit} options | | {Function) | | cb Callback(record) |

String~parseFile(path, opts, cb)

Parse a csv/txt/json stream at the specified path dependings on if the keys is

[] then record keys are determined by the first header record; 
[ 'key', 'key', ... ] then header keys were preset; 
null then raw text records are returned; 
function then use to parse records.  

The file is chunked using the (newline,limit) chinkFile parameters.
Callsback cb(record) for each record with cb(null) at end.

Kind: inner method of String

| Param | Type | Description | | --- | --- | --- | | path | String | source file | | opts | Object | {keys,comma,newline,limit} options | | cb | function | Callback(record || null) |

String~streamFile(path, opts, cb)

Stream file at path containing comma delimited values. The file is split using the (keys,comma) file splitting parameters, and chunked using the (newline,comma) file chunking parameters. Callsback cb( [record,...] ) with the record batch or cb( null ) at end.

Kind: inner method of String

| Param | Type | Description | | --- | --- | --- | | path | String | source file | | opts | Object | {keys,comma,newline,limit,batch} options | | cb | function | Callback( [record,...] || null ) |

String~trace(msg, req, res)

Trace message to console with optional request to place into syslogs

Kind: inner method of String

| Param | Type | Description | | --- | --- | --- | | msg | String | message to trace | | req | Object | request { sql, query, client, action, table } | | res | function | response callback(msg) |

String~serial()

Serialize this String to the callback(results) given a sync/asyn fetcher(rec,res) where rec = {ID, arg0, arg1, ...} contains args produced by regex. Provide a unique placeholder key to back-substitute results.

Kind: inner method of String
Example

"junkabc;junkdef;"
	.serial( (rec,cb) => cb("$"), /junk([^;]*);/g, "@tag", msg => console.log(msg) )

produces:

	"$$"

String~get()

Fetch using supplied url.

Kind: inner method of String

Clock

Create a clock object with specified trace switch, every interval, on-off times, and start date. See the clock tick method for more information.

| Param | Type | Description | | --- | --- | --- | | trace | Boolean | tracking switch | | every | String | Float | tick interval | | on | Float | on-time or 0 implies infinity | | off | Float | off-time or 0 | | start | Date | start date |

• Clock
  • ~now() ⇒ Date
  • ~tick(cb) ⇒ Date

Clock~now() ⇒ Date

Kind: inner method of Clock
Returns: Date - Current clock time

Clock~tick(cb) ⇒ Date

Return the wait time to next event, with callback(wait,next) when at snapshot events.

Example below for ON = 4 and OFF = 3 steps of length clock.every = sec|min|hour|...

Here S|B|* indicates the end of snapshot|batch|start events. The clock starts on epoch = OFF with a wait = 0. The clock's host has 1 step to complete its batch tasks, and OFF steps to complete its snapshot tasks. Here, the work CYCLE = ON+OFF with a utilization = ON/CYCLE.
Use OFF = 0 to create a continious process.

	S			*	B	B	B	S			*	B	B	B
	|			|	|	|	|	|			|	|	|	|		...
	|			|	|	|	|	|			|	|	|	|		
	x-->x-->x-->x-->x-->x-->x-->x-->x-->x-->x-->x-->x-->x-->x-->x-->x-->x

epoch 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

	|<-- OFF -->|<---- ON ----->|

Kind: inner method of Clock
Returns: Date - Wait time

| Param | Type | Description | | --- | --- | --- | | cb | function | Callback |

Contacting, Contributing, Following

Feel free to

• submit and status TOTEM issues
• contribute to TOTEM notebooks
• revise TOTEM requirements
• browse TOTEM holdings
• or follow TOTEM milestones

License

MIT

© 2012 ACMESDS