npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@totemorg/totem

v3.15.0

Published

Provides basic, customizable and extensible web service

Downloads

74

Readme

TOTEM

TOTEM provides a barebones web service with customizable features:

  • attach agents (local, registered, and notebook) to process endpoint requests
  • configure https service
  • denial-of-service protection
  • secure connections
  • client logins and profiles
  • PKI encryption and authentication
  • fault protected run states
  • indexing, uploading, downloading and cacheing static files
  • crud interface to mysql and neo4j databases
  • task queuing and regulation
  • periodic watchdog and polling services
  • file streams
  • data fetching using various protocols
  • smartcard reader
  • site skinning

TOTEM provides the following:

/DATASET.TYPE ? QUERY
/AREA/FILE ? QUERY
/AREA/ ? QUERY
/json/STORE ? QUERY
/graph/GRAPH ? QUERY
/AGENT ? QUERY

where the optional TYPE:

db | csv | txt | html | json 

returns the dataset in the specified form.

Install

npm install @totemorg/totem		# install base service
npm update

Start

npm run ?env					# Show env variables
npm run ?config					# show configuration
npm run ?						# List start options
npm run start PLUGIN PLUGIN...	# Start totem with optional plugins

npm run setprot					# Configure for protected mode
npm run setdebug				# Configure for debugging mode
npm run setoper					# Configure for operational mode
npm run setprod					# Configure for production mode
npm run	startdbs				# Start database servers

Manage

npm run verminor				# Roll minor version
npm run vermajor				# Roll major version
npm run redoc					# Regen documentation
npm run pubminor				# republish as minor version
npm run pubmajor				# republish as major version

Usage

import * as TOTEM from "@totemorg/totem";

TOTEM.config({
	key: value, 						// set key
	"key.key": value, 					// indexed set
	"key.key.": value					// indexed append
}, sql => {
	console.log( sql ? "look mom - Im running!" : "something evil is lurking" );
});

where configuration keys follow ENUMS deep copy conventions.

Program Reference

TOTEM

Provides a basic web service. This module documented IAW jsdoc.

Requires: module:enums, module:jsdb, module:securelink, module:skin, module:agent, module:fs, module:constants, module:cluster, module:child_process, module:stream, module:vm, module:crypto, module:mime, module:xml2js, module:toobusy-js, module:json2csv, module:js2xmlparser, module:cheerio, module:markdown
Author: ACMESDS
Example

// npm test T1
// Create simple service but dont start it.
Log({
	msg: "Im simply a Totem interface so Im not even running as a service", 
	default_fetcher_endpts: TOTEM.nodeRouters,
	default_protect_mode: TOTEM.guard,
	default_cores_used: TOTEM.cores
});

Example

// npm test T2
// Totem service running in fault protection mode, no database, no UI; but I am running
// with 2 workers and the default endpoint routes.

config({
	mysql: null,
	guard: true,
	cores: 2
}, sql => {

	Log( 
`I'm a Totem service running in fault protection mode, no database, no UI; but I am running
with 2 workers and the default endpoint routes` );

});

Example

// npm test T3
// A Totem service with no workers.

config({
}, sql => {
	Log( 
`I'm a Totem service with no workers. I do, however, have a mysql database from which I've derived 
my startup options (see the openv.apps table for the Nick="Totem1") with just the standard endpoints.
	`
	);
});

Example

// npm test T4
// Only 1 worker, unprotected, a mysql database, and two endpoints.

config({
	nodeRouters: {
		dothis: function dothis(req,res) {  //< named handlers are shown in trace in console
			res( "123" );

			Log("", {
				do_query: req.query
			});
		},

		dothat: function dothat(req,res) {

			if (req.query.x)
				res( [{x:req.query.x+1,y:req.query.x+2}] );
			else
				res( _Error("We have a problem huston") );

			Log("", {
				msg: `Like dothis, but needs an ?x=value query`, 
				or_query: req.query,
				or_user: req.client
			});
		}
	}
}, sql => {
	Log("", {
		msg:
`As always, if the openv.apps Encrypt is set for the Nick="Totem" app, this service is now **encrypted** [*]
and has https (vs http) endpoints, here /dothis and /dothat endpoints.  Ive only requested only 1 worker (
aka core), Im running unprotected, and have a mysql database.  
[*] If my NICK.pfx does not already exists, Totem will create its password protected NICK.pfx cert from the
associated public NICK.crt and private NICK.key certs it creates.`,
		my_endpoints: T.nodeRouters
	});
});

Example

// npm test T5
// no cores but a mysql database and an anti-bot shield

config({
	"login.challenge.extend": 20
}, sql => {
	Log("", {
		msg:
`I am Totem client, with no cores but I do have mysql database and I have an anti-bot shield!!  Anti-bot
shields require a Encrypted service, and a UI (like that provided by DEBE) to be of any use.`, 
		mysql_derived_parms: T.siteContext
	});
});

Example

// npm test T6
// Testing tasker with database, 3 cores and an additional /test endpoint.

config({
	guard: false,	// ex override default 
	cores: 3,		// ex override default

	"nodeRouters.": {  // define endpoints
		test: function (req,res) {
			res(" here we go");  // endpoint must always repond to its client 
			if (isMaster)  // setup tasking examples on on master
				switch (req.query.opt || 1) {  // test example runTask
					case 1: 
						T.runTask({  // setup tasking for loops over these keys
							keys: "i,j",
							i: [1,2,3],
							j: [4,5]
						}, 
							// define the task which returns a message msg
							($) => "hello i,j=" + [i,j] + " from worker " + _Worker + " on " + $.node, 

							// define the message msg handler
							(msg) => console.log(msg)
						);
						break;

					case 2:
						T.runTask({
							qos: 1,
							keys: "i,j",
							i: [1,2,3],
							j: [4,5]
						}, 
							($) => "hello i,j=" + [i,j] + " from worker " + _Worker + " on " + $.node, 
							(msg) => console.log(msg)
						);
						break;

					case 3:
						break;
				}

		}
	}

}, sql => {
	Log( "Testing runTask with database and 3 cores at /test endpoint" );
});

Example

// npm test T7
// Conduct db maintenance

config({
}, sql => {				
	Log( "db maintenance" );

	if (isMaster)
		switch (process.argv[3]) {
			case 1: 
				sql.query( "select voxels.id as voxelID, chips.id as chipID from openv.voxels left join openv.chips on voxels.Ring = chips.Ring", function (err,recs) {
					recs.forEach( rec => {
						sql.query("update openv.voxels set chipID=? where ID=?", [rec.chipID, rec.voxelID], err => {
							Log(err);
						});
					});
				});
				break;

			case 2:
				sql.query("select ID, Ring from openv.voxels", function (err, recs) {
					recs.forEach( rec => {
						sql.query(
							"update openv.voxels set Point=geomFromText(?) where ?", 
							[ `POINT(${rec.Ring[0][0].x} ${rec.Ring[0][0].y})` , {ID: rec.ID} ], 
							err => {
								Log(err);
						});
					});
				});
				break;

			case 3:
				sql.query( "select voxels.id as voxelID, cache.id as chipID from openv.voxels left join openv.cache on voxels.Ring = fileCache.geo1", function (err,recs) {
					Log(err);
					recs.forEach( rec => {
						sql.query("update openv.voxels set chipID=? where ID=?", [rec.chipID, rec.voxelID], err => {
							Log(err);
						});
					});
				});
				break;

			case 4:
				sql.query("select ID, geo1 from openv.cache where bank='chip'", function (err, recs) {
					recs.forEach( rec => {
						if (rec.geo1)
							sql.query(
								"update openv.cache set x1=?, x2=? where ?", 
								[ rec.geo1[0][0].x, rec.geo1[0][0].y, {ID: rec.ID} ], 
								err => {
									Log(err);
							});
					});
				});
				break;

			case 5: 
				var parms = {
ring: "[degs] closed ring [lon, lon], ... ]  specifying an area of interest on the earth's surface",
"chip length": "[m] length of chip across an edge",
"chip samples": "[pixels] number of pixels across edge of chip"
				};
				//get all tables and revise field comments with info data here -  archive parms - /parms in flex will
				//use getfileds to get comments and return into

			case 6:
				var 
					RAN from "../randpr"),
					ran = new RAN({
						models: ["sinc"],
						Mmax: 150,  // max coherence intervals
						Mstep: 5 	// step intervals
					});

				ran.config( function (pc) {
					var 
						vals = pc.values,
						vecs = pc.vectors,
						N = vals.length, 
						ref = vals[N-1];

					vals.forEach( (val, idx) => {
						var
							save = {
								correlation_model: pc.model,
								coherence_intervals: pc.intervals,
								eigen_value: val,
								eigen_index: idx,
								ref_value: ref,
								max_intervals: ran.Mmax,
								eigen_vector: JSON.stringify( vecs[idx] )
							};

						sql.query("INSERT INTO openv.pcs SET ? ON DUPLICATE KEY UPDATE ?", [save,save] );	
					});
				});
				break;	
		}
});		

Example

// npm test T8
// Conduct neo4j database maintenance

const $ from "../man/man.js");
config();
neoThread( neo => {
	neo.cypher( "MATCH (n:gtd) RETURN n", {}, (err,nodes) => {
		Log("nodes",err,nodes.length,nodes[0]);
		var map = {};
		nodes.forEach( (node,idx) => map[node.n.name] = idx );
		//Log(">map",map);

		neo.cypher( "MATCH (a:gtd)-[r]->(b:gtd) RETURN r", {}, (err,edges) => {
			Log("edges",err,edges.length,edges[0]);
			var 
				N = nodes.length,	
				cap = $([N,N], (u,v,C) => C[u][v] = 0 ),
				lambda = $([N,N], (u,v,L) => L[u][v] = 0),
				lamlist = $(N, (n,L) => L[n] = [] );

			edges.forEach( edge => cap[map[edge.r.srcId]][map[edge.r.tarId]] = 1 );

			//Log(">cap",cap);

			for (var s=0; s<N; s++)
				for (var t=s+1; t<N; t++) {
					var 
						{cutset} = $.MaxFlowMinCut(cap,s,t),
						cut = lambda[s][t] = lambda[t][s] = cutset.length;

					lamlist[cut].push([s,t]);
				}

			lamlist.forEach( (list,r) => {
				if ( r && list.length ) Log(r,list);
			});

		});
	});
});	

TOTEM.openvACL

Route nodes according to security requirements.

Kind: static constant of TOTEM

TOTEM.filters

Endpoint filters cb(data data as string || error)

Kind: static constant of TOTEM
Cfg: Object

TOTEM.watchDogs

Kind: static constant of TOTEM

TOTEM.siteContext

Site context extended by the mysql derived query when service starts

Kind: static constant of TOTEM
Cfg: Object

TOTEM.uploadFile(client, srcStream, sinkPath, tags, cb)

Uploads a source stream srcStream to a target file sinkPath owned by the specified client; optional tags are tagged to the upload and the callback cb is made if the upload was successful.

Kind: static method of TOTEM

| Param | Type | Description | | --- | --- | --- | | client | String | file owner | | srcStream | Stream | source stream | | sinkPath | String | path to target file | | tags | Object | hash of tags to add to file | | cb | function | callback(file) if upload successful |

TOTEM~agents : object

Endpoint agents each of the form agent(req,res)

Kind: inner namespace of TOTEM

agents.attach(req, res)

Register named agents at the specified port serviced by specified number of server cores, charging clients a specified usage cost, or collect util info from registered agents. See attachAgents for details.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.build(req, res)

Endpoint to build script for specified &install keys.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.favicon(req, res)

Endpoint to return favorite icon

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.help(req, res)

Endpoint to return system help on requested agent and section goto.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.ping(req, res)

Endpoint to test connectivity.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem request | | res | function | Totem response |

agents.GET(req, res)

Endpoint to select records from a dataset or redirect to an agent.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.PUT(req, res)

Endpoint to update records in a dataset within the client's focus.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.DELETE(req, res)

Endpoint to delte records from a dataset within the client's focus.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.POST(req, res)

Endpoint to add records to a dataset within the client's focus.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.EXEC(req, res)

Endpoint to execute records in a dataset within the client's focus.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

TOTEM~_Errors

Client error messages

Kind: inner constant of TOTEM

_Errors.noFile

Kind: static property of _Errors

_Errors.badFilter

Kind: static property of _Errors

_Errors.noData

Kind: static property of _Errors

_Errors.noAgent

Kind: static property of _Errors

_Errors.noDB

Kind: static property of _Errors

_Errors.badReturn

Kind: static property of _Errors

_Errors.badLogin

Kind: static property of _Errors

_Errors.noSocket

Kind: static property of _Errors

_Errors.badQuery

Kind: static property of _Errors

_Errors.noEndpoint

Kind: static property of _Errors

_Errors.noID

Kind: static property of _Errors

_Errors.noPost

Kind: static property of _Errors

_Errors.noSQL

Kind: static property of _Errors

_Errors.noNEO

Kind: static property of _Errors

_Errors.noReader

Kind: static property of _Errors

TOTEM~agentThread(agent, req, res)

Start an agent thread by Route NODE = /DATASET.TYPE requests using the configured agents.

The routeThread adds client data:

encrypted: bool		// true if request on encrypted server
site: {...}			// site context
mimi: "type"		// mime type of response

Kind: inner method of TOTEM

| Param | Type | Description | | --- | --- | --- | | agent | function | agent to run | | req | Object | session request | | res | responseCallback | session response |

TOTEM~dataThread(req, cb)

Start a dataset thread by appending a

sql: {...}			// sql connector
ds:	"focus.table"	// fully qualified sql table
focus: user focus
//action: "select|update| ..."	// corresponding crude name

to the req request with callback cb(req).

Kind: inner method of TOTEM

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem endpoint request | | cb | function | callback(revised req) |

TOTEM~startDogs()

Start watchdogs

Kind: inner method of TOTEM

TOTEM~initServer()

Initialize the service.

Kind: inner method of TOTEM

TOTEM~stopServer()

Stop the server.

Kind: inner method of TOTEM

String

String~linkify(ref) ⇐ String

Expands "*"-tagged string into html link.

Kind: inner method of String
Extends: String

| Param | Type | | --- | --- | | ref | String |

Example

"test [abc](http:/junk)".linkify()
==> 'test <a href="http:/junk">abc</a>'

Example

"here is a link".linkify("test")
==> '<a href="test">here is a link</a>'

String~mailify(ref) ⇐ String

Expands string into a html email link.

Kind: inner method of String
Extends: String

| Param | Type | | --- | --- | | ref | String |

Example

"contact me".mailify("user.one")
'<a href="mailto:contact me">user.one</a>'

String~parseXML(cb) ⇐ String

Parse XML string into json then callsback cb(json).

Kind: inner method of String
Extends: String

| Param | Type | Description | | --- | --- | --- | | cb | function | callback( json || null if error ) |

Array

Array~groupify(dot)

Groups [ "x.y.z", "u,v", "x" , ... ] list entries into "x(y(z)), u(v), x, ..." string.

Kind: inner method of Array

| Param | Type | Description | | --- | --- | --- | | dot | string | item seperator |

Example

["a.b","x.y.z"].groupify(".")
==> 'a(b),x(y(z))'

Array~blog(req, keys, cb)

Blogs each string in the list.

Kind: inner method of Array
See: totem:blogify

| Param | Type | Description | | --- | --- | --- | | req | Object | Request context | | keys | List | list of keys to blog | | cb | function | callback(recs) blogified version of records |

Array~joinify(cb)

Joins a list with an optional callback cb(head,list) to join the current list with the current head.

Kind: inner method of Array

| Param | Type | | --- | --- | | cb | function |

Example

[	a: null,
	g1: [ b: null, c: null, g2: [ x: null ] ],
	g3: [ y: null ] ].joinify()

==>
	"a,g1(b,c,g2(x)),g3(y)"

TOTEM

TOTEM.openvACL

Route nodes according to security requirements.

Kind: static constant of TOTEM

TOTEM.filters

Endpoint filters cb(data data as string || error)

Kind: static constant of TOTEM
Cfg: Object

TOTEM.watchDogs

Kind: static constant of TOTEM

TOTEM.siteContext

Site context extended by the mysql derived query when service starts

Kind: static constant of TOTEM
Cfg: Object

TOTEM.uploadFile(client, srcStream, sinkPath, tags, cb)

Uploads a source stream srcStream to a target file sinkPath owned by the specified client; optional tags are tagged to the upload and the callback cb is made if the upload was successful.

Kind: static method of TOTEM

| Param | Type | Description | | --- | --- | --- | | client | String | file owner | | srcStream | Stream | source stream | | sinkPath | String | path to target file | | tags | Object | hash of tags to add to file | | cb | function | callback(file) if upload successful |

TOTEM~agents : object

Endpoint agents each of the form agent(req,res)

Kind: inner namespace of TOTEM

agents.attach(req, res)

Register named agents at the specified port serviced by specified number of server cores, charging clients a specified usage cost, or collect util info from registered agents. See attachAgents for details.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.build(req, res)

Endpoint to build script for specified &install keys.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.favicon(req, res)

Endpoint to return favorite icon

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.help(req, res)

Endpoint to return system help on requested agent and section goto.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.ping(req, res)

Endpoint to test connectivity.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem request | | res | function | Totem response |

agents.GET(req, res)

Endpoint to select records from a dataset or redirect to an agent.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.PUT(req, res)

Endpoint to update records in a dataset within the client's focus.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.DELETE(req, res)

Endpoint to delte records from a dataset within the client's focus.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.POST(req, res)

Endpoint to add records to a dataset within the client's focus.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

agents.EXEC(req, res)

Endpoint to execute records in a dataset within the client's focus.

Kind: static method of agents

| Param | Type | Description | | --- | --- | --- | | req | requestHash | session request | | res | responseCallback | session response |

TOTEM~_Errors

Client error messages

Kind: inner constant of TOTEM

_Errors.noFile

Kind: static property of _Errors

_Errors.badFilter

Kind: static property of _Errors

_Errors.noData

Kind: static property of _Errors

_Errors.noAgent

Kind: static property of _Errors

_Errors.noDB

Kind: static property of _Errors

_Errors.badReturn

Kind: static property of _Errors

_Errors.badLogin

Kind: static property of _Errors

_Errors.noSocket

Kind: static property of _Errors

_Errors.badQuery

Kind: static property of _Errors

_Errors.noEndpoint

Kind: static property of _Errors

_Errors.noID

Kind: static property of _Errors

_Errors.noPost

Kind: static property of _Errors

_Errors.noSQL

Kind: static property of _Errors

_Errors.noNEO

Kind: static property of _Errors

_Errors.noReader

Kind: static property of _Errors

TOTEM~agentThread(agent, req, res)

Start an agent thread by Route NODE = /DATASET.TYPE requests using the configured agents.

The routeThread adds client data:

encrypted: bool		// true if request on encrypted server
site: {...}			// site context
mimi: "type"		// mime type of response

Kind: inner method of TOTEM

| Param | Type | Description | | --- | --- | --- | | agent | function | agent to run | | req | Object | session request | | res | responseCallback | session response |

TOTEM~dataThread(req, cb)

Start a dataset thread by appending a

sql: {...}			// sql connector
ds:	"focus.table"	// fully qualified sql table
focus: user focus
//action: "select|update| ..."	// corresponding crude name

to the req request with callback cb(req).

Kind: inner method of TOTEM

| Param | Type | Description | | --- | --- | --- | | req | Object | Totem endpoint request | | cb | function | callback(revised req) |

TOTEM~startDogs()

Start watchdogs

Kind: inner method of TOTEM

TOTEM~initServer()

Initialize the service.

Kind: inner method of TOTEM

TOTEM~stopServer()

Stop the server.

Kind: inner method of TOTEM

Contacting, Contributing, Following

Feel free to

License

MIT


© 2012 ACMESDS