squared-functions

v4.14.0

Published

a month ago

Parent folder for @squared-functions packages.

Downloads

485

0High
0Medium
0Low

anpham6

squared squared-express sqd-cli sqd-serve

squared-functions 4.14

NodeJS - 10 (Minimum)
Chrome - 12 (Recommended)

Examples use squared 3/4 although the concepts can be used similarly with any NodeJS application. Using squared or squared-express is not required.

NOTE: Feature lock is in effect for squared-functions other than extension compatibility. Any further development will be under E-mc.

Protocols

HTTP/2
FTP/SFTP/BitTorrent/Metalink - aria2 (apt/brew install aria2)
Unix domain socket - { socketPath: "/tmp/static0.sock", uri: "file:///path/filename" }

Document

Inline bundling options are available with these HTML tag names.

saveAs: html + script + link + raw assets
exportAs: script + style
exclude: script + link + style

Files with the same path and filename will automatically create a bundle assuming there are no conflicts in call ordering.

More advanced configurations are possible using a JSON/YAML external configuration file. Inline commands are usually sufficient for simple web pages.

NOTE: MIME (mimeType) is required when bundling torrents.

// HTML configuration using YAML is supported

{
  "selector": "head > script:nth-of-type(2), head > script:nth-of-type(3)",
  "type": "js",
  "saveAs": "js/modules2.js"
}

JS and CSS files can be bundled together with the "saveAs" or "exportAs" action. Multiple transformations per bundle can be chained using the "+" symbol.


+ saveAs: location | ~ // Same
+ exportAs: location

- ::
- format (chain "+")

These are the available option modifiers:

* preserve (true)
  - prevent unused styles from being deleted
    + html
    + css
* inline (true)
    + js: Rendered inline with <script>
    + css: Rendered inline with <style>
    + image: Rendered as base64 from file
* module (true | false - config | no-module - inline)
    + js: ESM
    + css: SASS
* extract (true)
    + css: @import rules are inlined into parent file (same origin)
* blob (true)
    + image: HTML and CSS from base64
    + font: CSS from base64
* dynamic (true)
    + image: srcset uses query parameters with an image resizer
    + element (non-void): mixed content which uses a view engine template (e.g. ejs)
* compress (array)
    + image: imagemin
    + font: woff + woff2
    + png: TinyPNG service (jpeg + webp)
    + gz: Gzip
    + br: Brotli
* static: (true)
    + all: Remove query string from URL
* crossorigin (true)
    + all: Same as preserveCrossOrigin [download: false]
* download (true - explicit | false)
    + all: Source files that are not usually downloaded (e.g. link[rel=alternate])
* hash
  - filename + content hash (productionRelease=true)
  - value can be limited to the starting prefix (e.g. md5[8] - Minimum is 4)
    + js
    + css
    + map
    + img
    + font
    + audio
    + video
* remove
    + all: Remove element during finalization
- charset (string)
    + utf-8 | utf-16 | utf-16le | latin1 | utf-16be (unsupported) [default: utf-8]

NOTE: Whitespace can be used between anything for readability.

<link rel="stylesheet" href="css/dev.css" data-chrome-file="saveAs:css/prod.css::beautify" data-chrome-options="preserve|md5|compress[gz]|utf-16">
<style data-chrome-file='{ "exportAs": "css/prod.css", "process": ["minify", "beautify"] }' data-chrome-options='{ "preserve": true, "hash": "md5", "compress": [{ "format": "gz" }], "encoding": "utf-16" }'>
  body {
    font: 1em/1.4 Helvetica, Arial, sans-serif;
    background-color: #fafafa;
  }
</style>
<script src="/dist/squared.js" data-chrome-file="saveAs:js/bundle1.js::minify" data-chrome-metadata='{ "custom-rollup": { "import-maps": {} }, "custom-terser": { "config": {} } }'></script>
<script src="/dist/squared.base.js" data-chrome-file="saveAs:js/bundle1.js"></script>
<script src="/dist/chrome.framework.js" data-chrome-file='{ "saveAs": "js/bundle2.js", "process": ["lint"] }'></script>

Built-In plugins

JS and CSS files can be optimized further using these settings:

beautify
lint
minify
es5 (Babel)
es5-minify (UglifyJS)
minify-svg (svgo)
compile (sass)
custom name

You can define or undefine your own optimizations in squared.json:

html

css

js

These listed plugins (npm i package) can be configured using a plain object (chrome.settings.transform). Other non-builtin transpilers can similarly be applied and chained by defining a custom function.

NPM package
Local file using module.exports (e.g. ".cjs")
Local plain file with single function (e.g. ".js")
Inline function

More advanced plugins can be written and installed through NPM. The only difference is the context parameter is set to the Document module.

Examples can be found in the "@squared-functions/document/packages" folder.

// squared.json

{
  "document": {
    "chrome": {
      "handler": "@squared-functions/document/chrome",
      "eval": {
        "function": true, // Enable inline functions
        "absolute": false, // Enable absolute paths to local files
        "template": false, // Enable external template functions
        "userconfig": false // Enable functions inside local files from user queries
      },
      "settings": {
        "directory": {
          "package": "../packages" // Override built-in plugins (base directory + users/username? + rollup.js)
        },
        "transform": {
          "html": {
            "posthtml": { // Built-in transformer
              "transform": {
                "plugins": [
                  ["posthtml-doctype", { "doctype": "HTML 5" }], // Plugins have to be pre-installed from NPM
                  ["posthtml-include", { "root": "./", "encoding": "utf-8" }]
                ]
              },
              "transform-output": {
                "directives": [
                  { "name": "?php", "start": "<", "end": ">" }
                ]
              }
            },
            "prettier": {
              "beautify": {
                "parser": "html",
                "printWidth": 120,
                "tabWidth": 4
              }
            }
          },
          "js": {
            /* Inline asynchronous function */
            "terser": { // npm i terser
              "minify-example": "async function (terser, value, options, require) { return await terser.minify(value, options.outputConfig).code; }", // this = NodeJS.process
              "minify-example-output": {
                "keep_classnames": true // "minify-example-output" creates variable "options.outputConfig"
              }
            },
            "@babel/core": {
              "npm-example": "npm:babel-custom", // function(@babel/core, value, options) (npm i babel-custom)
              "npm-example-output": "npm:babel-custom-output", // Configuration object (npm i babel-custom-output)
              /* OR */
              "npm-example-output": {
                "presets": ["@babel/preset-env"]
              },
              "es5-example": "./es5.js" // Local file - startsWith("./ | ../")
            }
          },
          "css": {
            /* Built-in transformer */
            "postcss": {
              "transform": {
                "plugins": [
                  "autoprefixer",
                  ["cssnano", { preset: ["cssnano-preset-default", { "discardComments": false }] }]
                ]
              }
            },
            /* Inline synchronous function + Promise */
            "sass": { // npm i sass
              "sass-example": "function (sass, value, options, resolve, require) { resolve(sass.renderSync({ ...options.outputConfig, data: value }).css); }",
              "sass-example-output": {
                "outputStyle": "compressed",
                "sourceMap": true,
                "sourceMapContents": true
              }
            },
            "sass": { // function (sass, value, options)
              "transform": "./sass-local.cjs", // Uses module.exports (debuggable)
              "transform-output": { // options.outputConfig
                "sourceMap": true
              }
            },
            /* npm i sass-custom (public) */
            "sass-custom": { // function (context /* Document */, value, options)
              "transform": { // options.baseConfig
                "sourceMap": true
              }
            }
          }
        }
      }
    }
  }
}

NOTE: Script and settings files with ".cjs" extension (configurable) will be loaded with "require" when using squared-express.

interface TransformOutput {
    pathname?: string;
    filename?: string;
    mimeType?: string;
    sourceFile?: string;
    sourcesRelativeTo?: string;
    sourceMap?: SourceMapInput;
    metadata?: unknown;
    external?: PlainObject;
}

interface ITransformSeries extends Module, TransformOutput {
    type: "html" | "css" | "js";
    baseConfig: PlainObject;
    outputConfig: PlainObject; // Same as baseConfig when using an inline transformer
    sourceMap: SourceMap; // Primary sourceMap
    code: string;
    metadata: PlainObject; // Custom request values and modifiable per transformer
    productionRelease: boolean;
    supplementChunks: ChunkData[];
    imported: boolean; // ESM detected
    createSourceMap(code: string): SourceMap; // Use "nextMap" method for sourceMap (additional sourceMaps)

    /* ESM */
    getMainFile?(code?: string, imports?: StringMap): Undef<SourceInput<string>>;
    getSourceFiles?(imports?: StringMap): Undef<SourceInput<[string, string?, string?][]>>;

    /* Return values */
    out: {
        sourceFiles?: string[]; // ESM
        ignoreCache?: boolean;
        messageAppend?: string;
        logAppend?: LogStatus[];
        logQueued?: LogStatus[];
    };

    /* Deprecated */
    outSourceFiles?: string[];
    outIgnoreCache?: boolean;
    outMessageAppend?: string;
    outLogAppend?: LogStatus[];
}

// es5.js
function (context, value, options, resolve, require) {
  context.transform(value, options.outputConfig, function (err, result) {
    resolve(!err && result ? result.code : "");
  });
}

// es5.cjs
const path = require('path');

module.exports = async function (context, value, options) {
  return await context.transform(value, options.outputConfig).code;
}

The same concept can be used inline anywhere using a <script> tag with the type attribute set to "text/template". The script template will be completely removed from the final output.

// "es5-example" is a custom identifier (chrome -> eval_template: true)

<script type="text/template" data-chrome-template="js::@babel/core::es5-example">
async function (context, value, options, require) {
  const options = { ...options.outputConfig, presets: ["@babel/preset-env"], sourceMaps: true }; // https://babeljs.io/docs/en/options
  const result = await context.transform(value, options);
  if (result) {
    if (result.map) {
      options.sourceMap.nextMap("babel", result.code, result.map);
    }
    return result.code;
  }
}
</script>

External configuration

JSON (json/js) configuration is provided for those who prefer to separate the bundling and transformations from the HTML. Any assets inside the configuration file will override any settings either inline or from JavaScript.

interface OutputModifiers {
    inline?: boolean; // type: js | css | base64: image | font
    blob?: boolean; // type: image | font (base64)
    preserve?: boolean; // type: html | css | cross-origin: append/js | append/css
    extract?: boolean; // type: css
    module?: boolean; // type: js | css (ESM)
    static?: boolean; // Removes URL search params
    dynamic?: boolean; // type: image (srcset) | element (non-void)
    remove?: boolean; // Removes element from HTML page
    ignore?: boolean;
    exclude?: boolean; // type: js | css (remove from HTML)
}

interface AssetCommand extends OutputModifiers {
    selector: string;
    mergeType?: "none" | "over" | "under"; // Use when different selectors target same element

    saveAs?: string; // js | css | image
    exportAs?: string; //js | css
    saveTo?: string; // image | video | audio (transforms create multiple files and are given generated filename)
    hash?: "md5" | "sha1" | "sha224" | "sha256" | "sha384" | "sha512"; // md5[8] will shorten hash to the first 8 characters

    pathname?: string; // alias for "saveTo"
    filename?: string; // pathname + filename = "saveAs"
    process?: string[]; // js | css | svg
    commands?: string[]; // image
    download?: boolean; // Same as preserveCrossOrigin (default is "true")
    cloudStorage?: CloudService[];
    tasks?: string[];
    watch?: boolean | { interval?: number, expires?: string }; // type: js | css | image (expires: 1h 1m 1s)
    attributes?: ObjectMap<Optional<string>>;
    template?: {
        module: string;
        identifier?: string;
        value?: string;
    };

    dynamic?: boolean; // Will ignore misplaced child elements prerendered in the browser

    incremental: false | "none" | "etag" | "exists"; // Will override batch request.incremental
    incremental: true; // Will override request.incremental = false

    metadata?: PlainObject; // Custom values passed to transformers
    metadata: { __sourcemap__: "inline" }; // System post processing command

    type: "html" | "js" | "css" | "data"; // Script templates (optional)
    type: "append/js" | "append/css" | "append/[tagName]"; // Includes "prepend"

    type: "text" | "attribute" | "display"; // dynamic is valid only with "text"
    dataSource?: CloudDatabase; // "cloud" (source)
    dataSource?: {
        source: "mongodb" | "redis" | "mysql" | "postgres" | "uri" | "local" | "export";
        postQuery?: string;
        preRender?: string;
        whenEmpty?: string;
    };

    type: "replace";
    textContent: string; // Replace element.innerHTML

    document?: string | string[]; // Usually "chrome" (optional)
}

Only one command per element is supported (except data sources) with the last selector taking precedence except when mergeType is defined.

// sqd.config

/* Glob index.html* - ordinal 1 */
[
  {
    "selector": "img#picture1",
    "commands": ["png@"]
  },
  {
    "selector": "img#picture2",
    "commands": ["webp%"]
  }
]

/* Glob index.html?output=prod - ordinal 2 */
{
  "selector": "img",
  "mergeType": "under",
  "hash": "sha256", // Merged
  "commands": ["jpeg@"] // No effect
}
/* OR */
{
  "selector": "img",
  "mergeType": "over",
  "hash": "sha256", // Merged
  "commands": ["jpeg@"] // All images will be JPEG
}
/* OR */
{
  "selector": "img",
  "mergeType": "none",
  "hash": "sha256", // No effect when "dev" is present
  "commands": ["jpeg@"] // Same
}

squared.saveAs("bundle.zip", { config: { uri: "http://localhost:3000/chrome/bundle.yml", mimeType: "text/yaml" } }); // "mimeType" (optional)
squared.saveAs("bundle.zip", { config: "http://localhost:3000/chrome/bundle.yml" });

// http://hostname/example.html -> http://hostname/example.html.json
squared.saveAs("example.zip", { config: { mimeType: "json" } });
squared.saveAs("example.zip", { config: "json" }); // json | yml | yaml

Here is the equivalent page using only inline commands with "data-chrome-file" and "data-chrome-tasks".

bundle_inline.html

Cloud storage

Manual installation of the SDK is required including an account with at least one of these cloud storage provider.

* Amazon
  - https://aws.amazon.com/free (5GB - 12 months)

  + npm i aws-sdk (aws)
    <!-- OR -->
  + npm i @aws-sdk/client-s3 (aws-v3)

* Microsoft
  - https://azure.microsoft.com/en-us/free (5GB - 12 months)

  + npm i @azure/storage-blob (azure/az)

* Google
  - https://cloud.google.com/free (5GB - US)
  - https://firebase.google.com/products/storage (Cannot create new buckets - Use CLI)

  + npm i @google-cloud/storage (gcp/gcloud)
    <!-- OR -->
  + npm i firebase

* IBM
  - https://www.ibm.com/cloud/free (25GB)
  - Uses S3 compatibility API (v2)

  + npm i ibm-cos-sdk (ibm/ibm-v1)

* Oracle
  - https://www.oracle.com/cloud/free (10GB standard + up to 30GB)
  - Uses S3 compatibility API (v2)
  - Cannot create new public buckets

  + npm i aws-sdk (oci)

* MinIO
  - https://min.io/download (Local)
  - S3 compatible object storage

  + npm i minio (minio)

Environment variables can also be used for authorization.


* AWS
  - AWS_ACCESS_KEY_ID
  - AWS_SECRET_ACCESS_KEY
  - AWS_SESSION_TOKEN

* Azure
 - AZURE_TENANT_ID
 - AZURE_CLIENT_ID
 - AZURE_CLIENT_SECRET
 - npm i @azure/identity

* Azure (Unofficial)
  - AZURE_STORAGE_ACCOUNT
  - AZURE_STORAGE_KEY
  - AZURE_STORAGE_CONNECTION_STRING
  - AZURE_STORAGE_SAS_TOKEN
  - process.env.apply (squared.json)

* GCP
  - GOOGLE_APPLICATION_CREDENTIALS
  - npm i firebase-admin

* Minio
  - MINIO_ACCESS_KEY
  - MINIO_SECRET_KEY
  - MINIO_SESSION_TOKEN
  - MINIO_ENDPOINT
  - process.env.apply (squared.json)

// Optional fields are supported by all services

{
  "selector": "#picture1",
  "commands": [
    "png(100x200){90,180,270}" // Uploaded with UUID filename
  ],
  "cloudStorage": [
    {
      "service": "aws",
      "bucket": "squared-001",
      "credential": {
        "accessKeyId": "**********",
        "secretAccessKey": "**********",
        "region": "us-west-2",
        "sessionToken": "**********" // Optional
      },
      "credential": "main", // OR: Load host configuration from settings at instantiation
      /* Optional */
      "admin": {
        "publicRead": true, // Bucket + uploaded objects
        /* OR */
        "publicRead": 1, // Bucket only (s3::ListBucket*)
        /* OR */
        "acl": "public-read" | "public-read-write" | "authenticated-read" | "private", // Will modify existing bucket (equivalent policy) + uploaded files (canned ACL)

        "configBucket": {
          /* Newly created (during upload) */
          "create": {
            "ACL": "public-read", // CreateBucketRequest (https://docs.aws.amazon.com/AmazonS3/latest/userguide/acl-overview.html#canned-acl)
            "CreateBucketConfiguration": {
              "LocationConstraint": "us-west-1"
            }
          },
          /* Modify bucket policy (after upload) */
          "policy": {
            "Policy": "", // PutBucketPolicy (except IBM + OCI)
            /* OR */
            "ACL": "authenticated-read", // PutBucketAcl (except OCI)
            /* OR */
            "PublicAccessBlockConfiguration": { // PutPublicAccessBlock
              "BlockPublicAcls": false,
              "BlockPublicPolicy": false,
              "IgnorePublicAcls": false,
              "RestrictPublicBuckets": false
            }
          }
        }
      },
      /* Optional */
      "upload": {
        "active": false, // Separate from build (background) + Will not overwrite ACL
        "active": true, // Delays build until upload is complete
        "active": true, // Rewrites "src" to cloud storage location + Will overwrite ACL (public-read)
        "localStorage": false, // Remove current file from archive or local disk
        "filename": "picture1.webp", // Choose a different bucket filename
        "all": false, // Include transforms
        "overwrite": false, // Always use current filename
        "contentType": "application/octet-stream", // options.ContentType with primary has highest precedence

        "publicRead": true, // Will overwrite primary options.ACL (except OCI + Firebase)
        /* OR */
        "acl": "authenticated-read" | "aws-exec-read" | "bucket-owner-full-control" | "bucket-owner-read" | "private" | "public-read" | "public-read-write" // ObjectCannedlACL (optional)

        /* Optional - PutObjectRequest */
        "options": {
          "ContentType": "text/html", // Primary object only
          "ACL": "private", // All objects
          "Metadata": {} // All objects (primary + source maps + transforms)
        },
        "metadata": {
          "Content-Type": "text/html; charset=UTF-8", // Full replacement with primary only
          "Content-Encoding": "gzip",
          "Expires": "Wed, 21 Oct 2015 07:28:00 GMT"
        }
      },
      /* Optional */
      "download": {
        "filename": "picture2.png", // Required
        "versionId": "12345", // Retrieve a previous file snapshot
        "pathname": "download/images", // File adjacent or base directory when omitted (overrides "preservePath")
        "active": false, // Separate from build (background)
        "active": true, // Always write file or rename to main file when same extension
        "waitStatus": false, // Delay transaction until files are completely downloaded
        "overwrite": false, // Always write file
        "deleteObject": false, // Remove after successful download
        "deleteObject": {} // DeleteObjectRequest - All
      }
    },
    {
      "service": "azure", // OR: az
      "bucket": "squared-002",
      "credential": {
        "accountName": "**********", // +1 password option (required)
        "accountKey": "**********",
        "connectionString": "**********",
        "sharedAccessSignature": "**********",
        "computeHMACSHA256": "function (stringToSign) { *** require not available *** }" // Inline functions from a POST request
      },
      "admin": {
        "publicRead": true,
        /* OR */
        "acl": "blob" | "container",

        "configBucket": {
          "create": {
            "metadata": {} // ContainerCreateOptions
          }
        }
      },
      "upload": {
        "publicRead": false, // Not supported
        "acl": "", // Not supported

        "pathname": "a/b/c/", // Virtual directory in bucket (overrides "preservePath")
        "endpoint": "http://squared.azureedge.net/squared-002", // e.g. CDN

        /* Optional - BlockBlobUploadOptions */
        "options": {
          "blobHTTPHeaders": {
            "blobContentType": "text/html"
          },
          "metadata": {}
        }
      }
    },
    {
      "service": "gcp", // OR: gcloud
      "bucket": "squared-003", // UUID generated when omitted (optional)
      "credential": {
        "keyFilename": "./gcp.json", // Path to JSON credentials (Google Cloud)
        /* OR */
        "projectId": "squared", // Firebase
        "apiKey": "**********",
        "authDomain": "<project-id>.firebaseapp.com",
        "product": "firebase" // Required with using GOOGLE_APPLICATION_CREDENTIALS
      },
      "admin": {
        "publicRead": true, // New buckets (except OCI + Firebase)
        /* OR */
        "acl": "private" | "projectPrivate" | "authenticatedRead" | "publicRead" | "publicReadWrite" | "bucketAccessUniform" | "bucketAccessACL", // Will call setBucketPolicy after createBucket

        "emptyBucket": false, // More convenient than using "overwrite"
        "configBucket": {
          "create" {
            "location": "", // CreateBucketRequest (using credential for configuration is deprecated)
            "storageClass": ""
          },

          /* MakeBucketPrivateOptions - https://cloud.google.com/nodejs/docs/reference/storage/latest/storage/acl */
          "policy": {
            "acl": "private", // makePrivate + includeFiles + projectPrivate
            "acl": "projectPrivate", // makePrivate + allUsers (delete) + allAuthenticatedUsers (delete)
            "acl": "authenticatedRead", // projectPrivate + allAuthenticatedUsers:READER
            "acl": "publicRead", // makePublic + includeFiles
            "acl": "publicReadWrite", // publicRead + allUsers:WRITER
            "acl": [{ "entity": "allUsers", "role": "READER" } /* add */, { "entity": "allAuthenticatedUsers" } /* delete */] // Custom
            /* Unofficial aliases */
            "acl": "bucketAccessUniform", // Enable uniform bucket-level access
            "acl": "bucketAccessACL", // Revert uniform bucket-level access (within 90 days)
          },
          "website": {
            "indexPage": "index.html", // Optional (chrome: "true" for HTML page only)

            "errorPage": "404.html",
            /* OR: azure */
            "indexPath": "home.html", // Bucket name is included
            "errorPath": "errors/404.html"
          }
        },
        "preservePath": false // Use current pathname as base directory
      },
      "upload": {
        "publicRead": true, // Will not clobber existing ACLs
        "publicRead": 0, // Remove ACL without affecting other ACLs (GCP only)
        /* OR */
        "acl": "authenticatedRead" | "bucketOwnerFullControl" | "bucketOwnerRead" | "private" | "projectPrivate" | "publicRead", // PredefinedAcl (optional)

        "active": false, // Implicity "publicRead: true" except when explicitly "publicRead: false"

        /* Optional - UploadOptions */
        "options": {
          "contentType": "text/html",
          "predefinedAcl": "publicRead" // Supplementary are public

          "metadata": {}, // UploadMetadata
          /* OR */
          "customMetadata": {} // Firebase
        }
      }
    },
    {
      "service": "ibm",
      "bucket": "squared-004",
      "credential": {
        "apiKeyId": "**********",
        "serviceInstanceId": "**********",
        "region": "us-south",
        "endpoint": "https://s3.us-south.cloud-object-storage.appdomain.cloud", // Same as region (optional)
      },
      "admin": {
        "configBucket": { /* Same as AWS */ }
      },
      "upload": { /* Same as AWS */ }
    },
    {
      "service": "oci",
      "bucket": "squared-005", // New buckets are private when using S3 API
      "credential": {
        "region": "us-phoenix-1",
        "namespace": "abcdefghijkl",
        "accessKeyId": "**********",
        "secretAccessKey": "**********",
      },
      "admin": {
        "configBucket": { /* Same as AWS */ }
      },
      "upload": { /* Same as AWS */ }
    },
    {
      "service": "minio", // https://docs.min.io/minio/baremetal
      "bucket": "squared-006",
      "credential": {
        "accessKey": "**********",
        "secretKey": "**********",
        "endPoint": "127.0.0.1",
        "port": 9000, // Required
        "useSSL": false, // Required with "http"
        "region": "us-west-1" // Default is "us-east-1"
      },
      "admin": {
        "publicRead": true, // Will create a compatible S3 "public-read" policy
        /* OR */
        "acl": "readonly" | "writeonly" | "readwrite", //  // Will create a compatible MinIO policy

        "emptyBucket": true,
        "recursive": false, // Used with emptyBucket (default is "true")

        "configBucket": {
          "policy": "readonly" | "writeonly" | "readwrite", // Canned ACL (https://min.io/docs/minio/linux/administration/identity-access-management/policy-based-access-control.html)
          "policy": "public-read", // Same as AWS
          "policy": {
            "Version": "2012-10-17",
            "Statement": [{
              "Sid": "PublicRead",
              "Effect": "Allow",
              "Principal": {
                "AWS": ["*"]
              },
              "Action": ["s3:GetObject"],
              "Resource": ['arn:aws:s3:::bucketName/*']
            }
          }
        }
      },
      "upload": {
        "publicRead": true, // S3 request header "x-amz-acl" to "public-read"
        /* OR */
        "acl": "authenticated-read" | "aws-exec-read" | "bucket-owner-full-control" | "bucket-owner-read" | "private" | "public-read" | "public-read-write", // S3 Object Canned ACL

        "endpoint": "http://squared.min.io/squared-006", // Required when different from credential

        /* Optional - ItemBucketMetadata */
        "options": {
          "Content-Type": "image/webp" // Supplementary objects
        },
        "metadata": {
          "Content-Type": "image/png" // Primary object
        }
      }
    }
  ]
}

Creating a NPM scoped package with action function handlers can be used to perform cloud transactions.

// ICloudServiceClient

@oci-scoped/client
@oci-scoped/upload (optional)
@oci-scoped/download (optional)

{
  "cloudStorage": [
    {
      "service": "@oci-scoped"

      /* Same as above */
    }
  }
}

Samples can be found in the @squared-functions/cloud directory.

squared.saveAs("index.zip", {
  config: "http://localhost:3000/chrome/bundle.yml",
  saveAs: {
    html: {
      cloudStorage: [{ // Create static website
        service: "aws-v3",
        bucket: "squared-001",
        credential: {
          credentials: { // Preferred
            accessKeyId: "**********", // Only access key logins are supported with v3
            secretAccessKey: "**********",
            sessionToken: "" // Optional
          },
          /* ...accessKeyId = Will be copied into "credentials" */
          region: "us-west-2"
        },
        upload: {
          active: true,
          endpoint: "https://squared-001.s3.us-west-2.amazonaws.com", // Optional
          overwrite: true
        }
      }]
    },
    image: { // Non-element images using url() method
      cloudStorage: [{
        service: "aws",
        bucket: "squared-001",
        settings: "main",
        upload: {
          active: true
        }
      }]
    }
  }
});

Data Source

Static content can be generated using an AssetCommand with the "dataSource" property to perform basic text and attribute replacement.

Cloud

Each DocumentDB provider has a different query syntax. Consulting their documentation is recommended if you are writing advanced queries.

* Amazon DynamoDB
  - https://aws.amazon.com/dynamodb (25GB + 25 RCU/WCU)

  + npm i aws-sdk (aws)
    <!-- OR -->
  + npm i @aws-sdk/client-dynamodb (aws-v3)
  + npm i @aws-sdk/lib-dynamodb

* Microsoft Cosmos DB
  - https://azure.microsoft.com/en-us/services/cosmos-db (5GB + 400RU/s)

  + npm i @azure/cosmos (azure/az)

* Google Firestore / Datastore / BigQuery / Realtime Database / Bigtable / Spanner
  - https://cloud.google.com/firestore (1GB + 50K/20K r/w@day)
  - https://cloud.google.com/bigquery (10GB + 1TB queries/month)
  - https://firebase.google.com/products/realtime-database (1GB + 10GB)
  - https://cloud.google.com/bigtable (Paid)
  - https://cloud.google.com/spanner (Paid)

  + npm i @google-cloud/firestore (gcp/gcloud)
  + npm i @google-cloud/datastore
  + npm i @google-cloud/bigquery
  + npm i @google-cloud/bigtable
  + npm i @google-cloud/spanner
  + npm i firebase
  + npm i firebase-admin (optional)

* IBM Cloudant
  - IBM: https://www.ibm.com/cloud/cloudant (1GB + 20/10 r/w@sec)

  + npm i @cloudant/cloudant (ibm) [Deprecated]
    <!-- OR -->
  + npm i @ibm-cloud/cloudant (ibm-v1)

* Oracle Autonomous DB
  - https://www.oracle.com/autonomous-database (20GB)
  - https://www.oracle.com/autonomous-database/autonomous-json-database (Paid - 1TB)

  + npm i oracledb (oci)

* MongoDB Atlas
  - https://www.mongodb.com/atlas/database
  - URI authentication only when using MongoDB v3

  + npm i mongodb (atlas)

Environment variables can also be used for authorization.

* AWS
  - AWS_ACCESS_KEY_ID
  - AWS_SECRET_ACCESS_KEY
  - AWS_SESSION_TOKEN
  - AWS_REGION

* Azure (Unofficial)
  - AZURE_COSMOS_ENDPOINT
  - AZURE_COSMOS_KEY
  - process.env.apply (squared.json)

* GCP
  - GOOGLE_APPLICATION_CREDENTIALS
  - npm i firebase-admin

* IBM
  - CLOUDANT_URL
  - CLOUDANT_APIKEY
  - CLOUDANT_USERNAME
  - CLOUDANT_PASSWORD
  - serviceName = CLOUDANT

interface CloudDatabase {
    source: "cloud";
    name?: string;
    table?: string; // Required except with BigQuery + Realtime Database
    id?: string;
    query?: string | PlainObject | any[];
    value?: string | ObjectMap<string | string[]>; // Uses innerHTML for replacement when undefined
    index?: number;
    limit?: number;

    ignoreEmpty?: boolean; // Do not interpret empty results

    template?: string; // chrome.settings.directory.template (base directory + users/username?)
    encoding?: BufferEncoding; // utf-8 (default)

    params?: unknown[]; // Client params
    options?: PlainObject; // Client config (Coercible: Date | RegExp | URL | function)

    partitionKey?: string; // AWS + Azure + IBM + OCI

    // Function callback
    postQuery?: string | Function; // (ResultArray[], dbObject, require)
    preRender?: string | Function; // (string, dbObject, require)
    whenEmpty?: string | Function; // (EmptyArray[], dbObject, require)

    viewEngine?: {
        name: string; // NPM package name
        singleRow?: boolean; // Template data is sent in one pass using an Array[]
        outputEmpty?: boolean; // Pass empty results to template engine
        options?: {
            compile?: PlainObject; // template = engine.compile(value, options)
            output?: PlainObject; // template({ ...options, ...result[index] })
        };
    };

    // Caching - user@server/database
    credential: {
        uuidKey?: string; // Faster and better caching (UUID v4)
    };
    ignoreCache?: boolean | 0 | 1; // 0 - reset cache expiration | 1 - purge current + store latest
}

You can also use named callbacks for "postQuery" and "preRender" anywhere inside the HTML. It is more readable than inside a configuration file and can be reused for queries with the same mapping or formatting.

Instructions for coercing string values into native objects can be found in the MongoDB section.

NOTE: Using "template" (external) is the same as "value" (inline) except the reusable content is stored inside a local file inside a predefined template directory.

interface CloudDatabase {
    /* AWS */
    key?: Key;
    update?: UpdateItemInput | UpdateCommandInput /* v3 */; // Uses Key object or partitionKey/id (precedes GetItemInput during read operation)

    /* Azure */
    update?: PatchRequestBody; // partitionKey + id (JSON Patch: http://jsonpatch.com/)

    /* GCP */
    update?: StandardMap | string; // Firestore (id + single) | Firebase (query + multiple) | Spanner (DML)

    /* IBM-v1 */
    update?: PostDocumentParams; // Uses "postDocument" (id)

    /* OCI */
    update?: StandardMap; // Uses "replaceOne" (id)

    /* Atlas */
    update?: UpdateFilter<Document>; // Update document during a read operation
}

Update occurs during a get operation and will return the item with the latest changes only if it exists.

// "postQuery-example" is a custom identifier (chrome -> eval_template: true)

<script type="text/template" data-chrome-template="data::postQuery-example">
async function (items, dbObject) { // items - PlainObject[]
  if (items.length) {
    return await fetch("/db/url", { method: "POST", body: JSON.stringify(items) }).then(result => result.map(item => ({ name: item.key, value: item.value })));
  }
  return null; // "items" will display unmodified when not an array
}
</script>

<script type="text/template" data-chrome-template="data::preRender-example">
function (value, dbObject) { // value - string
  return value.replaceAll("<", "&lt;");
}
</script>

<script type="text/template" data-chrome-template="data::whenEmpty-example">
function (result, dbObject) { // result - PlainObject[]
  result[0] = { value: "Empty" }; // Array.length is 0
}
</script>

View engines with a "compile" template string to function (e.g. EJS) can be used instead for "text" and "attribute". Results from any data source are treated as an array with multiple rows being concatenated into one string.

/* AWS: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GettingStarted.NodeJs.html */
{
  "selector": "main p",
  "type": "text",
  "dataSource": {
    "source": "cloud",
    "service": "aws",
    "credential": { // Coercible: Date | RegExp | function
      "accessKeyId": "**********",
      "secretAccessKey": "**********",
      "region": "us-east-1", // Endpoint specified (optional)
      "endpoint": "https://dynamodb.us-east-1.amazonaws.com" // Local development (required)
    },
    "table": "demo",
    "query": {
      "KeyConditionExpression": "#name = :value",
      "ExpressionAttributeNames": { "#name": "id" },
      "ExpressionAttributeValues": { ":value": "1" }
    },
    "limit": 1, // Optional
    "value": "<b>${title}</b>: ${description}", // Only one field per template literal (optional)
    /* OR */
    "value": "`<b>${this.title}</b>: ${this.description} (${this.index * 2})`" // Function template literal - (chrome.settings.eval.function)
  }
}

/* Azure: https://docs.microsoft.com/en-us/azure/cosmos-db/sql-query-getting-started */
{
  "selector": "main p",
  "type": "text",
  "dataSource": {
    "source": "cloud",
    "service": "azure",
    "credential": {
      "endpoint": "https://squared-001.documents.azure.com:443",
      "key": "**********"
    },
    "name": "squared", // Database name (required)
    "table": "demo",
    "partitionKey": "Pictures", // Optional
    "query": "SELECT * FROM c WHERE c.id = '1'", // OR: storedProcedureId + partitionKey? + params?
    "query": {
      "query": "SELECT * FROM c WHERE c.lastName = @lastName AND c.address.state = @addressState", // SqlQuerySpec
      "parameters": [
        { "name": "@lastName", "value": "Wakefield" },
        { "name": "@addressState", "value": "CA" }
      ]
    },
    "value": "<b>${__index__}. ${title}</b>: ${description}" // "__index__": Result row index value
  }
}

/* GCP: https://firebase.google.com/docs/firestore/query-data/queries */
{
  "selector": "main p",
  "type": "text",
  "dataSource": {
    "source": "cloud",
    "service": "gcp",
    "product": "firestore", // Only GCP (recommended)
    "credential": {
      "keyFilename": "./gcp.json",
      "projectId": "squared" // Optional
    },
    "table": "demo",

    "query": [
      ["group", "==", "Firestore"], ["id", "==", "1"] // API: where (required) (deprecated)
    ],
    /* endAt + endBefore + limit + limitToLast + offset + orderBy + select + startAfter + startAt + where + withConverter */
    "query": [
      ["where", "group", "==", "Firestore"], // Method name + arguments (required)
      ["where", "id", "==", "1"],
      ["limitToLast", 2],
      ["orderBy", "title", "asc"]
    ],

    "orderBy": [
      ["title", "asc"] // Optional
    ],

    "value": "{{if !expired}}<b>${title}</b>: ${description}{{else}}Expired{{end}}", // Non-nested single conditional truthy property checks

    /* golang - html/template comparison syntax */
    "value": "{{if not expired}}<b>${title}</b>: ${description}{{else}}Expired{{end}}", // Case sensitive
    "value": "{{if and (user.total) (ge user.total postMin) (lt user.total postMax)}}<b>${title}</b>: ${description}{{else if (eq user.total 0)}}Expired{{end}}" // Parenthesis and spaces between conditions are required (and/or/not)
  }
}

/* Datastore: https://googleapis.dev/nodejs/datastore/latest */
{
  "selector": "main p",
  "type": "text",
  "dataSource": {
    "source": "cloud",
    "service": "gcp",
    "product": "datastore",
    "credential": {
      "keyFilename": "./gcp.json",
      "projectId": "squared" // Optional
    },

    "keys": [["kind", "name"]], // PathType[] | KeyOptions | string (required)
    "limit": 5, // Optional
    "removeEmpty": false, // Optional

    "kind": "Task", // Optional
    /* end + filter + groupBy + hasAncestor + limit + offset + order + select + start */
    "query": [
      ["filter", "done", "=", false], // Method name + arguments (required)
      ["filter", "priority", ">=", 4],
      ["order", "priority", { "descending": true }]
    ],

    "options": {}, // RunQueryOptions (optional)

    "viewEngine": "ejs", // npm i ejs (https://ejs.co/#docs)
    "template": "sub_dir/ejs/content.ejs", // chrome.settings.directory.template (base directory + users/username?)
    "encoding": "utf-8" // Optional
  }
}

/* BigQuery: https://googleapis.dev/nodejs/bigquery/latest */
{
  "selector": "main p",
  "type": "text",
  "dataSource": {
    "source": "cloud",
    "service": "gcp",
    "product": "bigquery",
    "credential": {
      "keyFilename": "./gcp.json",
      "projectId": "squared" // Optional
    },
    "query": "SELECT name, count FROM `demo.names_2014` WHERE gender = 'M' ORDER BY count DESC LIMIT 10", // Required
    "value": "<b>${title}</b>: ${description}"
  }
}

/* Realtime Database: https://firebase.google.com/docs/database/web/lists-of-data#filtering_data */
{
  "selector": "main p",
  "type": "text",
  "dataSource": {
    "source": "cloud",
    "service": "gcp",
    "product": "firebase", // Recommended
    "credential": {
      "apiKey": "**********",
      "authDomain": "<project-id>.firebaseapp.com",
      "databaseURL": "https://<database-name>.firebaseio.com",
      "product": "firebase" // Required with using GOOGLE_APPLICATION_CREDENTIALS
    },

    "query": "path/to/ref",  // Required
    /* endBefore + endAt + equalTo + limitToFirst + limitToLast + orderByChild + orderByKey + orderByPriority + orderByValue + startAt + startAfter */
    "orderBy": [
      ["orderByChild", "path/to/child"], // Optional
      ["startAfter", 5, "name"],
      ["limitToFirst", 1 /* Leading path is removed */]
    ],

    "viewEngine": "ejs" // Using a view engine is recommended due to object structure
  }
}

/* Bigtable: https://googleapis.dev/nodejs/bigtable/latest */
{
  "selector": "main p",
  "type": "text",
  "dataSource": {
    "source": "cloud",
    "service": "gcp",
    "product": "bigtable",
    "credential": {
      "keyFilename": "./gcp.json",
      "projectId": "squared", // Optional
      "apiEndpoint": "localhost:8086" // Optional
    },

    "name": "squared", // Instance (required)
    "table": "demo", // Required

    "query": {}, // Overrides "filter" options attribute (optional)

    "id": "rowKey1", // Uses "get" [single] (required)
    "columns": ["column1", "column2"], // Optional
    "options": { filter: {} }, // GetRowOptions (optional)
    /* OR */
    "options": { filter: {} }, // Uses "getRows" [multiple] + GetRowOptions (optional)

    "value": "<b>${title}</b>: ${description}"
  }
}

/* Spanner: https://googleapis.dev/nodejs/spanner/latest */
{
  "selector": "main p",
  "type": "text",
  "dataSource": {
    "source": "cloud",
    "service": "gcp",
    "product": "spanner", // Recommended
    "credential": {
      "keyFilename": "./gcp.json",
      "projectId": "squared" // Optional
    },

    "name": "squared", // Instance (required)
    "database": "sample", // Required

    "table": "demo",
    "query": {
      "columns": [], // transaction.ReadRequest (required)
      "keys": []
    },
    "update": {}, // object | object[] (optional)
    "updateType": 0 | 1 | 2, // 0 - update | 1 - insert | 2 - replace
    /* OR */
    "query": "SELECT 1" | { "sql": "SELECT 1", "params": {} }, // ExecuteSqlRequest (required)
    "update": {}, // DML statement (Same as "query")

    "options": {
      "databasePool": {}, // session-pool.SessionPoolOptions
      "databaseQuery": {}, // protos.IQueryOptions
      "tableRead": {}, // transaction.TimestampBounds
      "tableUpdate": {} // table.UpdateRowsOptions
    },

    "value": "<b>${title}</b>: ${description}"
  }
}

/* IBM-v1: https://github.com/IBM/cloudant-node-sdk#readme */
{
  "selector": "main p",
  "type": "text",
  "dataSource": {
    "source": "cloud",
    "service": "ibm-v1",
    "credential": {
      "serviceName": "squared", // Cloudant instance (required)

      "username": "**********", // Basic auth
      "password": "**********",
      /* OR */
      "apikey": "**********", // IAM tokem

      "url": "https://<username>.cloudantnosqldb.appdomain.cloud" // External endpoint (required)
    },
    "table": "demo", // "db" property during transaction

    /* Find */
    "query": {
      "selector": {
        "id": { "$eq": "1" } // Required
      }
    },
    /* Search */
    "query": {
      "ddoc": "demo-doc", // PostSearchParams
      "index": "demo-index", // https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-cloudant-search#index-functions
      "partitionKey": "", // Optional
      "query": "id:'1' AND title:'Bristol'" // Lucene syntax
    },
    /* View */
    "query": {
      "ddoc": "demo-doc", // PostViewParams
      "view": "demo-view", // https://cloud.ibm.com/docs/Cloudant?topic=Cloudant-using-views
      "partitionKey": "" // Optional
    },
    /* Partition */
    "partitionKey": "Partition1", // When "query" undefined (PostPartitionAllDocsParams)

    "value": "<b>${title}</b>: ${description}"
  }
}

/* IBM (deprecated): https://github.com/cloudant/nodejs-cloudant#readme */
{
  "selector": "main p",
  "type": "text",
  "dataSource": {
    "source": "cloud",
    "service": "ibm",
    "credential": {
      "username": "**********", // Legacy credentials (alias for "account")
      "password": "**********",
      "url": "https://<username>.cloudantnosqldb.appdomain.cloud",
      /* OR */
      "url": "https://<username>:<password>@<username>.cloudantnosqldb.appdomain.cloud", // Basic auth
      /* OR */
      "iamApiKey": "**********", // IAM tokem
      "url": "https://<username>.cloudantnosqldb.appdomain.cloud"
    },
    "table": "demo",

    "query": { "selector": { "id": { "$eq": "1" } } }, // Find
    /* OR */
    "partitionKey": "Partition2", // All documents in partition

    "limit": 5,
    "value": "<b>${title}</b>: ${description}"
  }
}

/* OCI: https://docs.oracle.com/en/database/oracle/simple-oracle-document-access/adsdi/oracle-database-introduction-simple-oracle-document-access-soda.pdf */
{
  "selector": "main p",
  "type": "text",
  "dataSource": {
    "source": "cloud",
    "service": "oci",
    "credential": {
      "username": "**********", // alias for "user"
      "password": "**********",
      "connectionString": "tcps://adb.us-phoenix-1.oraclecloud.com:1522/abcdefghijklmno_squared_high.adb.oraclecloud.com?wallet_location=/Users/Oracle/wallet"
    },
    "table": "demo",

    "query": "SELECT * from demo WHERE id = '1'", // Column names might be UPPERCASED
    "query": "SELECT d.* from demo NESTED json_document COLUMNS(id, title, description) d WHERE d.id = '1'", // SODA
    /* OR */
    "query": { "id": { "$eq": "1" } }, // SODA

    "options": { "resultSet": true } // Optional

    "viewEngine": "ejs", // npm i ejs
    "preRender": "./pre-render.cjs", // Local files are supported (chrome -> eval_userconfig: true)
    "postQuery": "postQuery-example",
    "whenEmpty": "whenEmpty-example"
  }
}

/* Atlas: https://docs.mongodb.com/compass/master/query/filter */
{
  "selector": "main p",
  "type": "attribute",
  "dataSource": {
    "source": "cloud",
    "service": "atlas",
    "uri": "mongodb+srv://<username>:<password>@cluster0.abcde.mongodb.net/<database name>", // Required
    "name": "squared", // Database name (optional)
    "table": "demo", // Required

    /* MongoClientOptions */
    "credential": {
      "username": "**********", // Optional
      "password": "**********",
      /* OR */
      "auth": {
        "username": "**********", // node-mongodb v4
        "password": "**********"
      }
    },

    "query": {
      "id": {
        "$eq": "{{id}}"
      },
      "start_date": {
        "$gt": "new Date('2021-01-01')" // new Date("2021-01-01")
      },
      "$in": ["new RegExp(^mongodb, i)"], // Quotes are optional [/^mongodb/i]
      "$where": "function() { return this.name == 'mongodb.com'; }" // async is supported
    },
    "value": {
      "html_attribute": "db_column" // Required
    }
  }
}

// Retrieval using ID is supported by all providers

{
  "selector": "main img",
  "type": "attribute",
  "dataSource": {
    "source": "cloud",
    "service": "azure",
    "credential": "db-main",
    "name": "squared", // Azure (required)
    "table": "demo",

    "partitionKey": "Pictures", // Azure and IBM (optional)
    "id": "2", // OCI (server assigned)

    /* AWS */
    "key": { // Alias for partitionKey
      "id": "2", // Same
      "type": "image" // Additional fields are supported
    }

    /* Result: { src: "", other: {} } */
    "value": {
      "src": "src", // Use direct property access
      "alt": "{{if not expired}}other.alt{{else}}:text(Expired){{end}}", // Only one conditional per attribute
      "style": [":join(; )" /* " " (optional) */, "other.style[0]", "other.style[1]", ":text(display: none)"] // Same as: [":join(; )", "other.style", ":text(display: none)"]
    }
  }
}

Some queries use an optional parameters array (params) or configuration object (options) which is sent with the query when applicable. If you require this advanced usage then further instructions can be found in the database provider documentation.

When in development mode you can save read units by setting a timeout value for the DB cache.

// squared.json

"cloud": {
  "cache": {
    "aws": 0, // No cache per reload
    "azure": 60, // 1 minute
    "gcp": {
      "timeout": "1d", // 1 day
      "when_empty": true
    }
  }
}

Results are cached using the supplied credentials and queries will individually be cleared when the amount of time has expired.

Reusing configuration templates is possible using URL query parameters. Output values can be replaced using the {{param}} syntax.

// http://localhost:3000/project/index.html?table=demo&id=1

{
  "service": "azure",
  "credential": "db-main",
  "name": "squared",
  "table": "{{table}}",
  "partitionKey": "Pictures",
  "query": "SELECT * FROM c WHERE c.id = '{{id}}'",
  "value": "<b>${title}</b>: ${description}" // Not parsed
}

MongoDB

Local development may be faster using MongoDB instead of a cloud DocumentDB. It is completely free to use and includes a GUI data explorer.

* MongoDB Community Server
  - https://www.mongodb.com/try/download/community
  - https://docs.mongodb.com/drivers/node/current/fundamentals/authentication/mechanisms
  - https://docs.mongodb.com/compass/master/query/filter (query)

  - npm i mongodb

MongoDB Atlas installations also use the "mongodb" source format. All MongoDB authentication mechanisms are supported.

interface MongoDBDataSource {
    source: "mongodb";

    /* Required */
    uri?: string; // Connection string
    /* OR */
    credential?: string | MongoDBCredential;

    /* Optional */
    query?: Filter<Document> | { value: DocumentFilter, options: CommandOperationOptions }; // "value" is required when using "options"
    id?: string; // Uses ObjectId
    sort?: Sort | string | { value: Sort, direction: SortDirection };

    value?: string | ObjectMap<string | string[]>;

    options?: MongoClientOptions; // Overriden by credential
    client?: {
        db?: DbOptions; // Used as options with "name"
        collection?: CollectionOptions; // Used as options with "table"
    };
    execute?: {
        insert?: BulkWriteOptions; // Used as options with "update - insert"
        update?: UpdateOptions;
    };
    aggregate?: Document[] | { pipeline: Document[], options: AggregateOptions };

    cascade?: string; // UriDataSource
    fallback?: unknown; // object

    update?: UpdateFilter<Document> | OptionalUnlessRequiredId<unknown>[];
    updateType?: 0 | 1 | 2 | 3; // 0 - update | 1 - insert | 2 - replace | 3 - delete

    /* DB shared */
    usePool?: boolean | UUID; // UUIDv1-5 + db.settings.user_key (username@server/database)
    parallel?: false; // Used for batched queries (implicit: true)
    streamRow?: boolean | ((row: unknown) => Undef<Error | false>)>; // Not cached + postgres not supported
    willAbort?: boolean; // Module abort is called bypassing settings

    /* Same as CloudDatabase */
}

NOTE: Enabling usePool with a UUID key will also copy the value into credential.uuidKey. (without override)

// http://localhost:3000/project/index.html?id=1

{
  "selector": "main img",
  "type": "attribute",
  "dataSource": {
    "source": "mongodb",

    "uri": "mongodb://<username>@<password>:localhost:27017", // Recommended
    /* OR */
    "credential": { // Same as cloud settings "db-main"
      "server": "localhost:27017", // OR: 0.0.0.0
      /* OR */
      "hostname": "cluster0.abcdef.mongodb.net", // Required
      "port": 8080, // Default is "27017"

      "username": "**********", // Passed as URI query parameter (node-mongodb v3)
      "password": "**********",
      /* OR */
      "auth": {
        "username": "**********", // Passed in as options.auth (node-mongodb v4)
        "password": "**********"
      },

      /* Optional */
      "protocol": "mongodb+srv:", // "mongodb:" (default)

      "authMechanism": "MONGODB-X509",
      "authMechanismProperties": {}, // AuthMechanismProperties | string (SERVICE_NAME)
      "authSource": "",

      "tlsCertificateFile": "/path/tsl/x509/cert.pem", // node-mongodb v4
      "tlsCertificateKeyFile": "/path/tsl/x509/key.pem",
      "tlsCAFile": "",
      "tlsCertificateKeyFilePassword": "",
      "tlsAllowInvalidHostnames": false,
      "tlsAllowInvalidCertificates": false,
      "tlsInsecure": false,
      /* OR */
      "sslKey": "/path/ssl/x509/key.pem",
      "sslCert": "/path/ssl/x509/cert.pem",
      "sslCA": "",
      "sslPass": "",
      "sslValidate": true
    },

    "query": {
      "id": {
        "$eq": "{{id}}"
      },
      "name": {
        "$regex": "mongodb.*\\.com", // $regex: /mongodb.*\.com/si
        "$options": "si"
      },
      "start_date": {
        "$gt": "new Date('2021-01-01')" // new Date("2021-01-01")
      },
      "$in": ["new RegExp(^mongodb, i)"], // Quotes are optional [/^mongodb/i]
      "$where": "function() { return this.name == 'mongodb.com'; }" // "async" is supported
    },
    "value": {
      "src": "column_src", // Required
      "alt": "column_alt"
    },

    "usePool": true, // Optional
    "options": {
      "minPoolSize": 0,
      "maxPoolSize": 10
    }
  }
}

// IF conditional to completely remove an element (outerHTML)
{
  "selector": "main div",
  "type": "display",
  "dataSource": {
    "source": "mongodb",
    "uri": "mongodb://localhost:27017",
    "removeEmpty": true, // Includes invalid conditions (optional)
    /* Required */
    "value": "attr1", // Remove when: null or undefined
    "value": "-attr2", // Remove when: attr2=falsey
    "value": "+attr3", // Remove when: attr3=truthy
    "value": ["attr1" /* AND */, ":is(OR)", "-attr2" /* OR */, "-attr3" /* OR */, ":is(AND)", "+attr4" /* AND */] // Remove when: attr1=null + attr2|attr3=falsey + attr4=truthy
  }
}

To remove an element all AND conditions have to be TRUE and one OR per group is TRUE. Using a view engine is recommended if you require a more advanced conditional statement.

Returning an empty result or a blank string (view engine) is FALSE.

Redis

Using a key-value store is sufficient for generating simple static web pages.

  - https://redis.io/download (Linux)
  - https://redis.com/try-free (1 database + 30MB)

  - npm i redis

interface RedisDataSource {
    source: "redis";

    uri: string; // redis://localhost:6379 | redis://<username>:<password>@hostname:6380
    username?: string;
    password?: string;
    /* OR */
    credential?: string | ServerAuth;

    key?: string | string[];
    format?: "HASH" | "JSON"; // Default is "HASH"
    query?: string; // jsonpath + jmespath

    cascade?: string; // UriDataSource
    fallback?: unknown; // object

    field?: string; // GET - hash
    path?: string; // MGET - json

    /* https://github.com/redis/node-redis/tree/master/packages/search */
    search?: {
        index: string; // Preexisting schema
        /* OR */
        schema: CreateSchema; // Temporary schema
        schema: string; // chrome.settings.directory.schema (base directory + users/username?)
        /* OR */
        schema: CreateSchema; // Create schema as "index" (Not recommended - use CLI)
        index: string;

        query: string;
        options?: { // CreateOptions
            ON?: "HASH" | "JSON";
            PREFIX?: string | Array<string>;
        };
    };
    aggregate?: { /* Same as search */ };

    /* Advanced configuration (optional) */
    format?: "HKEYS" | "HVALS";
    update?: RedisSetValue | RedisSetValue[] | RedisJSONValue | RedisJSONValue[]; // @squared-functions/types/lib/db

    options?: {
        client?: RedisClientOptions;
        command?: RedisCommandOptions;
        get?: PlainObject;
        search?: SearchOptions;
        aggregate?: IAggregateOptions;
    };

    /* Same as MongoDBDataSource */
}

// http://localhost:3000/project/index.html?file=demo

{
  "selector": "main img",
  "type": "attribute",
  "dataSource": {
    "source": "redis",

    "uri": "redis://localhost:6379",
    "username": "**********",
    "password": "**********",
    "database": 1,
    /* OR */
    "uri": "redis://<username>:<password>@localhost:6379/<database>",
    /* OR */
    "credential": "main";
    /* OR */
    "credential": {
      "protocol": "", // Optional
      "server": "localhost:6379",
      "username": "**********",
      "password": "**********",
      "database": 1
    }

    "key": "demo:1",
    "query": "$.name", // jsonpath + jmespath (optional)
    /* OR */
    "search": {
      "schema": {
        "name": {
          "type": "TEXT", // SchemaFieldTypes.TEXT
          "sortable": true
        },
        "state": "TAG", // SchemaFieldTypes.TAG
        "age": "NUMERIC" // SchemaFieldTypes.NUMERIC
      },
      /* OR */
      "schema": "sub_dir/{{file}}.json", // yaml + json5 + toml + xml + cjs (parent: "schema")

      "query": "@state:{CA}",
      "options": {
        "ON": "HASH", // JSON
        "PREFIX": "noderedis:demo" // Optional
      }
    },
    "value": {
      "src": "column_src" // Required
    },

    "usePool": true, // Optional
    "options": {
      "client": {
        "isolationPoolOptions": {
          "min": 0,
          "max": 10
        }
      }
    }
  }
}

/* Auth - chrome.db */
{
  "redis": {
    "main": {
      "protocol": "", // Default is "redis:"
      "hostname": "", // Default is "localhost"
      "port": "", // Default is "6379"
      "username": "",
      "password": "",
      "database": 0 // SELECT index (number > 0)
    }
  },
  "sql": {
    "main": {
      "server": "localhost:3306", // Alias for hostname + port
      "username": "**********",
      "password": "**********",
      "database": "demo"
    }
  }
}

NOTE: Redis search will only return the "value" object with the id field appended as "__id__";

MySQL + PostgreSQL + Oracle + MSSQL

Querying SQL databases can be achieved using a simple SQL statement and optional parameters. These providers were designed to perform parallel SELECT statements and are not transactional.

  - MySQL Community Server
  - https://dev.mysql.com/downloads/mysql
  - https://www.npmjs.com/package/mysql#connection-options (Auth)

  - npm i mysql2

  - PostgreSQL
  - https://www.postgresql.org/download
  - https://node-postgres.com/features/connecting (Auth)

  - npm i pg

  - Oracle Database XE
  - https://www.oracle.com/database/technologies/xe-downloads.html
  - http://oracle.github.io/node-oracledb/doc/api.html#-16-connection-handling (Auth)

  - npm i oracledb

  - SQL Server
  - https://www.microsoft.com/en-us/sql-server/sql-server-downloads
  - https://tediousjs.github.io/tedious/api-connection.html (Auth)

  - npm i tedious
  - npm i tedious-connection-pool2 (optional)

interface SQLDataSource {
    source: "mysql" | "postgres" | "oracle" | "mssql";

    uri: "mysql://<username>:<password>@localhost:3306/database"; // uri (MySQL: Auth)
    uri: "postgresql://<username>:<password>@localhost:5432/database"; // connectionString (Postgres: Auth)
    uri: "<username>:<password>@localhost:1521/database"; // connectString | poolAlias (Oracle: Auth)
    uri: "<username>:<password>@localhost:1433/database"; // authentication.options + server + authentication (MSSQL: Auth)
    /* OR */
    credential: "main", // chrome.db[source]
    /* OR - Universal */
    credential: {
        hostname: "localhost", // Required
        port: 3306,
        username: "**********", // Required
        password: "**********",
        database: "example"
    };
    /* OR - Auth */
    credential: {
        host: "localhost", // mysql + postgres
        port: 5432,
        user: "**********",
        password: "**********",
        database: "example"
    };
    credential: {
        connectString: "localhost:1521/example", // oracle
        user: "**********",
        password: "**********"
    };
    credential: {
        server: "localhost", // mssql
        options: {
            port: 1433,
            database: "example",
            encrypt: true, // Azure
            trustServerCertificate: true // Local development
        },
        authentication: {
            type: "default",
            options: {
                userName: "**********",
                password: "**********"
            }
        }
    };

    query: "SELECT * FROM `table` WHERE `id` = ? AND `value` = ?";
    query: "./sub_dir/statement.sql"; // Extension ".sql" + chrome.settings.directory.sql (base directory + users/username?)

    params?: [1, "escaped"];

    /* mssql - http://tediousjs.github.io/tedious/parameters.html */
    params?: { "a": { value: "1", type: "VarChar", options: { length: 50 } }, "b": 2 /* Implicit: Int */ }; // MSSQLRequestParameters
    params?: [{ name: "c", type: "Decimal", value: 12.345, options: { precision: 10, scale: 2 } }];
    params?: [{ name: "d", type: "TVP", value: { columns: MSSQLRequestParameters[]; rows: unknown[][] }];
    params?: {
      input: Null<MSSQLRequestParameters>; // Two keys only for object detection (required)
      output: Null<MSSQLRequestParameters>; // Last row in result (data["__returnvalue__"] = true)
    };
    params_output?: MSSQLRequestParameters; // Deprecated
    storedProc?: boolean; // Call stored procedure
    usePool?: boolean | UUID | PoolConfig; // https://github.com/tediousjs/tedious-connection-pool + tedious 15 (auth)

    /* Same as MongoDBDataSource */
}

NPM packages can also be used to perform SQL statements.

// IDbSourceClient

{
  "dataSource": {
    "source": "sqlite-wrapper", // npm i sqlite-wrapper
    /* OR */
    "source": "@sqlite-scoped", // "client" appended + npm i @sqlite-scoped/client
    /* OR */
    "source": "@sqlite-scoped/query" // Settings uses full package name

    /* Same as above */
  }
}

Samples can be found in the