@stackql/openapi-doc-util
v1.4.0
Published
Command line utility to split OpenAPI documents into smaller, self contained, service oriented documents and prepare StackQL provider interfaces
Downloads
10
Readme
openapi-doc-util
Command line utility to split OpenAPI documents into smaller, self contained, service oriented documents and prepare StackQL provider interfaces Command line utility to help developers prepare and submit StackQL provider specs, see StackQL Readme
Installation
npm i @stackql/openapi-doc-util
yarn add @stackql/openapi-doc-util
Setup
from the package directory, run:
npm link
Background
The StackQL ORM provides a logical model to work with cloud resources similar to the way databases are organized into schemas. This object/resource hierarchy is summarized below:
provider/
├─ service/
│ ├─ resource/
│ │ ├─ fields
│ │ ├─ methods
an example would be:
google/
├─ compute/
│ ├─ instances/
│ │ ├─ id, name, status, ...
│ │ ├─ select, insert, delete, start, stop, ...
Enabling StackQL to interact with the google
provider using SQL semantics, for example:
Provider discovery operations such as..
SHOW RESOURCES IN google.compute;
DESCRIBE google.compute.instances;
Query operations such as..
SELECT status, COUNT(*) as num_instances
FROM google.compute.instances
WHERE project = 'myproject' and zone = 'us-west-1a'
GROUP BY status;
Provisioning operations such as creating a Compute Engine instance using an INSERT
statement or deprovisioning an instance using a DELETE
statement.
Usage
openapi-doc-util <command> [<arguments(s)>] [<flag(s)>]
Commands
validate
Validates an OpenAPI specification document.
split
Splits an Open API document into smaller, self contained, service scoped documents based upon a specified service discriminator (JSONPath expression relative to each operation). Takes an OpenAPI document as input and outputs the following structure:
{output_dir}/
├─ {provider_name}/
│ ├─ {provider_version}/
│ │ ├─ services/
│ │ │ ├─ {service_name}/
│ │ │ │ ├─ {service_name}.yaml
│ │ │ ├─ .../
The {service_name}.yaml
file is a self contained, OpenAPI specification document, containing only operations and components for the specified service.
provider-dev
Iterates through a directory of services (created using the split
function), generates an additional document for each service {service_name}-resources.yaml
and an index document for all services named provider.yaml
, this is useful for developing new StackQL providers. The output structure is as follows:
{api_doc_dir}/
├─ {provider_name}/
│ ├─ {provider_version}/
│ │ ├─ services/
│ │ │ ├─ {service_name}/
│ │ │ │ ├─ {service_name}.yaml
│ │ │ │ ├─ {service_name}-resources.yaml
│ │ │ ├─ .../
│ │ ├─ provider.yaml
The original OpenAPI documents are not modified.
The provider-dev
function will infer SQL routes for operations under each resources under the sqlVerbs
key, this should be reviewed by the developer, removing any duplicate or erroneous routes.
provider-build
Operates on the dev directory structure generated using the provider-dev
command, assembles a complete Open API specification per service including the StackQL resources for each service in each document under components/x-stackQL-resources
, and outputs the following structure to a new directory:
{output_dir}/
├─ {provider_name}/
│ ├─ {provider_version}/
│ │ ├─ services/
│ │ │ ├─ {service_name}/
│ │ │ │ ├─ {service_name}.yaml
│ │ │ ├─ .../
│ │ ├─ provider.yaml
The output from the provider-build
command can be used to test locally (see Test Using a Local Registry). Once you have tested locally you can raise a pull request to stackql/stackql-provider-registry.
Options
--svcDiscriminator
, -s
JSONPath expression OR svcName:servicename
The service discriminator option is used to determine how to split a large provider document into smaller, service scoped documents. The option is required for the split
command and ignored otherwise. If you do not wish to spilt the provider API spec, specify svcName:<servicename>
for this option which will define one service in the StackQL provider with the name provided in servicename.
Example:
-s "$['x-github'].category"
would split the given provider API spec into service documents by matching thex-github.category
value in each unique operation (combination of a path and an HTTP method) in API doc.
--resDiscriminator
, -r
JSONPath expression OR path_tokens
The resource discriminator option is used to determine how to identify StackQL resources in a given provider API spec. This option is used for the provider-dev
command and ignored otherwise.
Example:
-r "$['x-github'].subcategory"
would identify resources in the given provider API spec by matching thex-github.subcategory
value in each unique operation (combination of a path and an HTTP method) in API doc.
if path_tokens is specified for the --resDiscriminator
option, the resource name will be derived by joining the 'meaningful' path tokens (not equivalent to the service name) with an '_'. For instance a path of /sites/{site_id}/service-instances
would result in a resource name of sites_service_instances
assuming the service name was not sites
.
--methodkey
, -m
JSONPath expression
The method key option determines uniquely named operations which are mapped to SQL methods in the StackQL ORM. These methods are then mapped to default routes (SQL query and DML methods) in StackQL, the developer can override or update these mappings in the development docs which are outputted from the provider-dev
command. This option is used for the provider-dev
command and ignored otherwise.
If supplied it must be a JSONPath expression relative to the operation (http path and verb), if not supplied it will default to
operationId
in the OpenAPI specification for each operation.
--output
, -o
directory
The output directory option specifies the root directory to output files from the split
, or the root directory to output assembled provider specs from the provider-build
command. The default is the current working directory. If the directory exists in either case the program will exit with an error unless the --overwrite
option is set.
If not supplied it will default to the current working directory
--overwrite
The overwrite option specifies whether to overwrite files in the output directory. The default is false
.
--debug
, -d
debug flag which can be set for additional print statements.
Example
This example demonstrates creating a StackQL provider for github
and testing this provider locally using stackql
Example Project Structure
The following directory structure represents the target after an API dev workspace is set up and the StackQL provider for github
is built.
local-registry/
├─ ref/
│ ├─ github/
│ │ ├─ api.github.com.yaml
├─ dev/
│ ├─ github/
│ │ ├─ v0.1.0/
│ │ │ ├─ provider.yaml
│ │ │ ├─ services/
│ │ │ │ ├─ repos/
│ │ │ │ │ ├─ repos.yaml
│ │ │ │ │ ├─ repos-resources.yaml
│ │ │ │ ├─ .../
├─ src/
│ ├─ github/
│ │ ├─ v0.1.0/
│ │ │ ├─ provider.json
│ │ │ ├─ services/
│ │ │ │ ├─ repos/
│ │ │ │ │ ├─ repos-v0.1.0.json
local-registry/ref/github/api.github.com.yaml
is the source OpenAPI 3 specification for the github
api, this was sourced from GitHub.
The dev
directory contains the output of the dev docs generated by the openapi-doc-util split
command, which will then serve as input to the openapi-doc-util provider-dev
command.
The src
directory contains the output of the StackQL provider interface generated from the dev docs using openapi-doc-util provider-build
.
1. Split large spec into service specs
PowerShell:
cd local-registry
openapi-doc-util split `
-n github `
-v v0.1.0 `
-s "$['x-github'].category" `
-o .\dev `
./ref/github/api.github.com.yaml
Bash:
cd local-registry
openapi-doc-util split \
-n github \
-v v0.1.0 \
-s '$["x-github"].category' \
-o ./dev \
ref/github/api.github.com.yaml
2. Create dev provider docs
PowerShell:
cd local-registry
openapi-doc-util provider-dev `
-n github `
-v v0.1.0 `
-r "$['x-github'].subcategory" `
.\dev
Bash:
cd local-registry
openapi-doc-util provider-dev \
-n github \
-v v0.1.0 \
-r '$["x-github"].subcategory' \
./dev
Assemble final provider docs
PowerShell:
cd local-registry
openapi-doc-util provider-build `
-n github `
-v v0.1.0 `
-o .\src `
.\dev
Bash:
cd local-registry
openapi-doc-util provider-build \
-n github \
-v v0.1.0 \
-o ./src \
./dev
Test Using a Local Registry
cd local-registry
PROVIDER_REGISTRY_ROOT_DIR="$(pwd)"
REG_STR='{"url": "file://'${PROVIDER_REGISTRY_ROOT_DIR}'", "localDocRoot": "'${PROVIDER_REGISTRY_ROOT_DIR}'", "verifyConfig": {"nopVerify": true}}'
AUTH_STR='{"github": { "type": "null_auth" }}'
./stackql shell --auth="${AUTH_STR}" --registry="${REG_STR}"