Frosting Open Api Spec Ingestor

Table of Contents:

• Frosting Open Api Spec Ingestor - Table of Contents:
  • Build
  • Makefile Commands
  • Command Line Interface
    • Usage and Examples
    • Command options
    • Non Zero Exit Codes
  • Ingestor Transform Configuration File
  • Top Level File Structure:
  • Defining Source:
  • Defining Transformations:
  • Defining Operation Configuration:
  • Transformation Functions:
    • bremove
      • FunctionOptions:
    • preplace
      • FunctionOptions:
    • vadd
      • FunctionOptions:
    • vreplace
      • FunctionOptions:
  • Authoring New Transform Functions
    • Base Transform Class
    • Options Schema
    • Connecting New Transform Function to Ingest

Build

Suggested Node Version is >=18.10. We also use nvm for node version management.

To build, run make build

Then run ghoast in your shell and verify install ran correctly

Makefile Commands

make lint - Check linting of JS files

make lint-fix - Fixes all auto fixable linting issues

Command Line Interface

Usage and Examples

You can set an .env file with GITHUB_TOKEN=<your-token> to bypass providing the -t argument when using github urls.

# Example Success with --silent flag
ghoast ingest --input ./upstream/marketplaceOas.yaml --config=./transformations/marketplace.yaml --output ./dist/ --silent

Output[]
0

# Example Failure on silent flag (does not output info logs but does not fail silently)
ghoast ingest --input ./upstream/marketplaceOas.yaml --config=./transformations/marketplace.yaml --output ./dist/ --silent

Output[]
Transformation Failed: (transformation.name} operation {delete} not supported
Exit code 1

# Example Success not silent
ghoast ingest --input ./upstream/marketplaceOas.yaml --config=./transformations/marketplace.yaml

Output[]
Transformation {transformation.name} succeeded performing {preplace} on document
Transformation {transformation.name} succeeded performing {vadd} on document
Success! New Output documented located at {output path}

# Example Failure not silent
ghoast ingest --input ./upstream/marketplaceOas.yaml --config=./transformations/marketplace.yaml

Output[]
Transformation Failed: (transformation.name} operation {delete} not supported
Exit code 1

Command options

To see all command options run ghoast --help

ingest perform cli ingestor tools main transformation on upstream oas document

ingest [options] <spec-path> <config-path>

Arguments:

• <spec-path> : Path to the specfile we want to run transformations on.

• <config-path>: Path to folder containing configurations.

Options:

• --dry: Performs a dry run of the ingest command, showing the resulting specfile changes. (default: false)

• --token: Used if specifying a remote repository path. (default: "")

• --output <output-path>: Path to dump the transformed specfile. (default: "./outputs/")

• -h, --help: Displays help for this command.

Non Zero Exit Codes

As this will likely be integrated into CI pipelines, the tool will raise non zero in a number of cases when the command fails. If the command succeeds, a code 0 will be provided. The non zero exit code circumstances are provided below

The application cli tool will output a non zero exit code when:

• The operation provided within the transformation config is not supported

• The command ghoast transform is run and document is not found provided to the -i argument

• The json path provided within the transformation config evaluates to nothing (in some cases)

• Any uncaught exceptions when running the main ghoast transform command

|CODE|Description| |----|-----------| | 0 | successful| | 1 | general exception occured| | 9 | invalid input provided: command args, jsonPath, or FunctionOptions

Ingestor Transform Configuration File

YAML configuration file for targeting generated OAS documents, and transforming them in a declarative manner to suit our needs.

Top Level File Structure:

target:
  transformations: ...<transfrom options>...

The configuration yaml file requires two top level keys to be defined. The source is used along with the location options to find the source project that the OAS document is located in. This will be used to load the document prior to performing desired transformations.

The target key allows developers to write their desired transformation configurations that will be applied to the source OAS document.

Defining Transformations:

A transformation is comprised of three parts. The name and description for the transformation, and a jsonPath query. Finally you must provide an operation configuration that will be performed on the resulting jsonPath query results.

- name: <descriptive_name>
    description: <describe_operation>
    given: <jsonPath_query>
    then:
      ...<operation configuration>...

example usage:

- name: Update Frosting API Paths
    description: Replace upstream API paths with Gateway specific paths.
    given: $.paths[*]
    then:
      ...<operation configuration>...

This would apply the operation configuration to all the results returned by the jsonPath query. Note that we also gave both a name and description that defines what our intent was behind this transformation.

Defining Operation Configuration:

The operation configuration is how we perform work on the resulting jsonPath query. Each transformation can only have one associated operation, and it must be a child of the then key . Below is a simple example of how to define an operation.

Note: The keys under functionOptions are dependent on the type of operation you are trying to perform.

...<transformation definition>...
then:
  function: preplace
  functionOptions:
  field: '@key'
  pattern: '^(/v1)(/leaflink)'
  replace: '/json/marketplace'

The function key defines what operation we are going to run by name. functionOptions defines the required options of this function these vary by the type of function you are trying to use.

So putting it all together we end up with a single transform configuration:

target:
  transformations:
    - name: Update Frosting API Paths
      description: Replace upstream API paths with Gateway specific paths.
      given: $.paths[*]
      then:
        function: preplace
        functionOptions:
        field: '@key'
        pattern: '^(/v1)(/leaflink)'
        replace: '/json/marketplace'

This will query the OAS document for any elements that match the query $.paths[*]. On those resulting elements we will use the regex pattern to replace all key values with ‘/json/marketplace’

Transformation Functions:

bremove

Remove a key and its value entirely when the JSONPath is matched.

function: bremove
functionOptions: {}

FunctionOptions:

This function requires no options. However, an empty object must be passed for validation purposes.

example usage:

...
target:
  transformations:
    - name: Remove Private APIs
      description: Remove APIs which are tagged "private".
      given: $.paths[*][?(@.tags.indexOf('private') != -1)]
      then:
        function: bremove
        functionOptions: {}

preplace

Regex replace on either a key or value on the resulting elements matched from the jsonPath query.

function: preplace
functionOptions:
  field: '@key' | '@value'
  pattern: string
  replace: string

FunctionOptions:

• field (required): Specify what value to target on the element, either ‘@key’ or ‘@value’

• pattern (required): valid regex string used for matching.

• replace (required): The string value to use when regex replacing on the field.

example usage:

function: preplace
functionOptions:
  field: '@key'
  pattern: '^(/v1)(/leaflink)'
  replace: '/json/marketplace'

vadd

For Objects:

Adds a given object to the desired path location

Note: Both the target, and value must be valid JSON objects.

function: vadd
functionOptions:
  value:
  ...<keys / values to add>...

FunctionOptions:

• value (required): A combination of yaml keys / values to add

example usage:

function: vadd
functionOptions:
  value:
    security:
      - bearerAuth: []

For Arrays:

Appends to a given array at the desired path location

Note: Both the target and value must be Arrays. Be sure that your jsonPath resolves to an Array, and not an object containing the array

example usage:

function: vadd
functionOptions:
  value:
    - name: someObjName
      value: someObjValue
    - name: anotherObjName
      value: anotherObjValue

vreplace

Replaces values at the given location, the jsonPath given, must return a valid path.

Note: The target value will be fully overwritten with the supplied value.

function: vreplace
functionOptions:
  value:
  ...<Object | Value>

FunctionOptions:

• value (required): The value that will replace the target of the jsonPath query.

example usage:

function: vreplace
functionOptions:
  value:
    - url: https://api.leaflink.com
      description: LeafLink API production URL.
    - url: https://staging-api.leaflink.com
      description: LeafLink API staging URL.

Replacing the values of a targeted JSON array with new values.

function: vreplace
functionOptions:
  value:
    bearerAuth:
      type: http
      scheme: bearer
      bearerFormat: JWT

Replacing the JSON bearerAuth object value at the given path.

function: vreplace
functionOptions:
  value: 'some string value'

You can also replace simple values such as ints, strings or arrays.

Semantic Versioning Release

The number of times we've had to a manual release is: 5