@scramjet/api-client

v1.0.1

Published

9 months ago

This package is part of Scramjet Transform Hub. The package provides the API Client for use with Scramjet Transform Hub.

Downloads

0High
0Medium
0Low

The package provides an API Client for use with Scramjet Transform Hub.

Usage:

import { HostClient, SequenceClient } from "@scramjet/api-client";

const host = new HostClient("http://192.168.0.34:8000/api/v1");

host.getSequence("3e989fe7-a3b7-401e-a269-2fa0a7ac70f4")

Docs

See the code documentation here: scramjetorg/transform-hub/docs/api-client/modules.md

Scramjet Transform Hub

This package is part of Scramjet Transform Hub.

Scramjet Transform Hub is a deployment and execution platform. Once installed on a server, it will allow you to start your programs and keep them running on a remote machine. You will be able to start programs in the background or connect to them and see their output directly on your terminal. You will be able to pipe your local data to the program, as if it was running from your terminal. You can start your server in AWS, Google Cloud or Azure, start it on your local machine, install it on a Raspberry Pi or wherever else you'd like.

Use cases

There's no limit what you can use it for. You want a stock checker? A chat bot? Maybe you'd like to automate your home? Retrieve sensor data? Maybe you have a lot of data and want to transfer and wrangle it? You have a database of cities and you'd like to enrich your data? You do machine learning and you want to train your set while the data is fetched in real time? Hey, you want to use it for something else and ask us if that's a good use? Ask us via email or hop on our Scramjet Discord!

Quick guide

Host operations

Parameters

No parameters

Responses

Successful operation code: 200

{ "version" : "0.12.2" }

Parameters

No parameters

Responses

Successful operation code: 200

[
  {
    "instances": [], // a list of all running Instances of this Sequence
    "id": "eea8bc33-440f-4a17-8931-eb22a17d5d56", // Sequence ID
    "config": {
      "container": {
        "image": "scramjetorg/runner:0.12.2",
        "maxMem": 512,
        "exposePortsRange": [
          30000,
          32767
        ],
        "hostIp": "0.0.0.0"
      },
      "name": "@scramjet/hello-alice-out",
      "version": "0.12.2",
      "engines": {
        "node": ">=10"
      },
      "config": {},
      "sequencePath": "index", // a path to file with a main function
      "packageVolumeId": "eea8bc33-440f-4a17-8931-eb22a17d5d56"
    }
  },
  {
    "instances": [
      "02381acf-cb16-4cff-aa9b-f22f04ada94f"
    ],
    "id": "3ec02b93-4ca9-4d23-baab-048dab5ffda4",
    "config": {
      "container": {
        "image": "scramjetorg/runner:0.12.2",
        "maxMem": 512,
        "exposePortsRange": [
          30000,
          32767
        ],
        "hostIp": "0.0.0.0"
      },
      "name": "@scramjet/checksum-sequence",
      "version": "0.12.2",
      "engines": {},
      "config": {},
      "sequencePath": "index.js",
      "packageVolumeId": "3ec02b93-4ca9-4d23-baab-048dab5ffda4"
    }
  }
]

Parameters

No parameters

Responses

Successful operation code: 200

[
  {
    "id": "02381acf-cb16-4cff-aa9b-f22f04ada94f", // Instance ID
    "sequence": "3ec02b93-4ca9-4d23-baab-048dab5ffda4" // Sequence ID
  },
  {
    "id": "ab0272d8-c9b0-43f7-9e7e-bcac9ec0f21f",
    "sequence": "e4ca555c-ced1-4a13-b531-f43016eaf4ed"
  }
]

Parameters

No parameters

Responses

Successful operation code: 200

{
  "avgLoad": 0.08,
  "currentLoad": 5.190776257704936,
  "memFree": 4634816512,
  "memUsed": 8050364416,
  "fsSize": [
    {
      "fs": "/dev/sda1",
      "type": "ext4",
      "size": 20838993920,
      "used": 14939455488,
      "available": 5882761216,
      "use": 71.75,
      "mount": "/"
    },
    {
      "fs": "/dev/sda15",
      "type": "vfat",
      "size": 109422592,
      "used": 9621504,
      "available": 99801088,
      "use": 8.79,
      "mount": "/boot/efi"
    }
  ]
}

Parameters

No parameters

Responses

Content-type: application/octet-stream

Successful operation code: 200

2021-11-19T16:04:47.094Z log (object:Host) Host main called.
2021-11-19T16:04:47.100Z info (object:SocketServer) Server on: /tmp/scramjet-socket-server-path
2021-11-19T16:04:47.104Z info (object:Host) API listening on: 127.0.0.1:8000
2021-11-19T16:05:08.228Z info (object:Host) New sequence incoming...
2021-11-19T16:05:08.229Z log (object:LifecycleDockerAdapterSequence) Docker sequence adapter init.
2021-11-19T16:05:08.229Z log (object:DockerodeDockerHelper) Checking image scramjetorg/pre-runner:0.12.2
2021-11-19T16:05:12.234Z info (object:LifecycleDockerAdapterSequence) Docker sequence adapter done.
2021-11-19T16:05:12.246Z log (object:LifecycleDockerAdapterSequence) Volume created. Id:  c50fe4d3-89cc-4685-a82a-16cbc744733d
2021-11-19T16:05:12.246Z log (object:LifecycleDockerAdapterSequence) Starting PreRunner { image: 'scramjetorg/pre-runner:0.12.2', maxMem: 128 }
2021-11-19T16:05:13.536Z log (object:DockerodeDockerHelper) Checking image scramjetorg/runner:0.12.2
2021-11-19T16:05:16.670Z info (object:SequenceStore) New sequence added: c50fe4d3-89cc-4685-a82a-16cbc744733d
2021-11-19T16:05:16.672Z info (object:Host) Sequence identified: {
  container: {
    image: 'scramjetorg/runner:0.12.2',
    maxMem: 512,
    exposePortsRange: [ 30000, 32767 ],
    hostIp: '0.0.0.0'
  },
  name: '@scramjet/multi-outputs',
  version: '0.12.2',
  engines: {},
  config: {},
  sequencePath: 'index.js',
  packageVolumeId: 'c50fe4d3-89cc-4685-a82a-16cbc744733d'
}
2021-11-19T16:05:16.691Z debug (object:Host) Request date: 2021-11-19T16:05:08.239Z, method: POST, url: /api/v1/sequence, status: 202

Sequence operations

Parameters

| Name | Type | Description | Required | | ----------- | -------- | ----------------------------------- | -------- | | file | binary | compressed package in tar.gz format | yes | | appConfig | json | additional package.json config file | no |

Responses

Accepted operation code: 202

{
  "id": "2c3068e5-7c74-45bb-a017-1979c41fc6d0" // sequence id
}

Parameters

No parameters

Responses

Successful operation code: 200

[
  {
    "instances": [
      "742d2713-7ab6-4cde-82f3-a7beabdd4e98"       // list of sequence instances
    ],
    "id": "bdef63db-d3a0-45c8-85db-e94ebb96097f",  // sequence id
    "config": {
      "container": {
        "image": "scramjetorg/runner:0.12.2",
        "maxMem": 512
      },
      "name": "@scramjet/transform-hub",
      "version": "0.12.2",
      "engines": {},
      "config": {},
      "sequencePath": "index.js",
      "packageVolumeId": "bdef63db-d3a0-45c8-85db-e94ebb96097f"
    }
  }
]

Parameters

| Name | Type | Description | Required | | ----------- | ------ | ----------------------------------------------------- | -------- | | appConfig | json | additional package.json config file | no | | args | json | additional arguments that instance should starts with | no |

Responses

Successful operation code: 200

{
  "id": "681c856e-dfa4-46a1-951d-47b27345552e"
}

Parameters

No parameters

Responses

Successful operation code: 200

{
  "id": "2c3068e5-7c74-45bb-a017-1979c41fc6d0"
}

Conflict operation code: 409 - the instance is still running

{
  "error": "Can't remove sequence in use."
}

Instance basic operations

Parameters

No parameters

Responses

Successful operation code: 200

[
  {
    "id": "742d2713-7ab6-4cde-82f3-a7beabdd4e98",
    "sequence": "bdef63db-d3a0-45c8-85db-e94ebb96097f"
  },
  {
    "id": "681c856e-dfa4-46a1-951d-47b27345552e",
    "sequence": "bdef63db-d3a0-45c8-85db-e94ebb96097f"
  },
  {
    "id": "21f787ed-6b9e-4e9f-828e-afe428d84833",
    "sequence": "bdef63db-d3a0-45c8-85db-e94ebb96097f"
  }
]

Parameters

No parameters

Responses

| Name | Code | Description | | :------------------- | :---- | :------------------------------------------ | | Successful operation | 200 | Returns JSON data | | Not Found operation | 404 | For example if instance was already stopped |

{
  "created": "2021-10-29T16:08:36.524Z",
  "started": "2021-10-29T16:08:38.701Z",
  "sequenceId": "b0c02fdc-b05f-4f26-9d68-43a702eb7b44"
}

Parameters

| Name | Type | Description | Required | | ------------------ | --------- | ------------------------------------------------------------------------------- | -------- | | timeout | number | The number of milliseconds before the Instance will be killed. Default: 7000ms. | no | | canCallKeepalive | boolean | If set to true, the instance will prolong the running. Default: false. | no |

Responses

Successful operation code: 200

{
   "code": 0,
   "type": "string",
   "message": "string"
}

Parameters

No parameters

Responses

Accepted operation code: 202

No body returned

Parameters

No parameters

Responses

Successful operation code: 200

{
  "cpuTotalUsage": 529325247,
  "healthy": true,
  "limit": 536870912,
  "memoryMaxUsage": 16117760,
  "memoryUsage": 14155776,
  "networkRx": 1086,
  "networkTx": 0,
  "containerId": "1c993c4ff774fac06185aa9554cf40c23b03e1479a7e0d14827708161b08ae51"
}

Instance advanced operation

Parameters

| Name | Type | Description | Required | | :---------- | :------- | ---------------------------- | -------- | | eventName | string | Name of an event | true | | message | string | JSON formatted event payload | false |

Responses

Content-type: application/octet-stream

Parameters

No parameters

Responses

Content-type: application/octet-stream

Parameters

No parameters

Responses

Content-type: application/octet-stream

Parameters

No parameters

Responses

| Name | Code | Description | | :----------------------- | :---- | :----------------------------------------------------------- | | Successful operation | 200 | - | | Not Acceptable operation | 406 | Instance expects the input to be provided from the Topic API |

Parameters

No parameters

Responses

Content-type: application/octet-stream

Parameters

No parameters

Responses

| Name | Code | Description | | :----------------------- | :---- | :----------------------------------------------------------- | | Successful operation | 200 | - |

Parameters

No parameters

Responses

Successful operation code: 200

Parameters

No parameters

Responses

Content-type: application/octet-stream

Parameters

No parameters

Responses

Content-type: application/octet-stream

Parameters

No parameters

Responses

Content-type: application/octet-stream

Successful operation code: 200

2021-11-19T16:12:22.948Z log (Sequence) 42
2021-11-19T16:12:23.949Z log (Sequence) 41
2021-11-19T16:12:24.950Z log (Sequence) 40
2021-11-19T16:12:25.951Z log (Sequence) 39
2021-11-19T16:12:26.952Z log (Sequence) 38
2021-11-19T16:12:27.952Z log (Sequence) 37
2021-11-19T16:12:28.953Z log (Sequence) 36
2021-11-19T16:12:29.953Z log (Sequence) 35

Service Discovery: Topics

If a given topic does not exist, Transform-Hub creates it and stores the sent data in the newly created topic. The data is stored in the topic until the data is not consumed (either by the Topic API or by the Instances subscribing to this topic). When the data are sent to the topic they are written to the returned stream.

Parameters

No parameters

Request Headers

| Header | Type | Description | Default | Required | | -------------- | --------------------- | ----------------------------------------------------------------------- | ---------------------- | -------- | | x-end-stream | boolean | If set to true, then close topic stream after processing the request. | false | no | | content-type | text, application | Specifies data type of this topic | application/x-ndjson | no |

Supported types: text/x-ndjson, application/x-ndjson, application/x-ndjson, text/plain, application/octet-stream

Responses

| Name | Code | Description | | :------------------- | :---- | :-------------------------------------------------------------------- | | Successful operation | 200 | data was sent with the header indicating the end of data | | Successful operation | 202 | data was sent without the header indicating the end of data (default) |

Parameters

No parameters

Responses

Topic data stream.

Successful operation code: 200

{
  "source": "Twitter",
  "id": "850006245121695778",
  "content": "Natural wetlands make up ~30% of global total CH4 emissions",
  "user": {
    "id": 1234994945,
    "name": "Climate Change Conference",
    "screen_name": "Climate Change",
  }
}

Some important links

Scramjet, the company behind Transform Hub
The Scramjet Framework - functional reactive stream processing framework
The Transform Hub repo on github
You can see the Scramjet Transform Hub API docs here
You can see the CLI documentation here, but si help should also be quite effective.
Don't forget to ⭐ this repo if you like it, subscribe to releases and keep visiting us for new versions and updates.
You can open an issue - file a bug report or a feature request here

License and contributions

This module is licensed under AGPL-3.0 license.

The Scramjet Transform Hub project is dual-licensed under the AGPL-3.0 and MIT licenses. Parts of the project that are linked with your programs are MIT licensed, the rest is AGPL.

Contributions

We accept valid contributions and we will be publishing a more specific project roadmap so contributors can propose features and also help us implement them. We kindly ask you that contributed commits are Signed-Off git commit --sign-off.

We provide support for contributors via test cases. If you expect a certain type of workflow to be officially supported, please specify and implement a test case in Gherkin format in bdd directory and include it in your pull request. More info about our BDD test you will find here.

Help wanted 👩‍🎓🧑👱‍♀️

The project need's your help! There's lots of work to do and we have a lot of plans. If you want to help and be part of the Scramjet team, please reach out to us, on discord or email us: [email protected].

Donation 💸

Do you like this project? It helped you to reduce time spent on delivering your solution? You are welcome to buy us a coffee ☕ Thanks a lot! 😉

You can sponsor us on github

There's also a Paypal donation link if you prefer that:

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

Docs

More reading

Scramjet Transform Hub

Use cases

Quick guide

Host operations

Sequence operations

Instance basic operations

Instance advanced operation

Service Discovery: Topics

Some important links

License and contributions

Contributions

Help wanted 👩‍🎓🧑👱‍♀️

Donation 💸