Nats Core + Nats Jetstream introduction.

In cloudberry we are using "nats streaming" which will not be maintained after 2023.

The Nats team has been working on a new streaming implementation since before cloudberry started.

Jetstream is more integrated into the Core Nats publish/subscribe/request solution with additional features compared to "nats streaming".

Core Nats supports.

Supports "at-most-once" delivery, which is the same as TCP reliability.

• publish data using subjects (wildcard supported). If there are no subscribers, the event doesn't reach anyone. Many subscribers might listen using subject wildcarding.

• subscribing to events (but if the code crashes while processing the event, there is no way to get the event re-sent by Nats). The subscriber only receives events that are sent in the "future".

-request/reply. This is similar to http request/response but with the difference that many can answer the request but only one response is given back to requester.

Nats jetstream.

Supports "at-least-once" and "exactly-once" (within a time window) delivery.

• publish events into a stream. A stream is storage. It can be inmemory or file and have various limits. Streams must be created upfront, but the configuration can be updated.

• Each stream listens for events, based on it's subject configuration, and persists data into the stream.

• Subscribing to a subject (wildcard support) receives data from the stream. It can be from the beginning, only last, only new etc.

• A subscriber can either use "server push "or "client pull" to get data.

• Subscribers can work in a group, spreading out the events onto them "separated".

• Subscribers can be durable, meaning that Nats keep track of where in the stream you were.

• There is a built in "prevent duplication" feature, though it's has an active window period, so not bulletproof.

• There is support for "exactly once" delivery.

24hr 'nats-core' folder.

The folder name 'nats-core' contains a thin TypeScript layer around the nats client. The layer groups 'commonly used' functionality into two groups called 'nats' and 'jet'.

Nats group.

This group exposes non-persisten functionality like publish, subscribe and request.

Jet group.

This groups exposes persistent functionality like publish, subscribe using pull, subscribe using push.

24hr NatsWrapper.

This layer, uses the 'nats-core' jet functionality. It provides a more strict usage of nats jetstreaming along with extra failsafe features.

Stricter usage.

All publishing and subscribing requires strictly typed structure. The base of the structure that is allowed to be sent and received is described by the BaseEvent.

As an extension to the BaseEvent, there is a ReplicaEvent structure as well, that must be used when replication of database tables are required.

Failsafe features.

When publishing, the 'PersistorPublisher' should be used. It handles the case if nats is down by locally storing data and sending it later from a cron-job.

When subscribing, there are two options. Either 'standard' subscription or 'replica'. When replica subscription is used, it will handle events out of order, automatic write to database destination column. It supports upsert and delete of entries in the destination table.

The NatsWrapper support connecting multiple times. The listeners, both 'normal' and 'replica' are kept when connection again.

NatsWrapper feature list.

• connect
  • Can be called multiple times.
  • Stateful about subscription pulls.
• disconnect
  • Can only be called once.
• subscribe
  • A 'normal' subscribe (not replica subscribe)
  • User must handle msg.ack or msg.nak
• replicaSubscribe
  • Replication of data between services.
  • It handles msg.ack and msg.nak.
  • Requires mapToColumns callback.
  • Optional callbacks exist for 'hooking' into the replication.
• publish
  • A 'normal' publish.
• getPersistorPublisher
  • Publish using 'persistor'
  • It's a two step process
    • First, within the users transaction, persist event locally in database.
    • Secondly, after transaction has been commited, it publishes.
    • Third, if 2nd fails, cron-job will send later.
• trackServerStatus
  • Users can register for nats status updates.

Development.

Bring up.

make create_resources make port

Get nats logs

kubectl logs nats-0 -f

Run all tests.

npm run test

npm run test:ci

Run single test file once.

Requires that jest is globally installed.

jest --no-cache --runInBand --testTimeout 120000 src/replica/test/replica.test.ts

Run single test file with watch.

Requires that jest is globally installed.

jest --no-cache --runInBand --testTimeout 120000 --watch src/replica/test/replica.test.ts