Queue in Background (qbg)

A versatile JavaScript/TypeScript library for managing server actions in sequence, especially when actions are inter-dependent. This queue-based system allows you to queue actions with specific payloads and ensures that they are executed in a controlled manner. Each action can transform its payload just before execution, making it a flexible tool for complex workflows where actions depend on responses from prior actions.

Features

• Sequential Execution: Ensures that server actions are executed one after another, in the order they are enqueued.
• Just-in-time Transformation: Ability to transform payloads just before execution based on specific logic.
• Error Handling: Customizable error handling that allows failed actions to be retried, added to a dead-letter queue, or handled as per your needs.
• Persistent Queue: Store and retrieve queued actions using your own persistence layer.
• Thread Safety: All operations are thread-safe and protected against race conditions, allowing concurrent access from multiple parts of your application.
• Flexible Connectivity Check: Pass your own function to check network connectivity and manage retries.
• Minimal Dependencies: This package is free from platform-specific dependencies and can work in any JavaScript environment.

Installation

You can install the package via npm or yarn: npm install qbg or yarn add qbg

Quick Start

To get started quickly, follow this example:

import {init, Action} from 'qbg';

// Initialize the queue with basic hooks
const hooksRegistry = {
	SIMPLE_ACTION: async (payload) => {
		console.log("Action executed with payload:", payload);
	},
};

const queue = await init({hooksRegistry});

// Enqueue an action
const action: Action = {
	type: 'SIMPLE_ACTION',
	payload: {key: 'value'},
};

await queue.enqueue(action);

Usage

1. Initialization

   The package is initialized in your application. You must provide:

   • A registry of hooks that define how actions are processed.
   • A transformer registry for just-in-time payload transformation.
   • A persistence object for storing and retrieving queue actions.
   • Optional network connectivity and error processing functions to manage retries and failures.

   import {
     init,
     getQueue,
     Action,
     Persistence,
   } from 'qbg ';
   
   // Define how each action type should be handled
   const hooksRegistry = {
     ACTION_TYPE_1: async (payload) => {
       // Logic to execute for this action
     },
     ACTION_TYPE_2: async (payload) => {
       // Logic for another action type
     },
   };
   
   // Define payload transformers for just-in-time transformations
   const transformerRegistry = {
     ACTION_TYPE_1: (payload) => {
       return { ...payload, transformedKey: 'transformedValue' };
     },
   };
   
   // Implement persistence methods
   const persistence: Persistence = {
     saveQueue: async (queue) => {
       // Save the current state of the queue
     },
     saveDLQueue: async (dlQueue) => {
       // Save the dead-letter queue
     },
     readQueue: async () => {
       // Read and return the queue from storage
       return [];
     },
     readDLQueue: async () => {
       // Read and return the dead-letter queue from storage
       return [];
     },
   };
   
   
   // Initialize the queue with registries and persistence layer
   const queue = await init({
    hooksRegistry, 
    transformerRegistry, 
    persistence, 
   });
   
   // Now you can also access the queue instance via getQueue()

2. Enqueue Actions

   You can add actions to the queue using the enqueue() method. Each action should have a type and a payload. These actions will be processed in sequence, and the payloads can be transformed just before execution.

   const action: Action = {
     type: 'ACTION_TYPE_1',
     payload: { someKey: 'someValue' },
   };
   
   // Enqueue the action
   await queue.enqueue(action);

3. Connectivity State Changes

   If your application is reliant on network status, you can trigger queue boots on state changes by invoking the listen method.

   // Check network connectivity in react native apps. Same can be done for web apps using navigator.onLine
   import NetInfo from '@react-native-community/netinfo';
   
   NetInfo.addEventListener((state) => {
   if (state.isConnected && !this.networkStatus) {
     queue.listen();
   }
   });
   

4. Error Handling You can provide custom error-handling logic by passing a function that decides how errors should be processed. For example, retry failed actions, move them to a dead-letter queue, or handle them as per your use case.

   const errorProcessor = (error, action) => {
     if (error instanceof SomeKnownError) {
       // Retry or handle action
       return true; // Return true to retry
     }
     return false; // Move to dead-letter queue or discard
   };
   
   // Initialize with error handling logic
   const queue = await init({
     hooksRegistry,
     transformerRegistry,
     persistence,
     errorProcessor
   });

5. Accessing the Queue Once the queue is initialized, you can access it using getQueue() and interact with it.

   const queue = getQueue();
   console.log(queue.size); // Get the current size of the queue

6. Persistence The Persistence interface defines methods to save and read the queue and dead-letter queue. You need to implement this based on your app's storage requirements (e.g., local storage, database).

   export type Persistence = {
     saveQueue: (actions: Action[]) => Promise<void>;
     saveDLQueue: (actions: Action[]) => Promise<void>;
     readQueue: () => Promise<Action[]>;
     readDLQueue: () => Promise<Action[]>;
   };

7. Dead-Letter Queue (DLQ) If an action fails multiple times (depending on your error-handling logic), it will be moved to the dead-letter queue (DLQ). You can access the DLQ and take appropriate actions (e.g., logging, manual retries, etc.).

     console.log('Failed actions in DLQ:', queue.peekDLQ)

API Reference

init

Initializes the PatchyInternetQImpl instance.

• Parameters:

  • props (InitProps): An object containing:
    • hooksRegistry: A registry for action hooks.
    • transformerRegistry: A registry for transformers.
    • persistence: An instance of the Persistence interface.
    • errorProcessor: A function to process errors.

• Returns:

  • Promise<PatchyInternetQImpl>: A promise that resolves to the initialized PatchyInternetQImpl instance.

getQueue

Retrieves the singleton instance of the PatchyInternetQImpl.

• Returns:
  • PatchyInternetQImpl | undefined: The current instance of the queue or undefined if not initialized.

enqueue(action: Action): Promise<void>

• Adds an action to the queue and saves the queue to persistence.

clearDLQueue(): Promise<Action[]>

• Clears the dead-letter queue and returns its previous items.

listen(): Promise<void>

• Starts listening for and processing actions in the queue.

Getters

• ready:

  • Returns a promise that resolves when the queue is ready.

• size:

  • Returns the size of the main action queue.

• peek:

  • Returns the actions to be processed without removing it.

• dlQueueSize:

  • Returns the size of the dead-letter queue.

• peekDLQ:

  • Returns the actions in the dead-letter queue without removing it.

Example Usage

Please find example usage in the example.md file.

Contributing

Contributions are welcome! Please feel free to submit a pull request or open an issue if you encounter any problems.

License

This library is licensed under the MIT License.