worker-threads-manager
v1.0.2
Published
A TypeScript library for managing worker threads with serialization.
Downloads
35
Maintainers
Readme
Worker Threads Manager
Worker Threads Manager is a TypeScript library designed to simplify the management of worker threads in Node.js, enabling asynchronous method invocation between threads with strong typing, serialization/deserialization of data, and robust error handling. This library also supports the registration and use of custom classes, including those with recursively nested data.
Features
- Strong Typing: The library enforces strict typing across worker threads, ensuring type safety and reducing runtime errors.
- Serialization and Deserialization: Complex data structures, including custom classes and recursively nested objects, are automatically serialized before being sent to worker threads and deserialized upon receipt.
- Error Handling: Includes built-in mechanisms for timeout management and error handling in asynchronous method invocations.
- Class Registration: Allows for the registration of custom classes so that they can be properly serialized and deserialized during inter-thread communication.
Installation
npm install worker-threads-manager
Basic Usage
1. Define Handlers and Worker Controller
Handlers are defined in the worker thread. These handlers are functions that can be called from the main thread.
import { WorkerController } from "../src";
import { Handlers } from "../src/interfaces";
export class Matrix {
a: number;
constructor(x: number) {
this.a = x;
}
addMatrix(m: Matrix) {
this.a += m.a;
return this;
}
}
export class WorkerHandlers implements Handlers {
processData(data: string): string {
return `Processed: ${data}`;
}
calculateSum(a: number, b: number): number {
return a + b;
}
matrixOperations(m1: Matrix, m2: Matrix) {
return m1.addMatrix(m2);
}
}
WorkerController.initialize(new WorkerHandlers());
WorkerController.registerClasses([Matrix]);
2. Use the Worker Manager in the Main Thread
The WorkerManager
is used in the main thread to create worker threads, invoke methods in the worker, and manage their lifecycle.
import { WorkerManager } from "../src";
import { Matrix, WorkerHandlers } from "./worker";
import { resolve } from "path";
(async () => {
let manager = new WorkerManager<WorkerHandlers>();
// Create the first worker and invoke a method
let workerId1 = await manager.createWorkerWithHandlers(resolve(__dirname, 'worker.js'));
let res1 = await manager.call(workerId1, 'processData', '123');
// Create a second worker and invoke a different method
let workerId2 = await manager.createWorkerWithHandlers(resolve(__dirname, 'worker.js'));
let res2 = await manager.call(workerId2, 'calculateSum', 1, 2);
// Register classes for serialization
manager.registerClasses([Matrix]);
// Perform matrix operations involving custom classes
let m = await manager.call(workerId1, 'matrixOperations', new Matrix(2), new Matrix(3));
console.log(res1, res2, m);
// Terminate the workers
await manager.terminateWorker(workerId1);
await manager.terminateWorker(workerId2);
})().then(() => console.log('finish'));
Working with Custom Classes
Defining and Registering Classes
You can define custom classes and register them with the WorkerController
and WorkerManager
. This ensures that instances of these classes are correctly serialized and deserialized when sent between threads.
Example with Recursively Nested Data
export class TreeNode {
value: number;
left: TreeNode | null = null;
right: TreeNode | null = null;
constructor(value: number) {
this.value = value;
}
addLeft(child: TreeNode) {
this.left = child;
}
addRight(child: TreeNode) {
this.right = child;
}
calculateSum(): number {
let sum = this.value;
if (this.left) sum += this.left.calculateSum();
if (this.right) sum += this.right.calculateSum();
return sum;
}
}
export class TreeWorkerHandlers implements Handlers {
calculateTreeSum(root: TreeNode): number {
return root.calculateSum();
}
}
WorkerController.initialize(new TreeWorkerHandlers());
WorkerController.registerClasses([TreeNode]);
In the main thread:
import { WorkerManager } from "../src";
import { TreeNode, TreeWorkerHandlers } from "./treeWorker";
import { resolve } from "path";
(async () => {
let manager = new WorkerManager<TreeWorkerHandlers>();
// Register the TreeNode class
manager.registerClasses([TreeNode]);
// Create a tree structure
let root = new TreeNode(10);
root.addLeft(new TreeNode(5));
root.addRight(new TreeNode(15));
root.left?.addLeft(new TreeNode(3));
root.left?.addRight(new TreeNode(7));
// Create a worker and calculate the sum of the tree
let workerId = await manager.createWorkerWithHandlers(resolve(__dirname, 'treeWorker.js'));
let treeSum = await manager.call(workerId, 'calculateTreeSum', root);
console.log(`Tree sum: ${treeSum}`); // Expected output: 40
await manager.terminateWorker(workerId);
})().then(() => console.log('finish'));
Recursive Structures
The SerDe
class in the library can handle recursive structures. When serializing, it keeps track of the objects it has visited to avoid infinite loops and correctly reconstructs the object graph upon deserialization.
Key Principles
Serialization and Deserialization: Custom classes and complex data structures are serialized before being passed to the worker and deserialized on the worker side. The result is similarly serialized in the worker and deserialized in the main thread.
Class Registration: Before any class instances can be passed between threads, they must be registered using
WorkerManager.registerClasses()
andWorkerController.registerClasses()
.Thread Management: The library provides straightforward methods for creating, calling, and terminating worker threads, encapsulating the complexity of inter-thread communication.
Error Handling
Timeouts: By default, there is a timeout for worker responses, which can be customized or disabled by adjusting the
WorkerManager
constructor's timeout parameter.Exception Propagation: If an error occurs in the worker thread, it is captured, serialized, and passed back to the main thread, where it can be handled appropriately.
Conclusion
The Worker Threads Manager library provides an efficient and type-safe way to manage worker threads in a Node.js application, with robust support for complex data structures and custom classes. It simplifies the interaction between threads, making it easy to offload heavy computations or parallelize tasks in your applications.
This README.md
provides a solid foundation for users of your library, explaining how to get started and showcasing the library's features, including handling complex objects and recursive data structures.