npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

scarlet-task

v3.0.2

Published

An advanced task queue module for Node.js with support for multiple child queues, enabling controlled concurrent processing of asynchronous tasks.

Downloads

76

Readme

Scarlet Task is an advanced task queue module for Node.js, featuring the ability to configure multiple child queues for a single task queue.

Why named Scarlet? ๛ก(ー̀ωー́ก)

The story behind this module's name is quite serendipitous. One day, while browsing 萌否收音機, I stumbled upon a captivating song. Though its title slipped my mind, I distinctly remembered its considerable length.

Driven by a desire to rediscover this elusive track, I developed this queue module. It allowed me to perform an exhaustive search across the entire Moe FM platform. After a thorough exploration, I finally unearthed the song that had been haunting my memory: <the Embodiment of Scarlet Devil>.

To commemorate this musical treasure hunt and pay homage to my favorite character, Flandre Scarlet, I christened this module 'Scarlet Task'. It's a nod to both the journey of rediscovery and the Scarlet-themed song that sparked it all.

Use Case

Scarlet Task is particularly useful for scenarios requiring controlled asynchronous operations, such as web crawling. By allowing you to set up multiple child queues, it enables concurrent processing while maintaining control over parallelism, preventing issues like unintentional DDoS-like behavior on target websites.

Installation

$ npm install --save scarlet-task

If you're using it in browser, you may need to install events as well.

$ npm install --save events

Tutorials

First, require the module and instantiate a Scarlet object:

const { Scarlet } = require("scarlet-task");
const scarlet = new Scarlet(10);

The parameter for the constructor represents the number of child queues. If no parameter is passed, it defaults to 1 child queue.

Define a processor function to handle tasks. You can also pass an anonymous function:

function processor(taskObject) {
  // Get the task object
  const task = taskObject.task;
  
  // Perform task logic...
  // ...
  
  taskObject.done(); // Or call `scarlet.taskDone(taskObject);`

  console.log(scarlet.numberOfProcessed());
}

Note: In the processor function, you should call taskObject.done() or scarlet.taskDone(taskObject) when you consider the task complete. Then scarlet will process the next task. The taskObject parameter is passed to you by the scarlet instance.

You can push tasks into the queue at any time. The task object can be of any type - string, number, JSON, etc.:

const task = "This could be a URL, or an object that the processor can handle";
scarlet.push(task, processor);

For more reference, see test/hackernews.test.ts.

You can reset the number of processed tasks:

scarlet.resetNumberOfProcessed();

You can also set an after-finish callback function that Scarlet will call after a specified number of tasks are completed:

scarlet.afterFinish(20, done, false);
// This will call done() after 20 tasks are completed, without looping (meaning it executes only once unless you reset the processed count)

scarlet.clearAfterFinish();
// You can clear the afterFinish processor

For more reference, see test/hackernews.test.ts.

APIs

Scarlet

Constructor

const scarlet = new Scarlet(queueCount);
  • queueCount (optional): Number of child queues. Defaults to 1 if not specified.

Methods

  • push(task, processor): Adds a new task to the queue.

    • task: The task to be processed (can be any type).
    • processor: Function to process the task.
  • numberOfProcessed(): Returns the number of processed tasks.

  • resetNumberOfProcessed(): Resets the count of processed tasks to 0.

  • taskDone(taskObject): Marks a task as completed.

    • taskObject: The task object to be marked as done.
  • afterFinish(count, processor, loop): Sets a callback to be executed after a certain number of tasks are completed.

    • count: Number of tasks to complete before calling the processor.
    • processor: Function to be called after count tasks are completed.
    • loop (optional): If true, the processor will be called repeatedly every count tasks. Defaults to false.
  • clearAfterFinish(): Clears the after-finish processor.

TaskObject

Properties

  • task: The task data.
  • queueId: The ID of the queue processing this task.

Methods

  • done(): Marks the task as completed.

Contribute

You're welcome to make pull requests!

「雖然我覺得不怎麼可能有人會關注我」