npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

fs-capacitor-cjs

v8.0.0

Published

Filesystem-buffered, passthrough stream that buffers indefinitely rather than propagate backpressure from downstream consumers.

Downloads

58

Readme

Continuous Integration Current Version Supported Node.js Versions

FS Capacitor

FS Capacitor is a filesystem buffer for finite node streams. It supports simultaneous read/write, and can be used to create multiple independent readable streams, each starting at the beginning of the buffer.

This is useful for file uploads and other situations where you want to avoid delays to the source stream, but have slow downstream transformations to apply:

import fs from "fs";
import http from "http";
import { WriteStream } from "fs-capacitor";

http.createServer((req, res) => {
  const capacitor = new WriteStream();
  const destination = fs.createWriteStream("destination.txt");

  // pipe data to the capacitor
  req.pipe(capacitor);

  // read data from the capacitor
  capacitor
    .createReadStream()
    .pipe(/* some slow Transform streams here */)
    .pipe(destination);

  // read data from the very beginning
  setTimeout(() => {
    capacitor.createReadStream().pipe(/* elsewhere */);

    // you can destroy a capacitor as soon as no more read streams are needed
    // without worrying if existing streams are fully consumed
    capacitor.destroy();
  }, 100);
});

It is especially important to use cases like graphql-upload where server code may need to stash earler parts of a stream until later parts have been processed, and needs to attach multiple consumers at different times.

FS Capacitor creates its temporary files in the directory ideneified by os.tmpdir() and attempts to remove them:

  • after writeStream.destroy() has been called and all read streams are fully consumed or destroyed
  • before the process exits

Please do note that FS Capacitor does NOT release disk space as data is consumed, and therefore is not suitable for use with infinite streams or those larger than the filesystem.

Ensuring cleanup on termination by process signal

FS Capacitor cleans up all of its temporary files before the process exits, by listening to the node process's exit event. This event, however, is only emitted when the process is about to exit as a result of either:

  • The process.exit() method being called explicitly;
  • The Node.js event loop no longer having any additional work to perform.

When the node process receives a SIGINT, SIGTERM, or SIGHUP signal and there is no handler, it will exit without emitting the exit event.

Beginning in version 3, fs-capacitor will NOT listen for these signals. Instead, the application should handle these signals according to its own logic and call process.exit() when it is ready to exit. This allows the application to implement its own graceful shutdown procedures, such as waiting for a stream to finish.

The following can be added to the application to ensure resources are cleaned up before a signal-induced exit:

function shutdown() {
  // Any sync or async graceful shutdown procedures can be run before exiting…
  process.exit(0);
}

process.on("SIGINT", shutdown);
process.on("SIGTERM", shutdown);
process.on("SIGHUP", shutdown);

API

WriteStream

WriteStream extends stream.Writable

new WriteStream(options: WriteStreamOptions)

Create a new WriteStream instance.

.createReadStream(options?: ReadStreamOptions): ReadStream

Create a new ReadStream instance attached to the WriteStream instance.

Calling .createReadStream() on a released WriteStream will throw a ReadAfterReleasedError error.

Calling .createReadStream() on a destroyed WriteStream will throw a ReadAfterDestroyedError error.

As soon as a ReadStream ends or is closed (such as by calling readStream.destroy()), it is detached from its WriteStream.

.release(): void

Release the WriteStream's claim on the underlying resources. Once called, destruction of underlying resources is performed as soon as all attached ReadStreams are removed.

.destroy(error?: ?Error): void

Destroy the WriteStream and all attached ReadStreams. If error is present, attached ReadStreams are destroyed with the same error.

WriteStreamOptions

.highWaterMark?: number

Uses node's default of 16384 (16kb). Optional buffer size at which the writable stream will begin returning false. See node's docs for stream.Writable. For the curious, node has a guide on backpressure in streams.

.defaultEncoding

Uses node's default of utf8. Optional default encoding to use when no encoding is specified as an argument to stream.write(). See node's docs for stream.Writable. Possible values depend on the version of node, and are defined in node's buffer implementation;

.tmpdir

Used node's os.tmpdir by default. This function returns the directory used by fs-capacitor to store file buffers, and is intended primarily for testing and debugging.

ReadStream

ReadStream extends stream.Readable;

ReadStreamOptions

.highWaterMark

Uses node's default of 16384 (16kb). Optional value to use as the readable stream's highWaterMark, specifying the number of bytes (for binary data) or characters (for strings) that will be bufferred into memory. See node's docs for stream.Readable. For the curious, node has a guide on backpressure in streams.

.encoding

Uses node's default of utf8. Optional encoding to use when the stream's output is desired as a string. See node's docs for stream.Readable. Possible values depend on the version of node, and are defined in node's buffer implementation.