npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@e2fyi/streams

v1.0.1

Published

Nodejs stream library for various use cases: e.g. Auto-tagging object streams, streaming to mongoDb via mongoose models, etc.

Downloads

10

Readme

@e2fyi/streams

GitHub release Build Status Coverage Status Known Vulnerabilities styled with prettier

This NodeJS library provides custom NodeJS streams for specific use cases.

Currently, the following streams are available:

  • DocumentTagger: A Transform stream to tag an auto-increment field to each object in the stream. Can also mutate the stream objects through a function or default Object.

  • MongooseStream: A Transform stream which will bulk write the objects in the stream to mongodb via a mongoose model.

ChangeLog

  • v1.0.0: new streams: DocumentTagger, MongooseStream.

Quick start

Using CommonJS module

// importing DocumentTagger and MongooseStream
const {DocumentTagger, MongooseStream} = require('@e2fyi/streams');

API reference

The API documentation is also available at https://e2fyi.github.io/streams.

@e2fyi/streams

@e2fyi/streams.DocumentTagger ⇐ stream.Transform

Transform Object stream (objectMode=true) to tag with an autoIncrement id. An object or function can be optionally provided to mutate each object in the stream.

Kind: static class of @e2fyi/streams
Extends: stream.Transform
Emits: filtered

new DocumentTagger(opts)

Create a new DocumentTagger stream.

| Param | Type | Description | | --- | --- | --- | | opts | DocumentTaggerSettings | Settings for the stream. |

Example

const docTagger = new DocumentTagger({autoIncrement: 'id', mutate: { project: 'test' }});
someReadableStreamFromArray([{text: 'abc'}, {text: 'efg'}])
  .pipe(docTagger)
  .pipe(process.stdout);
// stdout >
// {"text": "abc", "id": 0, "project": "test"}
// {"text": "efg", "id": 1, "project": "test"}

@e2fyi/streams.MongooseStream ⇐ stream.Transform

A custom NodeJS Transform stream to mongo via mongoose.

Kind: static class of @e2fyi/streams
Extends: stream.Transform
Emits: mongoose-bulk-write

new MongooseStream(opts)

Create a Transform stream which bulkWrite to mongo based on the itemWaterMark. model (mongoose Model) is a required field.

| Param | Type | Description | | --- | --- | --- | | opts | MongooseStreamSettings | Configuration for MongoStream. Default value for itemWaterMark is 50. |

Example

var stream2mongo = new MongooseStream({mode: SomeMongooseModel});
someReadableStreamFromArray([{text: 'abc'}, {text: 'efg'}])
  .pipe(stream2mongo) // writes to mongo (while stream are also passthrough)
  .pipe(response); // stream same results back to some request

@e2fyi/streams.DocumentTaggerSettings : Object

Settings for DocumentTagger.

Kind: static typedef of @e2fyi/streams
Properties

| Name | Type | Description | | --- | --- | --- | | autoIncrement | String | The auto-increment field to tag onto the stream object. | | ignoreNonObject | Boolean | If true, no errors will be emitted when the chunk in the stream cannot be parsed into Object. | | readableObjectMode | Boolean | If true, Object will be emitted from the stream, otherwise a String or Buffer representation or will emitted instead. | | objectMode | Boolean | If true, Object will be emitted from the stream, otherwise a String or Buffer representation or will emitted instead. Overwrites writableObjectMode. | | filter | function | Function to filter the objects in the stream. Return true to keep the object. | | mutate | function | Object | Function or Object to mutate the stream. If an Object is provided, each stream object will be mutated with the Object.assign(streamObj, mutateObj). |

@e2fyi/streams.MongooseStreamSettings : Object

Settings for MongooseStream.

Kind: static typedef of @e2fyi/streams
Properties

| Name | Type | Default | Description | | --- | --- | --- | --- | | itemWaterMark | Number | 50 | The number of item collected before writing to mongodb. | | passThrough | Boolean | | If false nothing will be emitted from the stream. | | model | mongoose.Model | | mongoose Model. |

Development

Unit testing

npm test

Build documentation

npm run build