npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@ryn-bsd/file-processing

v0.9.0

Published

file processing made easy

Downloads

3

Readme

Processing files for Juniors or Seniors is verbose if it is not hard ☹️. and that why I made this package, I made it:

  • Simple 🍰
  • Blazing fast 🚀
  • Customizable 🔧
  • Safe 🔒
  • And more...

How ?

Simple

Did you try to config file processor before ? (yes) 🫡 Did you try to build an api for it ? (yes) 🤓 Did you enjoy this process ? (no) 🫠

Blazing fast

Did you try to build a model for handling many files ? (yes) 😏 Did you try to search for fast (image/video/audio/pdf/csv...) processors ? (yes) 🧐 Did you enjoy the process ? (no) 🫠

Customizable

Did you try to build an api to fit all cases ? (yes) 😩 Did you try to keep it simple and fast ? (yes) 🙃 Did you enjoy the process ? (no) 🫠

Safe

Did you try to sanitize, filter, compress... ? (yes) 😖 Did you try to secure your server ? (yes) 🙃 Did you enjoy the process ? (no) 🫠

Intro

This is an async/sync/stream file processor, that handle convert, compression, generate, decompression, metadata, parse, filter, customize... and more!

How it work ?

This package is build on top:

What you think is it fast ? (of course, yes!)

What you think is it safe ? (of course, yes!)

Environment

make sure to specify if you are in production/development/test environment, this has an impact on performance.

Prerequisite

  • In case you want to add file scanning and anti-virus, make sure to install clamav

Usage

Image

import { core } from "@ryn-bsd/file-processing";
import Image from "@ryn-bsd/file-processing/core/image.js";

const _image = new core.Image(Buffer.alloc(1));
const image = new Image(Buffer.alloc(1));

image.length; // current images length
// => number

image.getImages(); // return stored images
// => Buffer[]

// filter non images and return the new length of images
image.setImages(
  /* async */ (image, index) => {
    return Buffer.concat([image, Buffer.alloc(index)]);
  },
);
// => Promise<number>

// This method filter non images and return the new length of images
image.append(Buffer.alloc(1));
// => Promise<number>

image.extend(new Image(Buffer.alloc(1)));
// => number

image.clone();
// => Image

// Free memory (remove all stored images buffer), you can reuse this instance again.
image.clean();

// this method filter non images and return the new length of images
image.filter();
// => Promise<number>

// convert all the image to another format
image.convert("jpeg", { quality: 100 });
// => Promise<{ data: Buffer; info: sharp.OutputInfo; }[]>

image.custom(
  /* async */ (sharp, index) => {
    return sharp.resize({ width: 1280, height: 720, fit: "cover" }).blur(index).toBuffer();
  },
);
// => Promise<Buffer[]>

image.custom(
  /* async */ (sharp, index) => {
    sharp.resize({ width: 1280, height: 720, fit: "cover" }).blur(index).toBuffer();
    return index;
  },
);
// => Promise<number[]>

// return metadata for all images
image.metadata();
// => Promise<sharp.Metadata[]>

Image.fromFile("image1.png", "image2.png");
// => Promise<Image>

Image.fromFile("https://example.com/image1.png", "https://example.com/image2.png");
// Promise<Image>

// Create a new safe instance
Image.new([Buffer.alloc(1)]);
// => Promise<Image>

// Create a new sharp instance
Image.newSharp();

AV

import { core } from "@ryn-bsd/file-processing";
import { Video, Audio } from "@ryn-bsd/file-processing/core/av/index.js";

const _audio = new core.Audio();
const audio = new Audio();

const _video = new core.Video();
const video = new Video();

Text

import { core } from "@ryn-bsd/file-processing";
import Text from "@ryn-bsd/file-processing/core/text.js";

const _text = new core.Text();
const text = new Text();

CSV

import { core } from "@ryn-bsd/file-processing";
import CSV from "@ryn-bsd/file-processing/core/csv.js";

const _csv = new core.CSV(Buffer.alloc(1));
const csv = new CSV(Buffer.alloc(1));

PDF

import { core } from "@ryn-bsd/file-processing";
import PDF from "@ryn-bsd/file-processing/core/pdf.js";

const _pdf = new core.PDF(Buffer.alloc(1));
const pdf = new PDF(Buffer.alloc(1));

Note: There are more apis, all tested apis are documented.

Versions

0.9.0 - Working

  • Remove puppeteer.
  • Remove loader from Core (class).
  • Remove parser from Core (class).
  • Generate hash.
  • Generate Hmac.
  • Encryption/Decryption.
  • Custom Error handler.
  • Video draw text.
  • Optimize performance.
  • Fix parser.toBuffer bug.

0.8.0 - Current

  • Add examples.
  • Refactor code to optimize performance.
  • Full code documentation.
  • Full organize code to build next versions (scalability).
  • Add benchmarks.
  • Take screenshots from video.
  • Get video audio.
  • Get only video (no audio).
  • Video/Audio split.
  • Add OCR.
  • Add instance check for each processor.
  • Improve code quality.
  • Performance optimizations.

0.7.0

  • Text unzip.
  • Image watermark.
  • PDF merge.
  • Image to pdf.
  • Fast file save (Core.toFile)
  • Improve code quality.
  • Improve security by adding eslint security plugin.
  • Update doc.
  • Performance optimizations.
  • Add a minified version.
  • Add formatter prettier.
  • Add esbuild to create a minified version.
  • Return length of elements in append/extend.
  • Static fn (new) for all processors to generate safe new instance.
  • Improve filter for setters.

0.6.0

  • Sync support.
  • Images screenshots (take screenshots from websites).
  • PDF generator (generate pdf from websites).
  • Stream implementation (not well tested) for CSV/Text.
  • Readable support (Core.toReadable).
  • Base64 support (Core.toBase64).
  • Text more metadata (charactersMap).
  • Text more customizable (compress/decompress).
  • Some APIs changed.
  • Performance optimizations.
  • Filter path/URL loaders.
  • deprecate input2buffer use Core.toBuffer.
  • Disable filter for CSV/Text append.
  • Improve code quality.
  • Update doc.

0.5.0

  • Support for images, avs, pdfs, csvs, texts.
  • Async support.
  • Buffer support (Core.toBuffer).
  • File base loader.
  • URL loader.

Todo

🔴 - not implemented yet. 🟠 - implemented, not tested yet. 🟢 - tested.

  • fix bugs (https://github.com/RYNBSD/file-processing/issues)
  • add new features.
  • add streaming. 🟠
  • Add dir loader. 🟠
  • Add glob loader. 🟠
  • Merge videos/audios in one video/audio. 🟠
  • Merge audio and video in one video. 🔴
  • Pdf to image. 🔴
  • File scanner (antivirus). 🔴
  • PDF/Video/Audio watermark. 🔴
  • PDF remove page. 🔴
  • PDF split. 🔴
  • Extract image from pdf. 🔴
  • Convert images to svg. 🔴
  • Convert svg to image. 🔴
  • Add archiver. 🔴

Support