npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

s3-multipart

v1.2.2

Published

Easy multipart uploads to S3 from the browser

Downloads

3,215

Readme

s3-multipart

Features:

  • S3 Multipart uploads directly from the browser
  • Parallel transfer if needed
  • Retries
  • Progress information
  • Configurable for your backend

For large file uploads, larger than 5GB, S3 has the concept of multipart uploads, where the file is divided into smaller parts (max 5GB per chunk) and each part is transferred individually. This concept is well known, well documented, but if you want to do it directly from a browser, it is significantly less so.

To my knowledge, there are some other modules that does this, but had the wrong kind of dependencies for my taste, or were hard to configure.

s3-multipart aims to be very small, and very easy configure to your needs.

Note: To do multipart uploads from the browser, you need to use presigned URLs. These URLs most likely will have to be presigned by some kind of backend that you control. You need to set this up, it is not a part of this module.

API

The module exports a single class, S3Multipart:

import S3Multipart from "s3-multipart";

const s3multipart = new S3Multipart({
  createUpload: (file) => {
    /* ...return promise... */
  },
  getPartUrl: (file, uploadId, partNumber, partSize) => {
    /* ...return promise... */
  },
  completeUpload: (file, uploadId, etags) => {
    /* ...return promise... */
  },
  onProgress: (uploadedBytes, totalBytes) => {
    /* ... */
  },
});

s3multipart.upload(myFile).then(() => console.log("success!"));

The class has three mandatory options: createUpload, getPartUrl and completeUpload. Each of them are expected to return a promise.

  • createUpload: creates a multipart upload on S3; see CreateMultipartUpload for details. The function should return a promise that resolves to the newly created upload's id.
  • getPartUrl: Returns a promise that resolves to a presigned URL for a PUT request to the given file and part number
  • completeUpload: completes the multipart upload after each part has been transferred; see CompleteMultipartUpload for details. In addition the file and upload id for the current transfer, the method also receives an array of the parts' ETag headers that is used by S3 to verify the integrity of each part you uploaded

Optional configuration:

  • onProgress: callback to indicate how much of the file has been transferred
  • partSize: number of bytes for each part; must be at least 5 MB and maximum 5 GB; default is 5 GB.
  • parallelism: number of parts to attempt to transfer simulatenously; defaults to 3.
  • retries: the number of times a part will be retried if it fails; default to 3 (one initial attempt + three retries, four in total)
  • retryBackoffTimeMs: number of seconds to wait after a failed upload attempt before retrying; can be a number or a function that accepts the current number of attempts and returns the time delay; defaults to a function that waits 1, 4 and 9 seconds (quadratic backoff)