npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

node-file-uploader-s3

v2.0.5

Published

node file uploader for large file

Downloads

25

Readme

Uploader Class Explanation

The Uploader class is a JavaScript implementation designed to handle file uploads within a Node.js environment. It is equipped to manage uploads from multiple clients simultaneously by leveraging unique file identifiers and organizing files into user- and project-specific directories. Below is a detailed explanation of its components and functionality.

Class Structure

  • Constructor: Initializes the this.uploads object to track the status of ongoing uploads.

  • safeFileName (Static Method): Sanitizes the original file name to prevent directory traversal or invalid characters, returning a filename based on the uniqueFileId.

  • uploadStatus: Checks if the file already exists in the specified userID and projectID directory and creates the directory if necessary. If the file exists, it verifies if the uploaded size matches the expected size.

  • uploadFiles: Handles the file upload process, including validating headers, determining the byte range for partial uploads, and appending data to the target file. It supports uploading from the beginning or resuming part-way through a file.

  • uploadComplete: Finalizes the upload process by generating a URL for the uploaded file. It's intended for post-upload processing, such as moving the file or updating a database.

  • initializeUpload: Prepares for an upload by initializing tracking for the uploaded file's size. This method is essential for tracking progress but is not utilized in the provided code.

  • extractHeaders: Extracts necessary information from the request headers, including uniqueFileId, fileSize, userID, projectID, and contentRange.

  • parseContentRange: Parses the Content-Range header to determine the current upload chunk's start and end bytes, crucial for handling partial uploads.

Handling Multiple Client Uploads

The class manages uploads from multiple clients by using unique file identifiers and organizing files into specific directories based on the user and project.

Unique Identifiers

  • Utilizes uniqueFileId for each file, allowing the server to differentiate between files, even with identical names, and manage simultaneous uploads effectively.

Directory Structure

  • Creates separate directories for each user and project (uploads/${userID}/${projectID}/), preventing file conflicts and organizing uploads efficiently.

Partial Uploads

  • Supports partial uploads through Content-Range handling, enabling clients to resume interrupted uploads without starting from scratch.

Concurrency Management

  • Although not explicitly implementing concurrency controls, the class benefits from Node.js's non-blocking I/O model, efficiently handling concurrent upload streams.

Summary

The Uploader class provides a robust solution for managing file uploads in web applications, supporting features like resuming uploads, handling files from multiple clients, and organizing files in a structured manner. It leverages Node.js's asynchronous capabilities to efficiently process simultaneous upload requests.

Installation

Install the module using npm:

npm install --save node-file-uploader

Usage

const fs = require("fs");
const cors = require("cors");
const express = require("express");
const Uploader = require("node-file-uploader");
const app = express();

Initialize the Express app and Uploader instance:

const uploaderInstance = new Uploader();

Define Express routes to handle file upload actions:

const minioClientSettings = {
  endPoint: "your minio end point",
  port: port,
  useSSL: true|false,
  accessKey: "accessKey",
  secretKey: "secretKey",
  bucket: "bucket",
};

app.get("/upload/status", (req, res) => {
  uploaderInstance.uploadStatus(req, res);
});

app.post("/upload/files", (req, res) => {
  uploaderInstance.uploadFiles(req, res, minioClientSettings);

});

app.post("/upload/complete", (req, res) => {
  uploaderInstance.uploadComplete(req, res, minioClientSettings);
});