npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@aws/s3-managed-downloader

v0.9.0

Published

Managed Downloader for S3

Downloads

1,377

Readme

AWS SDK JavaScript S3 Managed Download

The JavaScript SDK Managed Download can be used for custom S3 downloads. The client can configure options for the Managed Download such as chunk size and number of concurrent transfers.

Installing

In Node.js

This package only works in Node.js versions 6+ currently. Use the npm package manager for Node.js to install the Managed Download package. Type the following command in a terminal window.

    npm install @aws/s3-managed-download

Documentation

You can find the full documentation for the Managed Download package here.

Configuration

You can customize the part size and number of concurrent downloads for the Managed Download by setting the maxPartSize and maxConcurrency options. The maxPartSize option controls the size in bytes of each part of the transfer. The maxConcurrency option specifies the number of parts that will be downloaded in parallel and the maximum number of parts that will be buffered in memory at any time.

Import the Managed Download package

You will need an S3 client in order to create a managed download, so import the aws-sdk along with the managed download package.

JavaScript

    const S3 = require('aws-sdk/clients/s3');
    const ManagedDownloader = require('@aws/s3-managed-download').ManagedDownloader;

TypeScript

    import * as S3 from 'aws-sdk/clients/s3';
    import { ManagedDownloader, GetObjectStreamInput, ManagedDownloaderOptions } from '@aws/s3-managed-download';

Create a Managed Download with the default part size and concurrency

Create an AWS S3 client and pass the client into the Managed Download constructor to create a Managed Download with a part size of 5MB and concurrency of 1.

JavaScript

    const s3 = new S3();
    const managedDownloader = new ManagedDownloader(s3);

TypeScript

    const s3:S3 = new S3();
    const managedDownloader:ManagedDownloader = new ManagedDownloader(s3);
    

Create a Managed Download with custom part size and concurrency

Create an AWS S3 client and Managed Download options. Pass the client and the options into the Managed Download constructor to create a Managed Download with a custom part size of 10MB and concurrency of 5.

JavaScript

    const s3 = new S3();
    const options = {
        maxPartSize: 10 * 1024 * 1024,
        maxConcurrency: 5
    };
    const managedDownloader = new ManagedDownloader(s3, options);

TypeScript

    const s3:S3 = new S3();
    const options:ManagedDownloaderOptons = {
        maxPartSize: 10 * 1024 * 1024,
        maxConcurrency: 5
    };
    const managedDownloader:ManagedDownloader = new ManagedDownloader(s3, options);

Examples

Currently, the Managed Download only contains a streaming download operation which can work for an entire file or a specific range or part of a file. Here are some examples on how to use the Managed Download's getObjectStream download operation.

1. Use the Managed Download object to create a download stream for a file on S3

Get the file 'example-key' from the bucket 'example-bucket' and use the getObjectStream method to create a local file at 'example-file-path'.

JavaScript

    const S3 = require('aws-sdk/clients/s3');
    const ManagedDownloader = require('@aws/s3-managed-download').ManagedDownloader;
    const fs = require('fs');

    const s3 = new S3();
    const managedDownloader = new ManagedDownloader(s3);

    const params = {
        Bucket: 'example-bucket',
        Key: 'example-key'
    };
    // create a write stream for a file
    const writeStream = fs.createWriteStream('example-file-path');
    
    managedDownloader.getObjectStream(params)
    .then((stream) => {
        stream.pipe(writeStream);
    }, (err) => {
        writeStream.end();
        console.error(err);
    });

TypeScript

    import * as S3 from 'aws-sdk/clients/s3';
    import { ManagedDownloader, GetObjectStreamInput, ManagedDownloaderOptions } from '@aws/s3-managed-download';
    import * as fs from 'fs';

    const s3:S3 = new S3();
    const managedDownloader:ManagedDownloader = new ManagedDownloader(s3);

    const params:GetObjectStreamInput = {
        Bucket: 'example-bucket',
        Key: 'example-key'
    };
    // create a write stream for a file
    const writeStream:fs.WriteStream = fs.createWriteStream('example-file-path');
    
    managedDownloader.getObjectStream(params)
    .then((stream) => {
        stream.pipe(writeStream);
    }, (err) => {
        writeStream.end();
        console.error(err);
    });

2. Use the Managed Download object to create a download stream for a range of bytes of a file on S3

Get the file 'example-key' from the bucket 'example-bucket' and use the getObjectStream method to write to bytes 100-150 of a local file at 'example-file-path'.

JavaScript

    const S3 = require('aws-sdk/clients/s3');
    const ManagedDownloader = require('@aws/s3-managed-download').ManagedDownloader;
    const fs = require('fs');
    
    const s3 = new S3();
    const managedDownloader = new ManagedDownloader(s3);

    const params = {
        Bucket: 'example-bucket',
        Key: 'example-key',
        Range: '100-150'
    };
    // create a write stream for a file starting at byte 100
    const writeStream = fs.createWriteStream('example-file-path', {start:100});
    
    managedDownloader.getObjectStream(params)
    .then((stream) => {
        stream.pipe(writeStream);
    }, (err) => {
        writeStream.end();
        console.error(err);
    });

TypeScript

    import * as S3 from 'aws-sdk/clients/s3';
    import { ManagedDownloader, GetObjectStreamInput, ManagedDownloaderOptions } from '@aws/s3-managed-download';
    import * as fs from 'fs';

    const s3:S3 = new S3();
    const managedDownloader:ManagedDownloader = new ManagedDownloader(s3);
    
    const params:GetObjectStreamInput = {
        Bucket: 'example-bucket',
        Key: 'example-key',
        Range: '100-150'
    };
    // create a write stream for a file starting at byte 100
    const writeStream:fs.WriteStream = fs.createWriteStream('example-file-path', {start:100});
    
    managedDownloader.getObjectStream(params)
    .then((stream) => {
        stream.pipe(writeStream);
    }, (err) => {
        writeStream.end();
        console.error(err);
    });

Opening Issues

If you find any bugs or want to request a new feature, please first check the existing issues. If there is not already an issue, then open a new issue. If the issue is regarding a bug, please include the stack trace and a method to recreate it.

License

This library is licensed under the Apache 2.0 License.