s3-zip

Download selected files from an Amazon S3 bucket as a zip file.

Changes

S3 File Names can be replaced with given alias if we provide File Alias in aliasConfig.

s3Zip
  .archive({ region: region, bucket: bucket, aliasConfig: aliasConfig }, [{ folder: 'folderName', file: 'file'}])
  .pipe(output)

Install

npm install s3-zip

AWS Configuration

Refer to the AWS SDK for authenticating to AWS prior to using this plugin.

Usage

Zip specific files

const fs = require('fs')
const join = require('path').join
const s3Zip = require('s3-zip')

const region = 'bucket-region'
const bucket = 'name-of-s3-bucket'
const folder = 'name-of-bucket-folder/'
const file1 = 'Image A.png'
const file2 = 'Image B.png'
const file3 = 'Image C.png'
const file4 = 'Image D.png'

const output = fs.createWriteStream(join(__dirname, 'use-s3-zip.zip'))

s3Zip
  .archive({ region: region, bucket: bucket}, [{ folder: 'folderName', file: file1}])
  .pipe(output)

You can also pass a custom S3 client. For example if you want to zip files from a S3 compatible storage:

const aws = require('aws-sdk')

const s3Client = new aws.S3({
  signatureVersion: 'v4',
  s3ForcePathStyle: 'true',
  endpoint: 'http://localhost:9000',
})

s3Zip
  .archive({ s3: s3Client, bucket: bucket }, [{ folder: 'folderName', file: 'file'}])
  .pipe(output)

Zip files with AWS Lambda

Example of s3-zip in combination with AWS Lambda.

Zip a whole bucket folder

const fs = require('fs')
const join = require('path').join
const AWS = require('aws-sdk')
const s3Zip = require('s3-zip')
const XmlStream = require('xml-stream')

const region = 'bucket-region'
const bucket = 'name-of-s3-bucket'
const folder = 'name-of-bucket-folder/'
const s3 = new AWS.S3({ region: region })
const params = {
  Bucket: bucket,
  Prefix: folder
}

const filesArray = []
const files = s3.listObjects(params).createReadStream()
const xml = new XmlStream(files)
xml.collect('Key')
xml.on('endElement: Key', function(item) {
  filesArray.push(item['$text'].substr(folder.length))
})

xml
  .on('end', function () {
    zip(filesArray)
  })

function zip(files) {
  console.log(files)
  const output = fs.createWriteStream(join(__dirname, 'use-s3-zip.zip'))
  s3Zip
   .archive({ region: region, bucket: bucket, preserveFolderStructure: true }, [{ folder: 'folderName', file: files[0]}])
   .pipe(output)
}

Tar format support

s3Zip
  .setFormat('tar')
  .archive({ region: region, bucket: bucket }, folder, [file1, file2])
  .pipe(output)

Archiver options

We use archiver to create archives. To pass your options to it, use setArchiverOptions method:

s3Zip
  .setFormat('tar')
  .setArchiverOptions({ gzip: true })
  .archive({ region: region, bucket: bucket }, [{ folder: 'folderName', file: file1}])

Organize your archive with custom paths and permissions

You can pass an array of objects with type EntryData to organize your archive.

const files = ['flower.jpg', 'road.jpg'];
const archiveFiles = [
  { name: 'newFolder/flower.jpg' },

  /* _rw_______ */
  { name: 'road.jpg', mode: parseInt('0600', 8)  }
];
s3Zip.archive({ region: region, bucket: bucket }, [{ folder: 'folderName', file: files[0]}], archiveFiles)

Debug mode

Enable debug mode to see the logs:

s3Zip.archive({ region: region, bucket: bucket, debug: true }, [{ folder: 'folderName', file: files[0]}])

Testing

Tests are written in Node Tap, run them like this:

npm t

If you would like a more fancy coverage report:

npm run coverage