s3-deploy
v1.4.0
Published
NodeJS bash utility for deploying files to Amazon S3
Downloads
94,912
Readme
s3-deploy
NodeJS bash utility for deploying files to Amazon S3
Usage
s3-deploy './dist/**' --cwd './dist/' --region AWS_REGION --bucket SOME_BUCKET_NAME --distId CF_DIST_ID --invalidate '/INV_PATH_1 /INV_PATH_2'
Deploys files found by the ./dist/**
glob patten to S3. Change AWS_REGION
with the AWS region of your bucket and SOME_BUCKET_NAME
with the name of your bucket where files should end up. It's a common scenario to invalidate files in CloudFront distribution, just pass distribution ID to --distId
to invalidate everything.
Optional parameters
--gzip [ETX,...]
Specifying --gzip
will gzip all files before sending them to S3, and adds appropriate Content-Encoding: gzip
metadata attribute on uploaded elements in S3, so that when they are opened in the browser, the browser knows to decompress them first before displaying. You can specify the list of file extensions to gzip, e.g. --gzip xml,html,htm,js,css,ttf,otf,svg,txt
.
--cache X
Use this parameter to specify the Cache-Control: max-age=X
header, where X is the number of seconds a given item will be kept in the cache for. By default this value is undefined.
This parameter cannot be used with --immutable
or --noCache
parameters. If those are used together, only the first one will be taken into account in the order 1) noCache
, 2) cache
, 3) immutable
.
--preventUpdates
Use this parameter to validate that the object either currently does not exist in the bucket, or
that it already matches the ETag hash and thus is skipped and not uploaded. This effectively ensures that S3 Objects will not be modified. This is useful with the --immutable
flag.
--immutable
When a page is refreshed, which is an extremely common social media scenario, elements that were previously marked immutable with an HTTP response header do not have to be revalidated with the server. It sets the Cache-Control: immutable
header - using-immutable-caching-to-speed-up-the-web This is useful with the --preventUpdates
flag.
This parameter cannot be used with --cache
or --noCache
parameters. If those are used together, only the first one will be taken into account in the order 1) noCache
, 2) cache
, 3) immutable
.
--noCache
Use this parameter to specify Cache-Control: no-cache, no-store, must-revalidate
header, where files with those headers won't be cached on the client side, and will force all browsers and CloudFront to grab the assets straight from S3.
This parameter cannot be used with --cache
or --immutable
parameters. If those are used together, only the first one will be taken into account in the order 1) noCache
, 2) cache
, 3) immutable
.
--cacheControl X
Use this parameter to specify an entirely custom Cache-Control header, which will result in Cache-Control: X
. Use this if you have more complicated cache-control needs.
The --cacheControl
option supercedes all other cache parameters (no-cache
, cache
, and immutable
), which will be ignored.
--etag X
You can also specify the ETag: X
header, where X is either user-defined value for this header, or MD5 of the content. To automatically fill this header with MD5 hash of the file, just use --etag
parameter without any value. Internally the tool will generate MD5 hash of the content and will set it as the ETag header value. By default this parameter is undefined.
--signatureVersion v4
You can also specify the signatureVersion
that should be used by S3 client. Current allowed values are the same as in the constructor of the S3 JS SDK Client. In the time of writing those docs those values were v2
, v3
and v4
.
--filePrefix
Use this parameter to specify a file prefix for all your destination files. For example, if you wanted to deploy a versioned history of your project to S3 whenever publishing to npm, you could use --filePrefix $npm_package_version
in a script in your project's package.json file.
--profile
You can specify a specific AWS profile to use to connect to S3 (defaults to default
). More information on how to setup AWS profiles is available in the AWS docs.
--private
Use this parameter to specify that objects being uploaded will be stored with private ACL (Owner gets FULL_CONTROL. No one else has access rights). By default, 'public-read' ACL is set. More information on the canned-acl is available in the AWS docs
--ext
Enables to set the correct content type header when files has no extension. For example, when the s3 bucket is used for webhosting and there is need to access paths like /about
instead of /about.html
so its possible to upload file named about
and set --ext html
--deleteRemoved
Removes files in S3, that are not available in the local copy of the directory. Useful to cleanup files that should no longer reside in the remote version. Now it also has the abilityto specify glob pattern that defines filter for files that should be considered to be deleted.
--index
If a filename in a subdirectory matches the index file, then upload two additional copies of the file: one with the same name as the directory, and one with the name of the directory followed by "/". This can be used to fake "index.html" on S3 and CloudFront, which normally has no concept of default index files.
--distId
CloudFront Distribution ID. Use along with --invalidate
to invalides files within given distribution. In case --invalidate
isn't provided everything is invalidated.
--invalidate
Paths to invalidate within CloudFront distribution. Value must be wrapped with quotes and each file separated by whitespace character. --distId
argument is mandatory. Defaults to ['/']
. More details here:
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html.
AWS Credentials
AWS credentials can be provided via environment variables, or in the ~/.aws/credentials
file. More details here:
https://docs.aws.amazon.com/cli/latest/topic/config-vars.html. Please make sure to define a default in your AWS credentials, this will help prevent a Missing Credentials
error during deployment.
Commands
Production build
npm run release
Runs eslint validation, runs all unit tests.
Run all tests
npm test
Invokes all unit tests in the project.
Generage coverage report
npm run coverage
Generates unit test coverage report.
Run linting
npm run lint
Invokes eslint validation based on rules defined in the .eslintrc
file.
Releasing
- Commit changes to the repository on a separate branch,
- Bump version in package.json file, after you are done with your changes (remember about SemVer!),
- After you are done with your functionality, or if you think it is large enough, create a pull request with master branch to be peer reviewed,
- After changes are merged into master branch, checkout master branch, run tests one more time, and publish this package to npm repository.
Changelog
1.4.0
Adding new parameter --cacheControl
that allows specifying custom Cache-Control
header value.
1.3.0
Adding new parameter --noCache
that prevents caching of assets, and forces them to be grabbed straight from S3.
1.2.0
Adding ability to specify glob pattern to removeDeleted
that defines filter for files that should be considered to be deleted.
1.1.5
- Bug fix for
invalidate
flag. https://github.com/import-io/s3-deploy/pull/79
1.1.4
Build process cleanup & vinerability fixes
- Updated build process to use
prepublishOnly
, - Updated dependencies to fix vunerability issues.
1.1.2
Delete S3 objects recursively
- Support delete objects recursively for option
--deleteRemoved
- Reason: S3 listObjects only limit
MaxKeys: 1000
, so in case your deploy contains many files (much more than 1000), it may cause unexpected result (usually, it cannot delete all the necessary files) - This fork make sure all files in S3 will be
--deleteRemoved
if they are not matched with local files.
1.1.0
API Additions
- Pass file extensions to
--gzip
to zip only the given list. E.g.--gzip xml,html,htm,js,css,ttf,otf,svg,txt
.
1.0.0
No changes
The v0.11 re-published to follow stricter SemVer.
0.11.0
API Additions
- Added a new feature to invalidate paths in CloudFront using
--distId
(distribution id) and--invalidate
(paths to invalidate).
0.10.0
API Additions
- Addition of the
--preventUpdates
&--immutable
parameters.
0.9.0
API Additions
- Addition of the
--index
parameter.
0.8.0
API Additions
- Fix for windows setup.
0.7.6
API Additions
- Adding he ability to cleanup old files in the remote location during the deployment
0.7.3
API Additions
- Adding the ability to set the extension if the files without one or there is need to override it
0.7.2
Bug Fix
- Reverting changes to readFile function from PR https://github.com/import-io/s3-deploy/pull/11 as unfortunately it caused other issues: https://github.com/import-io/s3-deploy/issues/14
0.7.1
API Additions
- Adding the ability to set private ACL for object
0.6.1
Bug fix
- Fixing incorrect folder structure when
s3-deploy
is used from windows machine.
0.6.0
*API Additions
- Adding the ability to specify
filePrefix
0.5.2
Bug fix
- Fixing the
aws-sdk
package version to2.3.19
, because of: https://github.com/aws/aws-sdk-js/issues/1035
0.5.0
API Additions
- Adding the ability to specify
signatureVersion
in S3 client, - Fixing the tool to work correctly when no
--etag
argument is used.
0.4.0
API Additions
- Adding ability to provide ETag header value.
0.3.1
Patch/Bug Fixes
- Fixing an issue with cache parameter.
0.3.0
API Additions
- Adding ability to specify Cache-Control max-age seconds.
0.2.1
Bug/Patch fixes
- Moving
babel
to be a dev-dependency.
0.2.0
API additons
- Adding new command line parameter
--gzip
. When this is added, all files will be gzipped before sending them to Amazon S3.
0.1.7
Patch fixes
- Updating the repository URL.
0.1.6
Patch fixes
- Adding ability to publish package from CircleCI.
0.1.5
Bug fixes
- Adding a missing
crypto
import in the utils.
0.1.4
Patch fix
- Publishing the package publicly.
0.1.3
Bug fixes
- Switching to a node script in .bin directory, as bash script doesn't work when it is used through the symlink.
0.1.2
Bug fixes
- Going back to babel pre-compilation, and adding an .sh script to run the command later on,
- Adding babel/polyfill to the package as we are using generators inside the command.
0.1.1
Bug fixes
- Fixed a bug which made teh package not run at all.
0.1.0
API additions
- Initial version of the project,
- Ability to deploy files in the given glob pattern to provided S3 bucket, on provided S3 region.