A tool that handles compressing, bundling, and unpacking production node_modules dependencies that you want to store in source control

Table of Contents

• Overview
• Requirements
  • Developer Requirements
• Usage
• Optimization
• Advanced Usage
• Performance
• Contributing
• License

Overview

Probundle is a project that brings pac back to life (and support modern versions of Node.js. To achieve the fastest deployments possible, the production node_modules/ (not devDependencies or any of that optional stuff) can be compressed by running probundle pack and stored in source control. Then, when deployment time comes all you have to do is uncompress or run probundle unpack. This works well when you also run node-prune on your node_modules/ folder, before compressing your production dependencies. We even take it a step further in some cases and create global shared configurations for libraries like ESLint and Prettier that dynamically load their plugins into node_modules/ by downloading compressed plugins from source control without having to run npm install. We fuss about the performance of our node_modules/ installation time because it is often a bottleneck.

Requirements

If you are simply including this library in your project, all you need is a recent version of Node.js. Node.js >14.18.0 is sometimes required and is the only version range we actively support. Albeit, it is highly probable that lower versions will work as well depending on the requirements that this project imports.

Developer Requirements

The following versions of Node.js and Python are required for development:

• Node.js >14.18.0
• Python >3.10.0

Other versions may work, but only the above versions are supported. Most development dependencies are installed automatically by our Taskfile.yml set-up (even Node.js and Python). Run bash start.sh to install Bodega (an improved fork of go-task) and run the initialization sequence. The taskfiles will automatically install dependencies as they are needed, based on what development tasks you are running. For more information, check out the CONTRIBUTING.md or simply run:

npm run help

npm run help will ensure Bodega is installed and then open an interactive dialog where you can explore and learn about various developer commands.

Usage

After ensuring your node_modules/ folder only contains the files you want in your compressed, production packages, simply run:

probundle pack

This will create compressed .tgz files for each of the packages in the node_modules/ folder. You can then save these in source control or wherever you like. The compressed files will be stored in the .modules/ folder.

When you are ready to deploy your compressed packages, perhaps after freshly cloning your repository, just run:

probundle unpack

Optimization

Before compressing your production node_modules/, you might want to take a couple steps to ensure that only what is needed is present in the node_modules/ folder.

Start off by clearing your node_modules/ folder:

rm -rf node_modules

Then, install only the dependencies by running:

npm install --only=prod

Next, ensure node-prune is installed and use it to remove unnecessary files from the node_modules/ folder by running:

node-prune

At this point, you should check the size of the node_modules/ folder and ensure that it is a modest size. If it is not and you still want to keep your production node_modules/ in your source control repository, then you might want to consider wiping the history of the .modules/ folder periodically. You can follow the instructions here, replacing rm -rf node_modules with rm -rf .modules.

Advanced Usage

The node_modules/ folder can get pretty big, pretty quickly, so you might want to consider saving your .modules to an S3 bucket. You can get really fancy with it and actually bind the .modules folder in your project to an S3 Fuse mount using software like goofys. If you are interested, check out our free Ansible role for goofys.

You can get even fancier with it by doing things like dynamically loading your node_modules/. Sometimes you might have a library that only needs a dependency sometimes. In that case, you can download the compressed .tgz wherever it is stored and load it on-demand. You can load a single module by running a command similar to this one:

probundle unpack eslint

For the above snippet to work, you would have to have the compressed eslint.tgz file stored in .modules/eslint.tgz.

Performance

For even more performance, consider using pnpm. Although we use pnpm normally, we actually use npm for building the node_modules/ folder that Probundle compresses. You can get a better idea of how we implement Probundle by checking out this Taskfile.yml which is powered by our custom task-runner called Bodega.

Contributing

Contributions, issues, and feature requests are welcome! Feel free to check the issues page. If you would like to contribute, please take a look at the contributing guide.

Brian Zalewski

License

Copyright © 2020-2021 Megabyte LLC. This project is MIT licensed.