@adobe/sizewatcher

v1.3.0

Published

19 days ago

Warns if your pull requests introduce large size increases.

Downloads

1,063

sizewatcher

sizewatcher is a CI tool that automatically warns about size increases in Github pull requests. This allows early detection of commonly undesirable issues such as

addition of large dependencies (and transient dependency trees)
accidental addition of large binary files to the git repository
sudden increase in build artifact size

While any custom file or folder path can be measured via configuration, various types are automatically measured, including git repository, Node module dependencies and npm package size - see comparators reference. More built-in languages & options are added over time and contributions are welcome.

sizewatcher runs as part of your CI and reports results as comment on the pull request or as github commit status (optional), allowing to block PRs if a certain threshold was exceeded.

:warning: The git size comparison feature (the git comparator) is currently not calculating correct values (as of 1.2.1). See issue #52 for more details.

This is an example of a sizewatcher Github PR comment with a failure (ignore the small numbers):

PR branch: testconfig @ 21b66dfe6c3f6d09d4929ea2dec1e62cd1a7e7f2
Base branch: main
Sizewatcher v1.0.0
Effective Configuration:

limits:
  fail: 100%
  warn: 30%
  ok: '-10%'
report:
  githubComment: true
  githubStatus: false
comparators:
  git:
    limits:
      fail: 50%
  custom: null

And here if everything looks good:

PR branch: testconfig @ 21b66dfe6c3f6d09d4929ea2dec1e62cd1a7e7f2
Base branch: main
Sizewatcher v1.0.0
Effective Configuration:

limits:
  fail: 100%
  warn: 200%
  ok: '-10%'
report:
  githubComment: true
  githubStatus: false
comparators: {}

How it works

By default sizewatcher will

checkout the before and after branch versions in temporary directories
go through all comparators that apply
measure the sizes, compare and report
- fail ❌ at a 100%+ increase
- warn ⚠️ at a 30%+ increase
- report ok ✅ if the size change is between -10 and +30%
- cheer 🎉 if there is a 10% decrease
print result in cli output
report result as PR comment
not set a commit status
- as this will block the PR if it fails
- opt-in using report.githubStatus: true, see Configuration below

Requirements

Nodejs version 12+ (since 1.3.0)
- recommended to use latest stable version "LTS"
Github or Github Enterprise
- if you want to run it on pull requests
CI system running on Github pull requests
- Github Actions (simplest setup)
- Travis CI
- CircleCI
- others might work too

Installation

Local: You can install the sizewatcher tool locally for testing or checking your changes before committing:

npm install -g @adobe/sizewatcher

For setup in Continuous Integration: see CI Setup section.

Usage

When run locally, sizewatcher will output its results on the command line.

See the command line help with:

sizewatcher -h

Help output:

Usage: sizewatcher [<options>] [<before> [<after>]]

Arguments:
  <before>   Before branch/commit for comparison. Defaults to default branch or main/master.
  <after>    After branch/commit for comparison. Defaults to current branch.

Options:
  -h     Show help

In the simplest case, run inside a git repository without arguments. This will compare the current checked out branch against the base (main or master):

sizewatcher

Example output:

> sizewatcher
Cloning git repository...
Comparing changes from 'main' to 'docs'

Calculating size changes for
- git
- node_modules

Sizewatcher measured the following changes:

  'main' => 'docs' (sha 15626a050330492da8b745dadb4f5d304b670e83)

+ ✅ git: 0.2% (179 kB => 179 kB)

  Largest files in new changes:

  360cccafa87c    974B .npmignore
...
Error: Cannot identify github repository. Cannot comment on PR or update status checks in github.
Done. Cleaning up...

Note that the message Error: Cannot identify github repository. will be normal if run locally.

To compare the current branch with a different base branch (base):

sizewatcher base

To compare arbitrary branches or revisions before with after:

sizewatcher before after

CI Setup

CI Overview

Run using npx

To run sizewatcher in your CI, which is where it needs to run automatically for checking pull requests, it is best run using npx, which comes pre-installed with nodejs and will automatically download and run the latest version in one go:

npx @adobe/sizewatcher

This command will always use the latest published version. In some cases it might be (temporarily) desireable to stick to a certain version, which can be achieved using:

npx @adobe/[email protected]

GITHUB_TOKEN

To be able to automatically comment on the PR or report a commit status, a github token must be set as environment variable:

GITHUB_TOKEN=....

This token should be a service/bot user that has read/pull permission on the repository (allowing to comment). Note that the comments will be shown under that user's name. For example, you might want to create a user named sizewatcher-bot or the like. With Github Actions this is not required, it has a built-in github-actions bot user.

GITHUB_API_URL

If you use Github Enterprise, set the custom Github API URL in the GITHUB_API_URL environment variable:

GITHUB_API_URL=https://mygithub.company.com/api/v3/

This is not required for public github.com.

Order of steps

You can run sizewatcher before, after or in parallel to your main build step. It does not rely on output of a build, since it will check out the code (before and after PR versions) separately and needs to run any build steps itself, such as using script in the custom comparator.

See below for CI specific setup.

Github Actions

For Github Actions you need to

ensure Node.js is installed using actions/setup-node
run npx @adobe/sizewatcher
set a GITHUB_TOKEN which can leverage the built-in secrets.GITHUB_TOKEN (no need to create the token yourself!)

Example workflow yaml snippet (.github/workflows/*.yml):

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - name: Install Node.js
      uses: actions/setup-node@v1
      with:
        node-version: '14'
    # ---------- this runs sizewatcher ------------
    - run: npx @adobe/sizewatcher
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Travis CI

For Travis CI you need to

use language: node_js
- if you already use a different language, find a way to ensure Nodejs 12+ is installed
under script run npx @adobe/sizewatcher
set a secret environment variable GITHUB_TOKEN in the Travis repository settings with a github token with permission to comment on PRs and reporting commit statuses for the repository

Example .travis.yml:

language: node_js

install:
  - npm install

script:
  # ---------- this runs sizewatcher ------------
  - npx @adobe/sizewatcher

CircleCI

For CircleCI you need to

use a docker image with Nodejs 12+ installed
- alternatively install using nvm
run npx @adobe/sizewatcher
set a secret environment variable GITHUB_TOKEN in the CircleCI project settings (or in a Context) with a github token with permission to comment on PRs and reporting commit statuses for the repository

Example .circleci/config.yml:

version: 2

jobs:
  build:
    docker:
      - image: circleci/node:14
    steps:
      - checkout

      # ---------- this runs sizewatcher ------------
      - run: npx @adobe/sizewatcher

Other CIs

This is not tested well but might work.

Ensure Nodejs 12+ is installed.

Set these environment variables in the CI job:

GITHUB_BASE_REF name of the base branch into which the pull request is being merged
GITHUB_HEAD_REF name of the branch of the pull request
GITHUB_TOKEN a github token with permission to comment on PRs and reporting commit statuses for the repository. This is a credential so use the proper CI credential management and never check this into the git repository!

Then simply run

npx @adobe/sizewatcher

Configuration

Configure the behavior of sizewatcher by creating a .sizewatcher.yml in the root of the git repository. The config file is entirely optional.

Complete example and reference:

report:
  # to report a github commit status (will block PR if it fails)
  # default: false
  githubStatus: true
  # to report a comment on the github PR
  # default: true
  githubComment: false

# global thresholds when to warn or fail a build
# note that one failing or warning comparator is enough to fail or warn
# can be either
# - percentage: "50%" ("-10%" for size decrease)
# - absolute limit, as byte string: "10 MB", "5 KB"
#   see https://www.npmjs.com/package/xbytes
# - absolute limit, as byte number: 1000000
limits:
  # when to fail - default: 100%
  fail: 50%
  # when to warn - default: 30%
  warn: 10%
  # below the ok limit you will get a cheers for making it notably smaller
  # default: -10%
  ok: -5%

# configure individual comparators
# see list below for available comparators - use exact names as yaml keys
# by default all comparators run if they detect their content is present
comparators:
  # set a comparator "false" to disable it
  git: false

  # customize comparator
  node_modules:
    # specific limits
    # same options as for the "limits" at the root
    limits:
      fail: 10 MB
      warn: 9 MB
      ok: 1 MB

  npm_package:
    # `dir` is supported for all comparators that support it and
    # specifies the relative directory inside the project inside which to run the comparator
    dir: "sub/folder"
    # can also be an array with multiple directories to check
    dir:
      - "sub/folder1"
      - "folder2"

  # custom comparator (only active if configured)
  custom:
    - name: my artifact
      # path to file or folder whose size should be measured
      # path must be relative to repo root
      # comparator only runs if that path exists
      path: build/artifact

    # there can be multiple custom comparators
    # name defaults to the path
    # path can include glob patterns
    - path: "artifact-*.tgz"
      # run some custom build command before measuring
      script: npm install
      # limits can be configured as well
      limits:
        fail: 10 MB

Comparators reference

git

Compares the size of the git repository by measuring the .git folder. Useful to detect if large files are added to the repo.

Name: git

Trigger: Runs if a .git directory is found.

Details: Shows the largest files (git objects) added in the PR:

Largest files in new changes:

710a7c687b06  2.8KiB lib/render.js
0a8f1a2ddb4f  2.4KiB test/config.test.js
6846cf298cd4  2.3KiB test/config.test.js
a643d322cc26  1.5KiB lib/config.js
933e4432aae5     73B .sizewatcher.yml
6db7d5c27a66     69B .sizewatcher.yml

node_modules

Compares the size increase of Node.js dependencies by measuring the size of the node_modules folder. Helps to prevent addition of needlessly large dependencies and slowing down npm install times.

Name: node_modules

Trigger: Runs if a package.json is found.

Details: Prints the largest modules using cost-of-modules:

Largest node modules:

┌───────────────┬─────────────┬────────┐
│ name          │ children    │ size   │
├───────────────┼─────────────┼────────┤
│ @octokit/rest │ 33          │ 11.25M │
├───────────────┼─────────────┼────────┤
│ js-yaml       │ 3           │ 0.72M  │
├───────────────┼─────────────┼────────┤
│ simple-git    │ 2           │ 0.24M  │
├───────────────┼─────────────┼────────┤
│ tmp           │ 13          │ 0.22M  │
├───────────────┼─────────────┼────────┤
│ debug         │ 1           │ 0.08M  │
├───────────────┼─────────────┼────────┤
│ deepmerge     │ 0           │ 0.03M  │
├───────────────┼─────────────┼────────┤
│ require-dir   │ 0           │ 0.02M  │
├───────────────┼─────────────┼────────┤
│ du            │ 1           │ 0.01M  │
├───────────────┼─────────────┼────────┤
│ pretty-bytes  │ 0           │ 0.01M  │
├───────────────┼─────────────┼────────┤
│ 9 modules     │ 34 children │ 4.57M  │
└───────────────┴─────────────┴────────┘

Configuration: supports dir

npm_package

Compares the size of an npm package tarball by running npm publish --dry-run.

Name: npm_package

Trigger: Runs if a package.json is found.

Details: Prints the package contents and metadata using the output of npm publish --dry-run.

Package contents:

📦  @adobe/[email protected]
=== Tarball Contents ===
11.3kB LICENSE
4.8kB  lib/checkout.js
5.2kB  lib/compare.js
1.7kB  lib/config.js
3.9kB  lib/comparators/custom.js
2.3kB  lib/comparators/git.js
2.6kB  lib/github.js
2.1kB  index.js
2.2kB  lib/comparators/node_modules.js
3.9kB  lib/render.js
4.6kB  lib/report.js
1.1kB  package.json
695B   CHANGELOG.md
23.9kB README.md
=== Tarball Details ===
name:          @adobe/sizewatcher
version:       1.0.0
package size:  18.6 kB
unpacked size: 70.3 kB
shasum:        80846caccca2194f3dd1122e8113206e20c202dc
integrity:     sha512-c3VjMQQvqcqN8[...]ybMS6kg2chjpA==
total files:   14

Configuration: supports dir

custom

Compares the size of a custom file or folder.

Name: custom

Trigger: Runs if the path is found in either before or after version.

Details: Shows the new file/folder size.

New size:

18.6 kB README.md

Configuration:

This comparator requires configuration in the .sizewatcher.yml:

comparators:
  custom:
    path: src/somefile

If a build step is required first, use script:

comparators:
  custom:
    path: "build/artifact-*.tgz"
    script: npm run build

To have multiple paths, with custom names:

comparators:
  custom:
    - name: my artifact 1
      path: "build/artifact"
    - name: my artifact 2
      path: "build/artifact2"

Options:

path (required) relative path to file or folder to measure; supports glob patterns. make sure to use quotes in the yaml if you use glob patterns
name (optional) custom label
script (optional) shell command to run before measuring path
limits can be set as usual

Contribute

Contributions are welcome! Read the Contributing Guide for general guidelines.

New comparator

If you want to add a new automatic comparator for language X or dependency manager Y, follow these steps:

search issues for any existing similar comparators being discussed
if not, create a new issue
fork this repo
under lib/comparators add your new comparator, say superduper.js

must export an object with these functions:

module.exports = {

    shouldRun: async function(beforeDir, afterDir) {
        // return true if it should run, otherwise false to skip
        // use to automatically detect language, package manager etc.
        return true;
    },

    compare: async function(before, after) {
        // measure differences between before (base branch) and after (pull request)
        // before and after are objects with
        // - `dir`: directory
        // - `branch`: name of the branch
        // - `sha`: (only on "after") commit sha of the PR currently being looked at

        // return an object with
        // - `beforeSize`: size of the before state
        // - `afterSize`: size of the after state
        // - `detailsLabel`: custom label for the details section (shown in expanded section on PR comment)
        // - `details`: text with custom details on the after state (largest files etc)
        return {
            beforeSize: 100
            afterSize: 200
            detailsLabel: "Largest files in new changes",
            details: "... details"
        }
    }
}

test and validate
create PR

Licensing

This project is licensed under the Apache V2 License. See LICENSE for more information.