conditional-aggregate-webpack-plugin: delays webpack aggregateTimeout triggered watch-build until some custom condition meets

This webpack plugin allows to add a custom callback which, if it returns true, allows webpack to proceed with the incremental build in --watch mode. Until the callback returns true, all the changes are accumulated, and webpack is not wasting CPU; instead, it waits until the condition is met.

Motivation

Sometimes webpack is used to bundle artifacts generated by other tools. An example is TypeScript's tsc in a large monorepo with cross-dependencies between packages, where tsc generates dist/ folders with *.{js,map,d.ts} files, and then webpack uses these dist/*.js and dist/*.map to build public/ and not waste time on dependent packages rebuild from their sources.

The problem is that, when the monorepo is large, there can be multiple tsc --watch processes running concurrently (one per each monorepo package), and the set of tsc builds does not settle immediately after e.g. git pull affecting multiple packages. For instance, if there are A -> B -> C packages dependencies in the monorepo, they all start watch-building after the pull, and until e.g. C is fully built, A and B builds may fail. Until they are all built successfully, there is no sense to let webpack to run the incremental build, because (in a large monorepo) it will just waste CPU and flood console with useless error messages.

This plugin allows to resolve this usecase. It allows to verify that all tsc sub-projects are built successfully without errors. If they aren't, then there is no sense in triggering a webpack watch-build (it burns lots of CPU cycles and freezes the laptop in a large monorepo project, especially when there are multiple webpack built packages involved); instead, it's better to wait until all monorepo packages are watch-built by tsc successfully.

Example

The plugin is useful together with tsc-watch tool which allows to hook into different tsc build stages:

# Running for each package in the monorepo:
tsc-watch \
  --onCompilationStarted 'bash -c "echo building > dist/status"' \
  --onSuccess 'bash -c "echo success > dist/status"' \
  --onFailure 'bash -c "echo building > dist/status"'

Once this command is running, there will be an always up-to-date dist/status file reflecting the current state of tsc build. Then, you can configure ConditionalAggregateWebpackPlugin to read these files and unfreeze webpack incremental build only when ALL the status files contain "success":

// webpack.config.ts
import { readFileSync } from "fs";
import { basename, dirname } from "path";
import ConditionalAggregateWebpackPlugin from "conditional-aggregate-webpack-plugin";

export default function() {
  return {
    // ...,
    plugins: [
      new ConditionalAggregateWebpackPlugin({
        recheckInterval: 200, // how often do we call the condition function
        condition: () => {
          const statuses: string[] = [];
          for (const file of glob.sync(`${__dirname}/packages/*/dist/status`)) {
            const content = readFileSync(file).toString();
            if (!content.match(/success/)) {
              statuses.push(
                basename(dirname(dirname(file))) + ":" + content.trim()
              );
            }
          }

          return statuses.length === 0 ? true : ["waiting for", ...statuses];
        },
      }),
      // ...
    ]
  }
}