regexpu-core

regexpu is a source code transpiler that enables the use of ES2015 Unicode regular expressions in JavaScript-of-today (ES5).

regexpu-core contains regexpu’s core functionality, i.e. rewritePattern(pattern, flag), which enables rewriting regular expressions that make use of the ES2015 u flag into equivalent ES5-compatible regular expression patterns.

Installation

To use regexpu-core programmatically, install it as a dependency via npm:

npm install regexpu-core --save

Then, require it:

const rewritePattern = require('regexpu-core');

API

This module exports a single function named rewritePattern.

rewritePattern(pattern, flags, options)

This function takes a string that represents a regular expression pattern as well as a string representing its flags, and returns an ES5-compatible version of the pattern.

rewritePattern('foo.bar', 'u', { unicodeFlag: "transform" });
// → 'foo(?:[\\0-\\t\\x0B\\f\\x0E-\\u2027\\u202A-\\uD7FF\\uDC00-\\uFFFF]|[\\uD800-\\uDBFF][\\uDC00-\\uDFFF]|[\\uD800-\\uDBFF])bar'

rewritePattern('[\\u{1D306}-\\u{1D308}a-z]', 'u', { unicodeFlag: "transform" });
// → '(?:[a-z]|\\uD834[\\uDF06-\\uDF08])'

rewritePattern('[\\u{1D306}-\\u{1D308}a-z]', 'ui', { unicodeFlag: "transform" });
// → '(?:[a-z\\u017F\\u212A]|\\uD834[\\uDF06-\\uDF08])'

regexpu-core can rewrite non-ES6 regular expressions too, which is useful to demonstrate how their behavior changes once the u and i flags are added:

// In ES5, the dot operator only matches BMP symbols:
rewritePattern('foo.bar', '', { unicodeFlag: "transform" });
// → 'foo(?:[\\0-\\t\\x0B\\f\\x0E-\\u2027\\u202A-\\uFFFF])bar'

// But with the ES2015 `u` flag, it matches astral symbols too:
rewritePattern('foo.bar', 'u', { unicodeFlag: "transform" });
// → 'foo(?:[\\0-\\t\\x0B\\f\\x0E-\\u2027\\u202A-\\uD7FF\\uDC00-\\uFFFF]|[\\uD800-\\uDBFF][\\uDC00-\\uDFFF]|[\\uD800-\\uDBFF])bar'

The optional options argument recognizes the following properties:

Stable regular expression features

These options can be set to false or 'transform'. When using 'transform', the corresponding features are compiled to older syntax that can run in older browsers. When using false (the default), they are not compiled and they can be relied upon to compile more modern features.

• unicodeFlag - The u flag, enabling support for Unicode code point escapes in the form \u{...}.

  rewritePattern('\\u{ab}', '', {
    unicodeFlag: 'transform'
  });
  // → '\\u{ab}'
  
  rewritePattern('\\u{ab}', 'u', {
    unicodeFlag: 'transform'
  });
  // → '\\xAB'

• dotAllFlag - The s (dotAll) flag.

  rewritePattern('.', '', {
    dotAllFlag: 'transform'
  });
  // → '[\\0-\\t\\x0B\\f\\x0E-\\u2027\\u202A-\\uFFFF]'
  
  rewritePattern('.', 's', {
    dotAllFlag: 'transform'
  });
  // → '[\\0-\\uFFFF]'
  
  rewritePattern('.', 'su', {
    dotAllFlag: 'transform'
  });
  // → '(?:[\\0-\\uD7FF\\uE000-\\uFFFF]|[\\uD800-\\uDBFF][\\uDC00-\\uDFFF]|[\\uD800-\\uDBFF](?![\\uDC00-\\uDFFF])|(?:[^\\uD800-\\uDBFF]|^)[\\uDC00-\\uDFFF])'

• unicodePropertyEscapes - Unicode property escapes.

  By default they are compiled to Unicode code point escapes of the form \u{...}. If the unicodeFlag option is set to 'transform' they often result in larger output, although there are cases (such as \p{Lu}) where it actually decreases the output size.

  rewritePattern('\\p{Script_Extensions=Anatolian_Hieroglyphs}', 'u', {
    unicodePropertyEscapes: 'transform'
  });
  // → '[\\u{14400}-\\u{14646}]'
  
  rewritePattern('\\p{Script_Extensions=Anatolian_Hieroglyphs}', 'u', {
    unicodeFlag: 'transform',
    unicodePropertyEscapes: 'transform'
  });
  // → '(?:\\uD811[\\uDC00-\\uDE46])'

• namedGroups - Named capture groups.

  rewritePattern('(?<name>.)\\k<name>', '', {
    namedGroups: 'transform'
  });
  // → '(.)\1'

• unicodeSetsFlag - The v (unicodeSets) flag

  rewritePattern('[\\p{Emoji}&&\\p{ASCII}]', 'v', {
    unicodeSetsFlag: 'transform'
  });
  // → '[#\\*0-9]'

  By default, patterns with the v flag are transformed to patterns with the u flag. If you want to downlevel them more you can set the unicodeFlag: 'transform' option.

  rewritePattern('[^[a-h]&&[f-z]]', 'v', {
    unicodeSetsFlag: 'transform'
  });
  // → '[^f-h]' (to be used with /u)

  rewritePattern('[^[a-h]&&[f-z]]', 'v', {
    unicodeSetsFlag: 'transform',
    unicodeFlag: 'transform'
  });
  // → '(?:(?![f-h])[\s\S])' (to be used without /u)

• modifiers - Inline i/m/s modifiers

  rewritePattern('(?i:[a-z])[a-z]', '', {
    modifiers: 'transform'
  });
  // → '(?:[a-zA-Z])([a-z])'

Experimental regular expression features

These options can be set to false, 'parse' and 'transform'. When using 'transform', the corresponding features are compiled to older syntax that can run in older browsers. When using 'parse', they are parsed and left as-is in the output pattern. When using false (the default), they result in a syntax error if used.

Once these features become stable (when the proposals are accepted as part of ECMAScript), they will be parsed by default and thus 'parse' will behave like false.

Miscellaneous options

• onNamedGroup

  This option is a function that gets called when a named capture group is found. It receives two parameters: the name of the group, and its index.

  rewritePattern('(?<name>.)\\k<name>', '', {
    onNamedGroup(name, index) {
      console.log(name, index);
      // → 'name', 1
    }
  });

• onNewFlags

  This option is a function that gets called to pass the flags that the resulting pattern must be interpreted with.

  rewritePattern('abc', 'um', '', {
    unicodeFlag: 'transform',
    onNewFlags(flags) {
      console.log(flags);
      // → 'm'
    }
  })

Caveats

• Lookbehind assertions cannot be transformed to older syntax.
• When using namedGroups: 'transform', regexpu-core only takes care of the syntax: you will still need a runtime wrapper around the regular expression to populate the .groups property of RegExp.prototype.match()'s result. If you are using regexpu-core via Babel, it's handled automatically.

For maintainers

How to publish a new release

1. On the main branch, bump the version number in package.json:

   npm version patch -m 'Release v%s'

   Instead of patch, use minor or major as needed.

   Note that this produces a Git commit + tag.

2. Push the release commit and tag:

   git push --follow-tags

   Our CI then automatically publishes the new release to npm.

3. Once the release has been published to npm, update regexpu to make use of it, and cut a new release of regexpu as well.

Author

| | |---| | Mathias Bynens |

License

regexpu-core is available under the MIT license.