Shell quoting DB

A collection of JSON-encoded configurations for escaping arguments on invoking shells.

License

MIT

About

This package has been created to document and consider different requirements of existing shells for properly escaping arguments to prevent unintended side effects e.g. due to passing functional characters.

All configurations in this package result from testing different ways of escaping every special character in ASCII code range with and without wrapping quotes. The resulting records can be read directly from a JSON file or via some convenience helper method.

To be clear: Quoting arguments in your code does not require to run any tests first. All tests mentioned here has been performed in advance to generate configuration files. Your code is going to simply use those configuration files to properly decide when to use what way of quoting and escaping characters based on an optionally (!) selected shell.

Testing procedure

Tests for assessing a shell are based on two scripts with one of the scripts repeatedly invoking the other one passing some data to be displayed by the latter script. The resulting output is then assessed according to whether data has been received by the second script as intended or not.

The tests cover special characters and whitespace in ASCII code page. They try passing them without any escaping as well as different common ways of escaping. In addition, they check how quoting an argument is affecting the result and how either tested character works in combination with regular letters.

Several Linux shells have been tested in a Debian-based Docker container. Windows' cmd.exe and powershell.exe have been manually tested on a Windows 11-based device.

Configurations

For every tested shell there is a JSON file describing test results prepared for quoting and escaping arguments when using either shell. Every file's data is divided into separate configurations per character optionally used to wrap an argument. This character is either the empty string for using no enclosing quotes or " or ' for using either character for enclosing the argument. A configuration for a quoting character may be missing in case the tested shell renders incapable of handling accordingly wrapped arguments.

Either configuration per quoting character consists of these properties:

• mapPattern is a string containing a regular expression matching any character that needs to be escaped.
• map is mapping every character requiring to be escaped into either one's escape sequence.
• rejectPattern is a string providing another regular expression matching characters that can't be escaped at all. Arguments matching this pattern should be rejected unless there is a different configuration working for them.
• invalid is mapping characters that can't be escaped into true for simplified lookups.
• prevent is map primarily for information purposes. It marks characters that should not be escaped with a preceding backslash or caret as those combinations will be visible to the invoked script. When using patterns as described before, it is safe to ignore this map, though.

In addition, there is a Javascript module providing a pre-compiled collection of all configurations for simplified integration e.g. with synchronously working code. This collection is limited to information necessary for the quoting process, thus lacking meta information per configuration and properties invalid and prevent as described above.

Usage

All configurations are provided as JSON files per supported shell and you can use them for whatever you intend to do.

const ashConfiguration = await import( "@cepharum/quoting-db/db/ash.json" );

However, this requires support for importing JSON modules. Another approach to fetching a shell's configuration is provided by a helper function:

const { getShellConfiguration } = require( "@cepharum/quoting-db" );

const ashConfiguration = await getShellConfiguration( "ash" );

The configuration can be used for quoting arguments on invoking a sub-process via some shell as illustrated in this example:

const { spawn } = require( "node:child_process" );
const { getShellConfiguration, quote } = require( "@cepharum/quoting-db" );

getShellConfiguration()
  .then( configuration => {
    const args = [ "--opt", "some argument with space", "--filter", "!$%><" ];
    const quoted = args.map( arg => quote( arg, configuration ) );
    
    const child = spawn( "some-tool", quoted, {
      shell: true,
    } );
  } );

Here, helper methods getShellConfiguration() and quote() are used.

• getShellConfiguration() promises the configuration of current system's default shell. You can provide a different shell's name as argument. The resulting configuration can then be used with quote() function.

• quote( rawArgument, configuration ) returns a sanitized version of a provided raw argument based on a shell's given configuration. It may throw in case the provided argument contains characters that can't be escaped at all.

Synchronous processing

If your code requires to work synchronously, fetching a full configuration from provided JSON files is not an option. The library provides a pre-compiled collection of those JSON files as Javascript module covering information necessary for quoting, only.

A modified version of previous example would look like this:

const { spawn } = require( "node:child_process" );
const { getShellConfigurationSync, quote } = require( "@cepharum/quoting-db" );

const configuration = getShellConfigurationSync();

const args = [ "--opt", "some argument with space", "--filter", "!$%><" ];
const quoted = args.map( arg => quote( arg, configuration ) );

const child = spawn( "some-tool", quoted, {
  shell: true,
} );

• getShellConfigurationSync() is the synchronous counterpart to getShellConfiguration(). It returns the configuration of current system's default shell. You can provide a different shell's name as argument. The resulting configuration can be used as second argument to the quote() function, too.

Additional API elements

• getShellName() is a helper function used internally by getShellConfiguration() and getShellConfigurationSync(). It delivers the normalized name of current system's default shell. You can also provide a shell's pathname as used with shell option of Node's spawn() method. In this case, the normalized basename of given shell is returned.

• The compiled collection of reduced configurations per shell is exposed as configurationPerShell.

  const { configurations, quote } = require( "@cepharum/quoting-db" );
    
  const quoted = quote( "$ome argument <to> e%scape", configurations.ash );