Processing Instructions

Script markdown documents

Uses the mkparse library to form a DSL based on tags declared in processing instructions <? ... ?>.

Parses and executes processing instructions according to macro functions defined by a grammar; reads newline-delimited JSON records from an input stream, interprets and executes the instructions and writes the modified AST to an output stream.

Install

npm i mkpi --save

For the command line interface install mkdoc globally (npm i -g mkdoc).

• Install
• Security
• Usage
• Example
• Sample
• Macros
  • Grammar
  • Custom Macros
  • Macro Functions
• Help
• API
  • pi
  • Grammar
• License

Security

By default his library assumes you have trusted input. The exec and macro directives can run arbitrary commands and execute arbitrary javascript if your input is untrusted set the safe option and these directives are no longer recognised.

Usage

var pi = require('mkpi')
  , ast = require('mkast');
ast.src('<? @exec {shell} pwd ?>')
  .pipe(pi())
  .pipe(ast.stringify({indent: 2}))
  .pipe(process.stdout);

Example

Execute processing instructions in a file:

mkcat README.md | mkpi | mkout

Disable unsafe macros for untrusted input:

mkcat README.md | mkpi --safe | mkout

Sample

This readme document (raw version) was built from the source file doc/readme.md shown below:

# Processing Instructions

<? @include readme/badges.md ?>

> Script markdown documents

<? @include {=readme} introduction.md install.md ?>

***
<!-- @toc -->
***

<? @include {=readme}
    security.md 
    usage.md 
    example.md 
    sample.md 
    macros.md 
    help.md ?>

<? @exec ./sbin/apidocs ?>
<? @include {=readme} license.md links.md ?>

Using the command:

mkcat doc/readme.md | mkpi | mkmsg | mkref | mkabs | mkout > README.md

Macros

The default processing instruction grammar includes functions for including markdown documents, executing commands and more.

Grammar

These macros are defined by the default grammar.

@include

Include one or more markdown documents into the AST stream:

<? @include intro.md install.md license.md ?>

Processing instructions in included files are also executed, paths are resolved relative to the owner document when a file is available.

You can specify a path to include from using the tag value:

<? @include {=path/to/folder} intro.md install.md license.md ?>

Use this as shorthand when all the files to include are in the same directory.

If you specify a directory to include this implementation will look for index.md within the directory:

<? @include path/to/folder ?>

Resolves to path/to/folder/index.md relative to the file containing the instruction.

Note that file paths passed to this macro cannot include whitespace.

@exec

Execute a command and parse the result into the AST stream:

<? @exec pwd ?>

To capture the stderr stream use the stderr keyword before the command:

<? @exec stderr pwd ?>

An error is reported when a command fails, to include the output of a command with a non-zero exit code use the @exec! tag:

<? @exec! stderr pwd ?>

Commands may contain newlines they are removed before execution:

<?
  @exec ls -la
          lib
          dist
          test
?>

To wrap the output in a fenced code block use a type:

<? @exec {javascript} cat index.js ?>

@source

Load a file and parse it as markdown or wrap it in a fenced code block, unlike the @include macro processing instructions are not executed.

Parse a markdown file into the AST stream but do not execute processing instructions:

<? @source file.md ?>

Load a file into a fenced code block:

<? @source {javascript} index.js ?>

Sometimes it is useful to perform a string replacement on the sourced file, this is particularly helpful when you want to include a usage example that can be run directly but show the final package name.

To do so specify a string substitution in the form s/{regexp}/{replace}/gimy as a value, for example:

<? @source {javascript=s/\.\.\/index/mkpi/gm} usage.js ?>

Will replace all occurences of ../index with mkpi in usage.js before the file is parsed into the stream.

@macro

Defines a macro function body; use this for application specific logic.

Return a value to inject some information into the stream:

<?
  @macro return require('./package.json').name;
?>

Or wrap the result in a fenced code block:

<?
  @macro {shell} return require('./package.json').scripts.test;
?>

For asynchronous operations you can callback with a string to write to the stream:

<?
  @macro cb(null, '*emph*');
?>

See the macro api docs for more detail.

Custom Macros

Create a vanilla object if you wish to discard the default grammar macros:

var mkpi = require('mkpi')
  , grammar = {}
  , id = 'custom-macro';

grammar[id] = function(cb) {
  // implement macro logic
  cb();
}

mkpi({grammar: grammar});

You macro function will then be executed when the <? @custom-macro ?> processing instruction is encountered.

To extend the existing grammar with a custom macro function use:

var mkpi = require('mkpi')
  , grammar = require('mkpi/lib/grammar')
  , id = 'custom-macro';

grammar[id] = function(cb) {
  // implement macro logic
  cb();
}

mkpi({grammar: grammar});

Macro Functions

A macro function accepts a single argument cb which must be invoked when processing is complete, an Error may be passed to the callback.

They access all pertinent information via this, for example:

function(cb) {
  var tag = this.tag;
  console.error(tag.name);
  cb();
}

Note the exception that @macro function body definitions that return a value other than undefined should not call the callback.

See the grammar api docs for more information.

Help

Usage: mkpi [options]

  Processing instruction macros.

Options
  -s, --safe              Disable the @exec and @macro directives
  -p, --preserve          Do not remove processing instructions
  -h, --help              Display help and exit
  --version               Print the version and exit

[email protected]

API

pi

pi([opts][, cb])

Execute processing instructions found in the AST.

Instructions are removed from the AST by default, use preserve to keep them in the output.

When no input and no output are specified the parser stream is returned and cb is ignored.

Returns an output stream.

• opts Object processing options.
• cb Function callback function.

Options

• input Readable input stream.
• output Writable output stream.
• grammar Object grammar macro functions.
• preserve Boolean=false keep processing instructions in the AST.
• safe Boolean=false disable command and code execution.

Grammar

Default map of tag names to grammar macro functions.

A macro function has the signature function(cb), it should always invoke the callback and may pass an error.

It can access the following fields via this:

• writer: output stream, write AST nodes to this stream.
• comment: parsed processing instruction comment.
• tag: the tag that fired the macro function.
• parser: markdown parser (parser.parse() to convert strings to nodes).
• grammar: grammar document, map of tag identifiers to macro functions.
• preserve: whether to preserve the processing instructions.

exec

exec(cb)

Run an external command, newlines are removed from the command so it may span multiple lines.

<? @exec pwd ?>

To capture the stderr stream set stderr before the command:

<? @exec stderr pwd ?>

By default an error is reported if the command fails, to include the output when a command returns a non-zero exit code use the @exec! tag:

<? @exec! pwd ?>

To wrap a result in a fenced code block specify a type:

<? @exec {javascript} cat index.js ?>

If you want the result in a fenced code block with no info string use:

<? @exec {} cat index.js ?>

• cb Function callback function.

include

include(cb)

Include one or more markdown files into the AST, processing instructions in included files are executed.

<? @include intro.md install.md ?>

If a type is given it is a relative or absolute path to include from:

<? @include {path/to/folder} intro.md install.md ?>

Include files are resolved relative to the including file when file data is available (mkcat file.md), but when no file data is available, for example from stdin (cat file.md | mkcat), then files are resolved relative to the current working directory.

• cb Function callback function.

macro

macro(cb)

Accepts a function body, compiles it and executes the function.

Use this for inline application-specific logic.

The function is assumed to be a standard macro function implementation that accepts the arguments:

• cb: callback function to invoke when not returning a value.

It is also passed an additional non-standard argument:

• require: alias to require files relative to the cwd.

If the function returns a value other than undefined the result is parsed as markdown and written to the stream and and control flow is returned (as if cb was invoked automatically).

<? @macro return require('package.json').name; ?>

Otherwise the macro must invoke the callback function and should pass an optional error and result string to the callback:

<? @macro cb(null, '*emph*'); ?>

• cb Function callback function.

Parser

new Parser([opts])

Finds processing instructions in the stream, parses those found with mkparse and invokes a macro function if it exists for a tag in the parsed processing instruction.

• opts Object processing options.

Options

• preserve Boolean=false keep processing instructions.
• safe Boolean=false disable command and code execution.

replace

replace(str)

Parse a substitution definition in the form s/{regexp}/{string}/gimy.

When the str can be parsed the returned object includes:

• regexp: RegExp compiled pattern.
• replace: String replacement string.
• flags: String regexp flags.

If it cannot be parsed null is returned.

Returns replacement object or null.

• str String substitution definition.

Throws

• SyntaxError if the regexp pattern is malformed.

source

source(cb)

Load and parse a file as markdown without executing processing instructions or wrap the file in a fenced code block.

<? @source file.md ?>
<? @source {javascript} index.js ?>

• cb Function callback function.

License

MIT

Created by mkdoc on April 18, 2016