AST Generator for TypeScript

This tool generates complete definitions for an abstract syntax tree (AST) in the TypeScript language. It reads an ordinary TypeScript file as a specification file, and will automatically generate classes, contstructors, union types, visitors, and predicates based on this specification.

Features

• Mix your own code with code generated by tsastgen. TypeScript goes in, TypeScript comes out.
• Efficient type-checking and dispatching using TypeScript enums and discriminated unions.
• Generate typed reflection functions such as Node.getChildNodes() and isFoo()
• Generate typed factory functions that are able to automatically lift simple primitive values to entire nodes (experimental).
• Ability to generate typed links to the parent node that only lists node types that can actually be a parent.

Basic Usage

First you have to install the package:

npm install -g tsastgen

Now a binary called tsastgen should be available in your favorite shell. If not, check your PATH variable and that npm is properly configured.

Next, create a specification file. Here's an example:

calc-spec.ts

export interface AST {}

export interface Definition extends AST {
  name: string;
  expression: Expression;
}

export interface Expression extends AST {
  lazyEvaluatedResult?: number;
}

export interface ConstantExpression extends Expression {
  value: number;
}

export interface BinaryExpression extends Expression {
  left: Expression;
  right: Expression;
}

export interface SubtractExpression extends BinaryExpression {

}

export interface AddExpression extends BinaryExpression {

}

export interface MultiplyExpression extends BinaryExpression {

}

export interface DivideExpression extends BinaryExpression {

}

export type CommutativeExpression 
  = AddExpression
  | SubtractExpression
  | MultiplyExpression;

Now all you need to do is to run tsastgen and make sure it knows what the output file and the root node is.

tsastgen --with-root-node=AST calc-spec.ts:calc.ts

Read the example for a much more detailed explanation of how code is generated.

How to match certain generated AST nodes

Here's an example of how the generated code might be used:

import { Expression } from "./calc"

export function calculate(node: Expression): number {
  switch (node.kind) {
    case SyntaxKind.AddExpression:
       return node.left + node.right;
    case SyntaxKind.SubtractExpression:
      // and so on ...
    default:
      throw new Error(`I did not know how to process the given node.`);
  }
}

In the above example, due to the way in which the code is generated, the compiler automatically knows when certain fields are present.

Alternatively, you can use the generated AST predicates combined with an if-statement to prevent casting:

const node = generateANewNodeSomehow();

if (isDefinition(node)) {
  // The fields 'name' and 'expression' are now available.
}

No matter which style you use, you will almost never have to cast to another expression.

How to create new AST nodes

Creating nodes is also very easy:

import {
  createAddExpression,
  createConstantExpression,
} from "./calc";

const n1 = createConstantExpression(1);
const n2 = createConstantExpression(2);
const add = createAddExpression(n1, n2);

console.log(`The result of 1 + 2 is ${calculate(add)}`);

It is recommended to not use the new operator. Instead, use the wrapping createX function. The motivation is that in the future we might use a more efficient representation than a class, using createX-functions guarantees forward compatibility.

API

In the following documented signatures, Foo stands for an arbitrary node type generated by tsastgen and node for an instance of Foo. Syntax refers to the node type specified with --with-root-node and field to a specified field of Foo.

SyntaxKind

A large enumeration that contains all of your node types. You can access the kind of a node using the node.kind property documented below.

isSyntax(value)

Check whether value is an AST node. Any JavaScript value may be passed in. Use this function if you don't know anything about value and before using a function such as isFoo.

isFoo(node)

Check whether the given node is of the specific node type Foo. For performance reasons, you should only pass in valid node objects. Do not use this function to check whether any random JavaScript value is a node object.

node.kind

Get the SyntaxKind enumeration value that discriminates the type of this node from the other node types.

node.getChildNodes()

Returns an iterable (not an array) of all nodes that can be found in the fields of node.

node.parentNode

A typed reference to a node that is the parent of this node, or null if this node is the root node or if the parent has not been set.

node.field

A property to get the value of a specified field of node.

FooParent

A type alias that lists all node types that in theory can be a parent of Foo.

FooChild

A type alias that lists all node types that in theory can occur in the fields of Foo.

createFoo(...fields)

Create a new node object of the type Foo. The required fields go first after which the optional fields may be specified. Use your code editor's hint dialogs to see what field goes where.

If a field refers to a node that only contains one field, you may be able to just specify that field directly and createFoo will convert it into the desired node type for you.

CLI Options

tsastgen [input-file[:output-file]..] --with-root-node=<name>

input-file

The specification file that AST definitions will be generated from.

output-file

If present, the file where the transformed input-file must be written to.

If not present, the program will output the result to standard output.

--with-coercions

Enable experimental code generation of coercion statements in factory functions. Currently, due to limitations in the built-in type checking algorithm, this will not work for complex ASTs.

--with-root-node

The name of the node that serves as the root node of the abstract syntax tree.It will automatically be converted to a union type containing all possible AST node types.

If --with-root-node is not specified, tsastgen will search for a declaration named Syntax.

--with-parent-member

The name of the field that is used to refer to the parent node of a node type. If enabled, tsastgen will treat this node specially and inject/replace fully typed member fields with the name you provided.

If --with-parent-member is not specified, this feature is not enabled and your member will be passed through without any modifications.

Known Issues and Limitations

Coercions in factory functions will not always be generated because we only emulate a small subset the typing rule of TypeScript. If something cannot be checked, the tool will error or skip it. This is due to poor support of the TypeScript compiler to inspect the types of AST nodes in detail. Unfortunately, this appears to be by design, so this won't get fixed easily. If we were to cover all cases of TypeScript's AST, we would effectively have written a second compiler.

License

I chose to license this piece of software under the MIT license, in the hope that you may find it useful. You may freely use this generator in your own projects, but it is always nice if you can give a bit of credit.