npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@asgerf/dts-tree-sitter

v0.1.0

Published

Generates TypeScript .d.ts files for using tree-sitter grammars.

Downloads

74

Readme

dts-tree-sitter

dts-tree-sitter generates TypeScript .d.ts files for interacting the AST from a given tree-sitter grammar.

Usage

npm i @asgerf/dts-tree-sitter

npx @asgerf/dts-tree-sitter INPUT > OUTPUT.d.ts

where INPUT is used to locate a node-types.json file in one of the following locations:

  • ${INPUT}
  • ${INPUT}/node-types.json
  • ${INPUT}/src/node-types.json
  • node_modules/${INPUT}/src/node-types.json

Example

The tree-sitter-javascript grammar can be compiled like this:

npm i tree-sitter-javascript
npx @asgerf/dts-tree-sitter tree-sitter-javascript > generated.d.ts

In the resulting grammar, two of the node types look like this:

export interface ClassDeclarationNode extends SyntaxNodeBase {
  type: SyntaxType.ClassDeclaration;
  bodyNode: ClassBodyNode;
  decoratorNodes?: DecoratorNode[];
  nameNode: IdentifierNode;
}

export interface ClassBodyNode extends SyntaxNodeBase {
  type: SyntaxType.ClassBody;
  memberNodes?: (MethodDefinitionNode | PublicFieldDefinitionNode)[];
}

This can be used like this (see full example):

import * as g from "./generated";

function getMemberNames(node: g.ClassDeclarationNode) {
    let result = [];
    for (let member of node.bodyNode.memberNodes) {
        if (member.type === g.SyntaxType.MethodDefinition) {
            result.push(member.nameNode.text);
        } else {
            result.push(member.propertyNode.text);
        }
    }
    return result;
}

Observe TypeScript do its magic: the type check in the if promotes the type of member to a MethodDefinitionNode in the 'then' branch, and to PublicFieldDefinitionNode in the 'else' branch.

Typed Tree Cursors

Tree sitter's TreeCursor allows fast traversal of an AST, and has two properties with correlated types: nodeType, and currentNode. Once you've checked nodeType, it's annoying to have to cast currentNode to the correponding type right afterwards:

if (cursor.nodeType === g.SyntaxType.Function) {
  let node = cursor.currentNode as g.Function; // annoying cast
}

There's another way, which is handy in large switches: Cast the cursor itself to a TypedTreeCursor before switching on nodeType. Then the guarded use of currentNode has the expected type. For example:

function printDeclaredNames() {
    let cursor = tree.walk();
    do {
        const c = cursor as g.TypedTreeCursor;
        switch (c.nodeType) {
            case g.SyntaxType.ClassDeclaration:
            case g.SyntaxType.FunctionDeclaration:
            case g.SyntaxType.VariableDeclarator: {
                let node = c.currentNode;
                console.log(node.nameNode.text);
                break;
            }
        }
    } while(gotoPreorderSucc(cursor));
}
  • node gets the type ClassDeclarationNode | FunctionDeclarationNode | VariableDeclaratorNode.
  • This allows safe access to node.nameNode, since each of those types have a name field.
  • We don't pay the cost of invoking currentNode for other types of nodes.

Trouble-shooting

I get an error about "excessive stack depth" during compilation

This happens if you compare types from the general tree-sitter.d.ts file with those from the generated .d.ts file. Every type from tree-sitter.d.ts has a stronger version in the generated file; make sure you don't mix and match.

I get UnnamedNode types in places where I don't expect them

This can happen if the grammar contains rules and literals with the same name. For example this grammar rule,

  func: $ => seq('func', $.name, $.body)

will produce a named node with type func, while the 'func' literal will produce an unnamed node with type func as well.

This means a check like node.type === 'func' is not an exact type check, and the type of node will only be restricted to FuncNode | UnnamedNode<'func'>. This is not a bug in the generated .d.ts file: there really are two kinds of nodes you need to handle after that check.

Some possible solutions are:

  • Change the grammar to avoid rules with the same name as a keyword.
  • Write the check as node.isNamed && node.type === 'func'.
  • Change the declared type of node from SyntaxNode to NamedNode.