ts-antlr4-scanner

A scanner for antlr4-based lexers.

Features

• Lookahead support
• Lookback support
• Advance to position
• Advance to token type

Installation

yarn add ts-antlr4-scanner
# or
npm install ts-antlr4-scanner

Usage

import { CommonTokenStream, ANTLRInputStream } from 'antlr4ts'
import { MySQLLexer } from 'ts-mysql-parser'
import { Scanner } from 'ts-antlr4-scanner'

const text = 'SELECT * FROM users'

const inputStream = new ANTLRInputStream(text)
const lexer = new MySQLLexer(inputStream) // Or any antlr4 lexer
const tokenStream = new CommonTokenStream(lexer)
const scanner = new Scanner(tokenStream)

// Move to character L of SELECT
scanner.advanceToPosition(3)

const tokenText = scanner.tokenText()
console.log(tokenText) // SELECT

API

new Scanner(tokenStream)

Creates a new instance of Scanner with the given token stream.

.next()

Advances to the next token.

scanner.next()

.previous()

Returns to the previous token.

scanner.previous()

.advanceToPosition(offset)

Advances to the token that covers the given zero-based offset.

scanner.advanceToPosition(4)

.advanceToType(type)

Advances to the next token with the given lexical type.

scanner.advanceToType(14)

.skipTokenSequence(sequence)

Steps over a number of tokens and positions.

scanner.skipTokenSequence([1, 5, 8])

.lookAhead()

Returns the type of the next token without changing the internal state.

scanner.lookAhead()

.lookBack()

Look back in the stream (physical order) what was before the current token, without modifying the current position.

scanner.lookBack()

.reset()

Resets the walker to be at the original location.

scanner.reset()

.push()

Store the current node on the stack, so we can easily come back when needed.

scanner.push()

.pop()

Returns to the location at the top of the token stack (if any).

scanner.pop()

.tokenText()

Returns the textual expression of the current token.

scanner.tokenText()

.tokenType()

Returns the type of the current token.

scanner.tokenType()

.tokenIndex()

Returns the (zero-based) index of the current token within the input.

scanner.tokenIndex()

.tokenLine()

Returns the (one-based) line number of the token.

scanner.tokenLine()

.tokenStart()

Returns the offset of the current token in its source string.

scanner.tokenStart()

.tokenLength()

Returns the length of the current token in bytes.

scanner.tokenLength()

.tokenChannel()

Returns the channel of the current token.

scanner.tokenChannel()

.tokenSubText()

Returns all the input text from the current token to the end.

scanner.tokenSubText()

Related

• ts-mysql-parser - A standalone, grammar-complete MySQL parser
• ts-mysql-analyzer - A MySQL query analyzer
• ts-mysql-schema - A schema extractor for MySQL
• ts-mysql-uri - Parse a MySQL connection URI

License

MIT

stevenmiller888.github.io  ·  GitHub @stevenmiller888  ·  Twitter @stevenmiller888