@scrapiloo/regex-tokenizer
v0.1.2
Published
Minimal Regex Tokenizer.
Downloads
4
Readme
Regex Tokenizer
Minimal Regex Tokenizer.
Installation
# NPM
npm i @scrapiloo/regex-tokenizer
# Yarn
yarn add @scrapiloo/regex-tokenizer
Usage
To use the tokenizer, call the tokenize()
function with a regular expression as input.
import tokenize from "@scrapiloo/regex-tokenizer";
const tokens = tokenize("(.+)@(?!gmail.com)");
console.log(tokens);
The code above will return the following output:
[
Token [”(”] {
type: 'capture-group-perimeter',
value: '(',
scope: 'root capture-group'
},
Token [”.”] {
type: 'character',
value: '.',
scope: 'root capture-group'
},
Token [”+”] {
type: 'quantifier',
value: '+',
scope: 'root capture-group'
},
Token [”)”] {
type: 'capture-group-perimeter',
value: ')',
scope: 'root capture-group'
},
Token [”@”] {
type: 'character',
value: '@',
scope: 'root'
},
Token [”(”] {
type: 'lookaround-perimeter',
value: '(',
scope: 'root negative-lookaround'
},
Token [”?!”] {
type: 'lookaround-type',
value: '?!',
scope: 'root negative-lookaround'
},
Token [”g”] {
type: 'character',
value: 'g',
scope: 'root negative-lookaround'
},
Token [”m”] {
type: 'character',
value: 'm',
scope: 'root negative-lookaround'
},
Token [”a”] {
type: 'character',
value: 'a',
scope: 'root negative-lookaround'
},
Token [”i”] {
type: 'character',
value: 'i',
scope: 'root negative-lookaround'
},
Token [”l”] {
type: 'character',
value: 'l',
scope: 'root negative-lookaround'
},
Token [”.”] {
type: 'character',
value: '.',
scope: 'root negative-lookaround'
},
Token [”c”] {
type: 'character',
value: 'c',
scope: 'root negative-lookaround'
},
Token [”o”] {
type: 'character',
value: 'o',
scope: 'root negative-lookaround'
},
Token [”m”] {
type: 'character',
value: 'm',
scope: 'root negative-lookaround'
},
Token [”)”] {
type: 'lookaround-perimeter',
value: ')',
scope: 'root negative-lookaround'
}
]