json-lexer

This is a JSON lexer based on the implementation in json3. It can split a JSON String into a list of annotated tokens. It will list whitespace as well, so it can used in-place editing of JSON documents.

Note that this doesn't check the validity of your JSON, so it will tokenize something like "token"}: happily.

var lexer = require('json-lexer')
lexer('{"hello": 1.0}')
// results in
[ { type: 'punctuator', value: '{', raw: '{' },
  { type: 'string', value: 'hello', raw: '"hello"' },
  { type: 'punctuator', value: ':', raw: ':' },
  { type: 'whitespace', value: ' ', raw: ' ' },
  { type: 'number', value: 1, raw: '1.0' },
  { type: 'punctuator', value: '}', raw: '}' } ]

types

whitespace

Allowed white space between the actual relevant tokens.

punctuator

The characters surrounding your data: {, }, [, ], : and ,

string

A JSON string "hi", not that the value will be the parsed String without "

number

A JSON number, like 1, -1 or 1e1000. The value will be the parsed number.

literal

One of the allowed literals true, false and null. The value will be the specific JS literal.