Microdata to RDF Streaming Parser

A fast and lightweight streaming and 100% spec-compliant Microdata to RDF parser, with RDFJS representations of RDF terms, quads and triples.

The streaming nature allows triples to be emitted as soon as possible, and documents larger than memory to be parsed.

Installation

$ npm install microdata-rdf-streaming-parser

or

$ yarn add microdata-rdf-streaming-parser

This package also works out-of-the-box in browsers via tools such as webpack and browserify.

Require

import {MicrodataRdfParser} from "microdata-rdf-streaming-parser";

or

const MicrodataRdfParser = require("microdata-rdf-streaming-parser").MicrodataRdfParser;

Usage

MicrodataRdfParser is a Node Transform stream that takes in chunks of Microdata to RDF data, and outputs RDFJS-compliant quads.

It can be used to pipe streams to, or you can write strings into the parser directly.

Configuration

Optionally, the following parameters can be set in the MicrodataRdfParser constructor:

• dataFactory: A custom RDFJS DataFactory to construct terms and triples. (Default: require('@rdfjs/data-model'))
• baseIRI: An initial default base IRI. (Default: '')
• defaultGraph: The default graph for constructing quads. (Default: defaultGraph())
• htmlParseListener: An optional listener for the internal HTML parse events, should implement IHtmlParseListener (Default: null)
• xmlMode: If the parser should assume strict X(HT)ML documents. (Default: false)
• vocabRegistry: A vocabulary registry to define specific behaviour for given URI prefixes. (Default: contents of http://www.w3.org/ns/md)

new RdfaParser({
  dataFactory: require('@rdfjs/data-model'),
  baseIRI: 'http://example.org/',
  defaultGraph: namedNode('http://example.org/graph'),
  htmlParseListener: new MyHtmlListener(),
  xmlMode: true,
  vocabRegistry: {
    "http://schema.org/": {
      "properties": {
        "additionalType": {"subPropertyOf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#type"}
      }
    },
    "http://microformats.org/profile/hcard": {}
  },
});

How it works

This tool makes use of the highly performant htmlparser2 library for parsing HTML in a streaming way. It listens to tag-events, and maintains the required tag metadata in a stack-based datastructure, which can then be emitted as triples as soon as possible.

Our algorithm closely resembles the suggested algorithm for transforming Microdata to RDF, with a few changes to make it work in a streaming way.

If you want to make use of a different HTML/XML parser, you can create a regular instance of MicrodataRdfParser, and just call the following methods yourself directly:

• onTagOpen(name: string, attributes: {[s: string]: string})
• onText(data: string)
• onTagClose()

Specification Compliance

This parser passes all tests from the Microdata to RDF test suite.

License

This software is written by Ruben Taelman.

This code is released under the MIT license.