npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@nbittich/rdfa-streaming-parser

v2.0.3-custom-html-extractor

Published

A fast and lightweight streaming RDFa parser

Downloads

1

Readme

RDFa Streaming Parser

Build status Coverage Status npm version

A fast and lightweight streaming and 100% spec-compliant RDFa 1.1 parser, with RDFJS representations of RDF terms, quads and triples.

The streaming nature allows triples to be emitted as soon as possible, and documents larger than memory to be parsed.

Installation

$ npm install rdfa-streaming-parser

or

$ yarn add rdfa-streaming-parser

This package also works out-of-the-box in browsers via tools such as webpack and browserify.

Require

import {RdfaParser} from "rdfa-streaming-parser";

or

const RdfaParser = require("rdfa-streaming-parser").RdfaParser;

Usage

RdfaParser is a Node Transform stream that takes in chunks of RDFa data, and outputs RDFJS-compliant quads.

It can be used to pipe streams to, or you can write strings into the parser directly.

While not required, it is advised to specify the profile of the parser by supplying a contentType or profile constructor option.

Print all parsed triples from a file to the console

const myParser = new RdfaParser({ baseIRI: 'https://www.rubensworks.net/', contentType: 'text/html' });

fs.createReadStream('index.html')
  .pipe(myParser)
  .on('data', console.log)
  .on('error', console.error)
  .on('end', () => console.log('All triples were parsed!'));

Manually write strings to the parser

const myParser = new RdfaParser({ baseIRI: 'https://www.rubensworks.net/', contentType: 'text/html' });

myParser
  .on('data', console.log)
  .on('error', console.error)
  .on('end', () => console.log('All triples were parsed!'));

myParser.write('<?xml version="1.0"?>');
myParser.write(`<!DOCTYPE html>
<html>

<head prefix="foaf: http://xmlns.com/foaf/0.1/">`);
myParser.write(`<link rel="foaf:primaryTopic foaf:maker" href="https://www.rubensworks.net/#me" />`);
myParser.write(`</head>`);
myParser.write(`<body>`);
myParser.write(`</body>`);
myParser.write(`</html>`);
myParser.end();

Import streams

This parser implements the RDFJS Sink interface, which makes it possible to alternatively parse streams using the import method.

const myParser = new RdfaParser({ baseIRI: 'https://www.rubensworks.net/', contentType: 'text/html' });

const myTextStream = fs.createReadStream('index.html');

myParser.import(myTextStream)
  .on('data', console.log)
  .on('error', console.error)
  .on('end', () => console.log('All triples were parsed!'));

Configuration

Optionally, the following parameters can be set in the RdfaParser constructor:

  • dataFactory: A custom RDFJS DataFactory to construct terms and triples. (Default: require('@rdfjs/data-model'))
  • baseIRI: An initial default base IRI. (Default: '')
  • language: A default language for string literals. (Default: '')
  • vocab: The initial vocabulary. (Default: '')
  • defaultGraph: The default graph for constructing quads. (Default: defaultGraph())
  • features: A hash of features that should be enabled. Defaults to the features defined by the profile. (Default: all features enabled)
  • profile: The RDFa profile to use. (Default: profile with all features enabled)
  • contentType: The content type of the document that should be parsed. This can be used as an alternative to the 'profile' option. (Default: profile with all features enabled)
  • htmlParseListener: An optional listener for the internal HTML parse events, should implement IHtmlParseListener (Default: null)
new RdfaParser({
  dataFactory: require('@rdfjs/data-model'),
  baseIRI: 'http://example.org/',
  language: 'en-us',
  vocab: 'http://example.org/myvocab',
  defaultGraph: namedNode('http://example.org/graph'),
  features: { langAttribute: true },
  profile: 'html',
  htmlParseListener: new MyHtmlListener(),
});

Profiles

On top of RDFa Core 1.1, there are a few RDFa variants that add specific sets of rules, which are all supported in this library:

  • HTML+RDFa 1.1: Internally identified as the 'html' profile with 'text/html' as content type.
  • XHTML+RDFa 1.1: Internally identified as the 'xhtml' profile with 'application/xhtml+xml' as content type.
  • SVG Tiny 1.2: Internally identified as the 'xml' profile with 'application/xml', 'text/xml' and 'image/svg+xml' as content types.

This library offers three different ways to define the RDFa profile or setting features:

  • Content type: Passing a content type such as 'text/html' to the contentType option in the constructor.
  • Profile string: Passing '', 'core', 'html', 'xhtml' or 'svg' to the profile option in the constructor.
  • Features object: A custom combination of features can be defined by passing a features option in the constructor.

The table below lists all possible RDFa features and in what profile they are available:

| Feature | Core | HTML | XHTML | XML | Description | | -------------------------------- | ---- |----- | ----- | --- | ----------- | | baseTag | | ✓ | ✓ | | If the baseIRI can be set via the <base> tag. | | xmlBase | | | | ✓ | If the baseIRI can be set via the xml:base attribute. | | langAttribute | | ✓ | ✓ | ✓ | If the language can be set via the language attribute. | | onlyAllowUriRelRevIfProperty | ✓ | ✓ | ✓ | | If non-CURIE and non-URI rel and rev have to be ignored if property is present. | | inheritSubjectInHeadBody | | ✓ | ✓ | | If the new subject can be inherited from the parent object if we're inside <head> or <body> if the resource defines no new subject. | | datetimeAttribute | | ✓ | ✓ | ✓ | If the datetime attribute must be interpreted as datetimes. | | timeTag | | ✓ | ✓ | ✓ | If the <time> tag contents should be interpreted as datetimes. | | htmlDatatype | | ✓ | ✓ | | If rdf:HTML as datatype should cause tag contents to be serialized to text. | | copyRdfaPatterns | ✓ | ✓ | ✓ | | If rdfa:copy property links can refer to rdfa:Pattern's for copying. | | xmlnsPrefixMappings | ✓ | ✓ | ✓ | ✓ | If prefixes should be extracted from xmlns. | | skipHandlingXmlLiteralChildren | | | | | If children of rdf:XMLLiteral should not be handled as RDFa anymore. This is not part of the RDFa spec. | | xhtmlInitialContext | | | ✓ | | If the XHTML initial context should be included in the initial prefixes. | | roleAttribute | | ✓ | ✓ | ✓ | If the role attribute should be handled. |

How it works

This tool makes use of the highly performant htmlparser2 library for parsing HTML in a streaming way. It listens to tag-events, and maintains the required tag metadata in a stack-based datastructure, which can then be emitted as triples as soon as possible.

Our algorithm closely resembles the suggested processing sequence, with a few minor changes to make it work in a streaming way.

If you want to make use of a different HTML/XML parser, you can create a regular instance of RdfaParser, and just call the following methods yourself directly:

  • onTagOpen(name: string, attributes: {[s: string]: string})
  • onText(data: string)
  • onTagClose()

Specification Compliance

This parser passes all tests from the RDFa 1.1 test suite. More specifically, the following manifests are explicitly tested:

  • HTML+RDFa 1.1 (HTML4)
  • HTML+RDFa 1.1 (HTML5)
  • HTML+RDFa 1.1 (XHTML5)
  • SVGTiny+RDFa 1.1
  • XHTML+RDFa 1.1
  • XML+RDFa 1.1

The following optional features for RDFa processors are supported:

The following optional features for RDFa processors are not supported (yet):

License

This software is written by Ruben Taelman.

This code is released under the MIT license.