n3-fork

v1.16.3

Published

2 years ago

Lightning fast, asynchronous, streaming Turtle / N3 / RDF library.

Downloads

0High
0Medium
0Low

egordm

turtle rdf n3 streaming asynchronous

Lightning fast, asynchronous, streaming RDF for JavaScript

The N3.js library is an implementation of the RDF.js low-level specification that lets you handle RDF in JavaScript easily. It offers:

Parsing triples/quads from Turtle, TriG, N-Triples, N-Quads, RDF* and Notation3 (N3)
Writing triples/quads to Turtle, TriG, N-Triples, N-Quads and RDF*
Storage of triples/quads in memory

Parsing and writing is:

🎛 asynchronous – triples arrive as soon as possible
🚰 streaming – streams are parsed as data comes in, so you can parse files larger than memory
⚡️ fast – triples are flying out at high speeds

Installation

For Node.js, N3.js comes as an npm package.

$ npm install n3

const N3 = require('n3');

N3.js seamlessly works in browsers via webpack or browserify. If you're unfamiliar with these tools, you can read webpack: Creating a Bundle – getting started or Introduction to browserify. You will need to create a "UMD bundle" and supply a name (e.g. with the -s N3 option in browserify).

You can also load it via CDN:

<script src="https://unpkg.com/n3/browser/n3.min.js"></script>

Creating triples/quads

N3.js follows the RDF.js low-level specification.

N3.DataFactory will give you the factory functions to create triples and quads:

const { DataFactory } = N3;
const { namedNode, literal, defaultGraph, quad } = DataFactory;
const myQuad = quad(
  namedNode('https://ruben.verborgh.org/profile/#me'),
  namedNode('http://xmlns.com/foaf/0.1/givenName'),
  literal('Ruben', 'en'),
  defaultGraph(),
);
console.log(myQuad.termType);              // Quad
console.log(myQuad.value);                 // ''
console.log(myQuad.subject.value);         // https://ruben.verborgh.org/profile/#me
console.log(myQuad.object.value);          // Ruben
console.log(myQuad.object.datatype.value); // http://www.w3.org/1999/02/22-rdf-syntax-ns#langString
console.log(myQuad.object.language);       // en

In the rest of this document, we will treat “triples” and “quads” equally: we assume that a quad is simply a triple in a named or default graph.

Parsing

From an RDF document to quads

N3.Parser transforms Turtle, TriG, N-Triples, or N-Quads document into quads through a callback:

const parser = new N3.Parser();
parser.parse(
  `PREFIX c: <http://example.org/cartoons#>
   c:Tom a c:Cat.
   c:Jerry a c:Mouse;
           c:smarterThan c:Tom.`,
  (error, quad, prefixes) => {
    if (quad)
      console.log(quad);
    else
      console.log("# That's all, folks!", prefixes);
  });

The callback's first argument is an optional error value, the second is a quad. If there are no more quads, the callback is invoked one last time with null for quad and a hash of prefixes as third argument. Pass a second callback to parse to retrieve prefixes as they are read. If no callbacks are provided, parsing happens synchronously.

By default, N3.Parser parses a permissive superset of Turtle, TriG, N-Triples, and N-Quads. For strict compatibility with any of those languages, pass a format argument upon creation:

const parser1 = new N3.Parser({ format: 'N-Triples' });
const parser2 = new N3.Parser({ format: 'application/trig' });

Notation3 (N3) is supported only through the format argument:

const parser3 = new N3.Parser({ format: 'N3' });
const parser4 = new N3.Parser({ format: 'Notation3' });
const parser5 = new N3.Parser({ format: 'text/n3' });

It is possible to provide the base IRI of the document that you want to parse. This is done by passing a baseIRI argument upon creation:

const parser = new N3.Parser({ baseIRI: 'http://example.org/' });

By default, N3.Parser will prefix blank node labels with a b{digit}_ prefix. This is done to prevent collisions of unrelated blank nodes having identical labels. The blankNodePrefix constructor argument can be used to modify the prefix or, if set to an empty string, completely disable prefixing:

const parser = new N3.Parser({ blankNodePrefix: '' });

From an RDF stream to quads

N3.Parser can parse Node.js streams as they grow, returning quads as soon as they're ready.

const parser = new N3.Parser(),
      rdfStream = fs.createReadStream('cartoons.ttl');
parser.parse(rdfStream, console.log);

N3.StreamParser is a Node.js stream and RDF.js Sink implementation. This solution is ideal if your consumer is slower, since source data is only read when the consumer is ready.

const streamParser = new N3.StreamParser(),
      rdfStream = fs.createReadStream('cartoons.ttl');
rdfStream.pipe(streamParser);
streamParser.pipe(new SlowConsumer());

function SlowConsumer() {
  const writer = new require('stream').Writable({ objectMode: true });
  writer._write = (quad, encoding, done) => {
    console.log(quad);
    setTimeout(done, 1000);
  };
  return writer;
}

A dedicated prefix event signals every prefix with prefix and term arguments.

Writing

From quads to a string

N3.Writer serializes quads as an RDF document. Write quads through addQuad.

const writer = new N3.Writer({ prefixes: { c: 'http://example.org/cartoons#' } });
writer.addQuad(
  namedNode('http://example.org/cartoons#Tom'),
  namedNode('http://www.w3.org/1999/02/22-rdf-syntax-ns#type'),
  namedNode('http://example.org/cartoons#Cat')
);
writer.addQuad(quad(
  namedNode('http://example.org/cartoons#Tom'),
  namedNode('http://example.org/cartoons#name'),
  literal('Tom')
));
writer.end((error, result) => console.log(result));

By default, N3.Writer writes Turtle (or TriG if some quads are in a named graph). To write N-Triples (or N-Quads) instead, pass a format argument upon creation:

const writer1 = new N3.Writer({ format: 'N-Triples' });
const writer2 = new N3.Writer({ format: 'application/trig' });

From quads to an RDF stream

N3.Writer can also write quads to a Node.js stream.

const writer = new N3.Writer(process.stdout, { end: false, prefixes: { c: 'http://example.org/cartoons#' } });
writer.addQuad(
  namedNode('http://example.org/cartoons#Tom'),
  namedNode('http://www.w3.org/1999/02/22-rdf-syntax-ns#type'),
  namedNode('http://example.org/cartoons#Cat')
);
writer.addQuad(quad(
  namedNode('http://example.org/cartoons#Tom'),
  namedNode('http://example.org/cartoons#name'),
  literal('Tom')
));
writer.end();

From a quad stream to an RDF stream

N3.StreamWriter is a Node.js stream and RDF.js Sink implementation.

const streamParser = new N3.StreamParser(),
      inputStream = fs.createReadStream('cartoons.ttl'),
      streamWriter = new N3.StreamWriter({ prefixes: { c: 'http://example.org/cartoons#' } });
inputStream.pipe(streamParser);
streamParser.pipe(streamWriter);
streamWriter.pipe(process.stdout);

Blank nodes and lists

You might want to use the […] and list (…) notations of Turtle and TriG. However, a streaming writer cannot create these automatically: the shorthand notations are only possible if blank nodes or list heads are not used later on, which can only be determined conclusively at the end of the stream.

The blank and list functions allow you to create them manually instead:

const writer = new N3.Writer({ prefixes: { c: 'http://example.org/cartoons#',
                                       foaf: 'http://xmlns.com/foaf/0.1/' } });
writer.addQuad(
  writer.blank(
    namedNode('http://xmlns.com/foaf/0.1/givenName'),
    literal('Tom', 'en')),
  namedNode('http://www.w3.org/1999/02/22-rdf-syntax-ns#type'),
  namedNode('http://example.org/cartoons#Cat')
);
writer.addQuad(quad(
  namedNode('http://example.org/cartoons#Jerry'),
  namedNode('http://xmlns.com/foaf/0.1/knows'),
  writer.blank([{
    predicate: namedNode('http://www.w3.org/1999/02/22-rdf-syntax-ns#type'),
    object:    namedNode('http://example.org/cartoons#Cat'),
  },{
    predicate: namedNode('http://xmlns.com/foaf/0.1/givenName'),
    object:    literal('Tom', 'en'),
  }])
));
writer.addQuad(
  namedNode('http://example.org/cartoons#Mammy'),
  namedNode('http://example.org/cartoons#hasPets'),
  writer.list([
    namedNode('http://example.org/cartoons#Tom'),
    namedNode('http://example.org/cartoons#Jerry'),
  ])
);
writer.end((error, result) => console.log(result));

Storing

N3.Store allows you to store triples in memory and find them fast.

In this example, we create a new store and add the triples :Pluto a :Dog. and :Mickey a :Mouse. Then, we find triples with :Mickey as subject.

const store = new N3.Store();
store.add(
  namedNode('http://ex.org/Pluto'),
  namedNode('http://ex.org/type'),
  namedNode('http://ex.org/Dog')
);
store.add(
  namedNode('http://ex.org/Mickey'),
  namedNode('http://ex.org/type'),
  namedNode('http://ex.org/Mouse')
);

// Retrieve all quads
for (const quad of store)
  console.log(quad);
// Retrieve Mickey's quads
for (const quad of store.match(namedNode('http://ex.org/Mickey'), null, null))
  console.log(quad);

`DatasetCore` Interface

This store adheres to the DatasetCore interface which exposes the following properties

Attributes:

size — A non-negative integer that specifies the number of quads in the set.

Methods:

add — Adds the specified quad to the dataset. Existing quads, as defined in Quad.equals, will be ignored.
delete — Removes the specified quad from the dataset.
has — Determines whether a dataset includes a certain quad.
match — Returns a new dataset that is comprised of all quads in the current instance matching the given arguments.
[Symbol.iterator] — Implements the iterator protocol to allow iteration over all quads in the dataset as in the example above.

Addition and deletion of quads

The store provides the following manipulation methods in addition to implementing the standard DatasetCore Interface (documentation):

addQuad to insert one quad
addQuads to insert an array of quads
removeQuad to remove one quad
removeQuads to remove an array of quads
remove to remove a stream of quads
removeMatches to remove all quads matching the given pattern
deleteGraph to remove all quads with the given graph
createBlankNode returns an unused blank node identifier

Searching quads or entities

The store provides the following search methods (documentation):

readQuads returns a generator of quads matching the given pattern
getQuads returns an array of quads matching the given pattern
match returns a stream of quads matching the given pattern
countQuads counts the number of quads matching the given pattern
forEach executes a callback on all matching quads
every returns whether a callback on matching quads always returns true
some returns whether a callback on matching quads returns true at least once
getSubjects returns an array of unique subjects occurring in matching quads
forSubjects executes a callback on unique subjects occurring in matching quads
getPredicates returns an array of unique predicates occurring in matching quad
forPredicates executes a callback on unique predicates occurring in matching quads
getObjects returns an array of unique objects occurring in matching quad
forObjects executes a callback on unique objects occurring in matching quads
getGraphs returns an array of unique graphs occurring in matching quad
forGraphs executes a callback on unique graphs occurring in matching quads

Compatibility

Format specifications

The N3.js parser and writer is fully compatible with the following W3C specifications:

In addition, the N3.js parser also supports Notation3 (N3) (no official specification yet).

The N3.js parser and writer are also fully compatible with the RDF* variants of the W3C specifications.

The default mode is permissive and allows a mixture of different syntaxes, including RDF*. Pass a format option to the constructor with the name or MIME type of a format for strict, fault-intolerant behavior. If a format string contains star or * (e.g., turtlestar or TriG*), RDF* support for that format will be enabled.

Interface specifications

The N3.js submodules are compatible with the following RDF.js interfaces:

N3.DataFactory implements DataFactory
- the terms it creates implement Term and one of NamedNode, BlankNode, Literal, Variable, DefaultGraph
- the triples/quads it creates implement Term, Triple and Quad
N3.StreamParser implements Stream and Sink
N3.StreamWriter implements Stream and Sink
N3.Store implements Store Source Sink DatasetCore

License and contributions

The N3.js library is copyrighted by Ruben Verborgh and released under the MIT License.

Contributions are welcome, and bug reports or pull requests are always helpful. If you plan to implement a larger feature, it's best to contact me first.