skos-tools - SKOS Model and utility methods for importing SKOS into EntryStore

A library for working with SKOS and importing / updating a terminology in an EntryStore instance.

Installing

npm install @entryscape/skos-tools

Or:

yarn add @entryscape/skos-tools

You can import the library either as ES6 Modules (import) or via CommonJS (require), i.e.:

import { Concept, ConceptScheme } from '@entryscape/skos-tools';

Or:

const { Concept, ConceptScheme } = require('@entryscape/skos-tools');

In the example below we will use the CommonJS style (require).

Usage

The library have four basic capabilities:

1. Model - interfaces for working with Concepts in ConceptSchemes without having to touch RDF.
2. Extract - extract concepts and conceptSchemes from RDF and add them to a model.
3. Store - creating, clearing and updating a SKOS model in an EntryStore instance.

SKOS model

The model provides an interface for working with SKOS independent of the RDF expression. There are two main classes, ConceptScheme and Concept. The methods correspond closely to the properties available on these classes in the SKOS vocabulary. For instance there are methods called broader, narrower, prefLabel, definition, relation, exactMatch to mention a few.

Each concept, as well as the concept scheme instance has its own RDF graph. This is useful when there is a need to store the graphs separately, see section on persisting the model into EntryStore.

Example of how to create a model

The following code shows how to create a concept scheme with two concepts programmatically.

const { ConceptScheme } = require('@entryscape/skos-tools');
const cs = new ConceptScheme("http://example.com", undefined, "en");
cs.title("Example terminology");
const c1 = cs.createConcept("http://example.com/c1");
cs.prefLabel("Concept 1");
const c2 = cs.createConcept("http://example.com/c2");
c2.prefLabel("Concept 2");
c2.prefLabel("Begrepp 2", "sv");
c2.broader(c1.getURI());
cs.ensureTopLevelConceptsExpressed();

Note that we set the default language in the constructor of the Concept Scheme and that is inherited in title and prefLabel (unless it is explicitly overridden like for the swedish prefLabel of concept 2).

The method call ensureTopLevelConceptsExpressed is to provide the skos:hasTopConcept relation is expressed only to the concepts that have no broader concept.

Extracting - how to extract a terminology from a single RDF file

The extract mechanism divides a larger graph into smaller graph chunks and provides and imports them into the model.

const { Graph } = require('@entryscape/rdfjson');
const { extractFromGraph } = require('@entryscape/skos-tools');
const graph = ... // Load or create a graph somehow
const cs = fromGraph(graph);

Preparing for communicating with an EntryStore instance

Any communication with an EntryStore instance requires creating a EntryStore instance for the API endpoint and authenticating. A technicality is that node.js does not allow us to use await in the global scope. To solve this we use a trick and wrap the code in an async function that we execute immediately.

const { EntryStore } = require('@entryscape/entrystore-js');
(async () => {
  const es = new EntryStore("https://some.entrystoreinstance.com");
  await es.getAuth().login("testuser", "testtest");
  // Insert your code here
})();

Creating a context as a container for a terminology

Before you can import a terminology you need to have a container for it, in EntryStore this is called a context. You can create a terminology context via EntryScape and then clear it, like this:

const { removeInStore } = require('@entryscape/skos-tools');

// Code below inside an async and authenticated function with entrystore available
const contextId = '19'; // Assuming you created context 19 in EntryScape.
const context = entrystore.getContextById(contextId);
await removeInStore(context);

Or you can create it programmatically:

const { createTermsContext } = require('@entryscape/skos-tools');

// Code below inside an async and authenticated function with entrystore available
const context = await createTermsContext(entrystore);

The context will be created in parallel with a group for managing who has access to work with entries within the context. You can mark the context to be an enhanced terminology context by providing true as the second parameter as well as provide a title for the context and the group via the third parameter. Finally, you can also define a unique name to be used for both the context and it's associated group:

const context = await createTermsContext(entrystore, true, 'Name of xyz', 'xyz');

In the examples below we will assume that a suitable context have already been created.

Creating a terminology into an EntryStore instance

In the code below we assume we have a model (conceptScheme) for the Concept Scheme already and that we have an EntryStore instance which is already authenticated.

First we need to require some methods from this library.

const { importToStore, removeInStore, consoleLogger } = require('@entryscape/skos-tools');

Second, assuming we authenticated an EntryStore instance we do the actual import from a Concept Scheme model (out of scope how we got the model in this section).

const conceptScheme = ... // A conceptScheme instance with some concepts we retrieved somehow.
const context = es.getContextById("7");
await importToStore(conceptScheme, context, false);

Note 1, we cannot have more than one Concept Scheme in the same context (a kind of container in EntryStore). Hence, we might have to clear the context first:

const conceptScheme = ... // A conceptScheme instance with some concepts we retrieved somehow.
const context = es.getContextById("7");
await removeInContext(context);
await importToStore(conceptScheme, context, false);

Note 2, a terminology can be created in an enhanced mode where every concept has two metadata graphs instead of one. In this case the EntryScape UI considers the first graph to be read-only which makes it possible to keep a distinction between imported information and manually edited information. To import information into this read-only graph we simply change the third argument to true:

await importCS(conceptScheme, context, true);

Note 3, the import step can be run in a dry run mode by setting the fourth parameter:

await importCS(conceptScheme, context, false, true);

Note 4, it is possible to get logging by providing the fifth parameter, e.g. the predefined consoleLogger:

await importCS(conceptScheme, context, false, true, consoleLogger);

Updating an existing terminology in an EntryStore instance

To update a Concept Scheme we need to use the updateInStore method.

const { updateInStore } = require('@entryscape/skos-tools');

Now we can do the update:

const conceptScheme = ... // A conceptScheme instance with some concepts we retrieved somehow.
const context = es.getContextById("7");
const report = await updateInStore(conceptScheme, context);

The updateInStore will return a report that looks like the following example:

{
  treated: 10,   // Amount of concepts in the model
  unchanged: 6,  // Amount of concepts where no change was detected
  updated: 3,    // Amount of concepts where a change where detected
  added: 1,      // Amount of new concepts that did not exist before 
  removed: 1     // Amount of concepts that was removed from the context
}

Note 1, concepts will be identified to be the same based on their URIs, hence if a concept has changed its URI the concept with the old URI will be removed and a new concept will be added. (This is one possible interpretation of the report above where a new concept was added and an old concept was removed, to figure out if that is actually what happened a deeper comparison of the URIs and metadata would be necessary.)

Note 2, it is important to know that the updateInStore looks into the concepts in the current context and detects if it was created as an enhanced terminology or not. If it was created as enhanced all changes made to the terminology using the UI will be preserved (for concepts where updated, not if they were removed altogether). If it was not created as an enhanced terminology all changes made in the UI will be overwritten.

Note 3, just like the importToStore function the updateInStore function also supports the parameter dryRun and logger.

Testing the codebase

The test can be run by executing:

yarn test

Note that one of the test suites requires that you have provided testConfig.js with information on which EntryStore instance to communicate with, see testConfig_example.js. Note that this test suite takes around 80 seconds to run due to intentional being slow. This is to make sure the EntryStore instance have time to properly index new information before verifying that imported / updated concepts are correct. In a real setting (when importing / updating) these delays can be skipped.