Intent tools

Tools for processing intents from RASA JSONs to Facebook fast-text learning sets.

Because there is good GUI: Rasa NLU Trainer

CLI Usage

• Convert RASA json to fast-text learning set

$ intools jsonToText ./testData.json ./testData.txt

• Convert RASA json to fast-text learning set and multiply by entities

$ intools jsonToText -m ./testData.json ./testData.txt

• Make word vectors learning set from wiki XML export

$ intools wikiToText ./testData.xml ./testData.txt

API

Classes

Functions

MultiplicatorStream

Kind: global class

• MultiplicatorStream
  • new MultiplicatorStream()
  • new MultiplicatorStream(getVariants)

new MultiplicatorStream()

Multiplicates a learning set data with available entities information

Example

const path = require('path');
const { EntitiesFromJson, MultiplicatorStream, jsonToText } = require('intent-tools');

const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');

const entities = new main.EntitiesFromJson(from);

const pipeline = [
    new MultiplicatorStream((cat, word) => entities.getWordList(cat, word))
];

entities.loadEntities()
    .then(() => main.jsonToText(from, to, pipeline))
    .catch(e => console.error(e));

new MultiplicatorStream(getVariants)

| Param | Type | | --- | --- | | getVariants | function |

MultiplicatorStream

Kind: global class

• MultiplicatorStream
  • new MultiplicatorStream()
  • new MultiplicatorStream(getVariants)

new MultiplicatorStream()

Multiplicates a learning set data with available entities information

Example

const path = require('path');
const { EntitiesFromJson, MultiplicatorStream, jsonToText } = require('intent-tools');

const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');

const entities = new main.EntitiesFromJson(from);

const pipeline = [
    new MultiplicatorStream((cat, word) => entities.getWordList(cat, word))
];

entities.loadEntities()
    .then(() => main.jsonToText(from, to, pipeline))
    .catch(e => console.error(e));

new MultiplicatorStream(getVariants)

| Param | Type | | --- | --- | | getVariants | function |

Pipeline

Kind: global class

• Pipeline
  • new Pipeline()
  • .add(pipe) ⇒ this
  • .promise() ⇒ promise

new Pipeline()

Simple tool, which creates a Promise from pipeline of streams

pipeline.add(pipe) ⇒ this

Append a stream

Kind: instance method of Pipeline

| Param | Type | Description | | --- | --- | --- | | pipe | ReadableStream | Writable | the transform stream |

pipeline.promise() ⇒ promise

Get a promise

Kind: instance method of Pipeline

jsonToText(input, output, [pipeline], [mapFn]) ⇒ Promise

Create fast-text learning set from Rasa intents json

Kind: global function

| Param | Type | Default | Description | | --- | --- | --- | --- | | input | string | ReadableStream | | path of Rasa intent set or stream | | output | string | Writable | | path or stream to write fast-text learning set | | [pipeline] | Array | | array of transform streams to modify the learning set | | [mapFn] | function | | text normalizer function |

Example

const path = require('path');
const { jsonToText } = require('intent-tools');

const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');

main.jsonToText(from, to)
    .catch(e => console.error(e));

wikiToText(input, output, [mapFn]) ⇒ Promise

Create a pretrained word vectors learning set from Wikipedia XML dump

Kind: global function

| Param | Type | Default | Description | | --- | --- | --- | --- | | input | string | ReadableStream | | path of Rasa intent set or stream | | output | string | Writable | | path or stream to write fast-text learning set | | [mapFn] | function | | text normalizer function |

normalize(str)

Preserves only letters (with or withour diacritics) and makes everything lowercased

Kind: global function Returs: string

| Param | Type | Description | | --- | --- | --- | | str | string | input string |