Intent tools

Tools for processing intents from RASA JSONs to Facebook fast-text learning sets.

Because there is good GUI: Rasa NLU Trainer

CLI Usage

• Convert RASA json to fast-text learning set

$ intools jsonToText ./testData.json ./testData.txt

• Convert RASA json to fast-text learning set and multiply by entities

$ intools jsonToText -m ./testData.json ./testData.txt

• Make word vectors learning set from wiki XML export

$ intools wikiToText ./testData.xml ./testData.txt

---

API

Classes

MultiplicatorStream

MultiplicatorStream

Pipeline

Functions

jsonToText(input, output, [pipeline], [mapFn]) ⇒ Promise

Create fast-text learning set from Rasa intents json

wikiToText(input, output, [mapFn]) ⇒ Promise

Create a pretrained word vectors learning set from Wikipedia XML dump

normalize(str)

Preserves only letters (with or withour diacritics) and makes everything lowercased

MultiplicatorStream

Kind: global class

• MultiplicatorStream
  • new MultiplicatorStream()
  • new MultiplicatorStream(getVariants)

new MultiplicatorStream()

Multiplicates a learning set data with available entities information

Example

const path = require('path');
const { EntitiesFromJson, MultiplicatorStream, jsonToText } = require('intent-tools');

const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');

const entities = new main.EntitiesFromJson(from);

const pipeline = [
    new MultiplicatorStream((cat, word) => entities.getWordList(cat, word))
];

entities.loadEntities()
    .then(() => main.jsonToText(from, to, pipeline))
    .catch(e => console.error(e));

new MultiplicatorStream(getVariants)

Param	Type
getVariants	function

MultiplicatorStream

Kind: global class

• MultiplicatorStream
  • new MultiplicatorStream()
  • new MultiplicatorStream(getVariants)

new MultiplicatorStream()

Multiplicates a learning set data with available entities information

Example

const path = require('path');
const { EntitiesFromJson, MultiplicatorStream, jsonToText } = require('intent-tools');

const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');

const entities = new main.EntitiesFromJson(from);

const pipeline = [
    new MultiplicatorStream((cat, word) => entities.getWordList(cat, word))
];

entities.loadEntities()
    .then(() => main.jsonToText(from, to, pipeline))
    .catch(e => console.error(e));

new MultiplicatorStream(getVariants)

Param	Type
getVariants	function

Pipeline

Kind: global class

• Pipeline
  • new Pipeline()
  • .add(pipe) ⇒ this
  • .promise() ⇒ promise

new Pipeline()

Simple tool, which creates a Promise from pipeline of streams

pipeline.add(pipe) ⇒ this

Append a stream

Kind: instance method of Pipeline

Param	Type	Description
pipe	ReadableStream | Writable	the transform stream

pipeline.promise() ⇒ promise

Get a promise

Kind: instance method of Pipeline

jsonToText(input, output, [pipeline], [mapFn]) ⇒ Promise

Create fast-text learning set from Rasa intents json

Kind: global function

Param	Type	Default	Description
input	string | ReadableStream		path of Rasa intent set or stream
output	string | Writable		path or stream to write fast-text learning set
[pipeline]	Array		array of transform streams to modify the learning set
[mapFn]	function		text normalizer function

Example

const path = require('path');
const { jsonToText } = require('intent-tools');

const from = path.resolve(process.cwd(), 'sample.json');
const to = path.resolve(process.cwd(), 'trainingData.txt');

main.jsonToText(from, to)
    .catch(e => console.error(e));

wikiToText(input, output, [mapFn]) ⇒ Promise

Create a pretrained word vectors learning set from Wikipedia XML dump

Kind: global function

Param	Type	Default	Description
input	string | ReadableStream		path of Rasa intent set or stream
output	string | Writable		path or stream to write fast-text learning set
[mapFn]	function		text normalizer function

normalize(str)

Preserves only letters (with or withour diacritics) and makes everything lowercased

Kind: global function Returs: string

Param	Type	Description
str	string	input string