hermes-protocol

A JavaScript wrapper around the the Hermes protocol.

Context

The hermes-protocol library provides bindings for the Hermes protocol formely used by Snips components to communicate together. hermes-protocol allows you to interface seamlessly with the Rhasspy and Hermod ecosystems and create Voice applications with ease!

hermes-protocol abstracts away the connection to the MQTT bus and the parsing of incoming and outcoming messages from and to the components of the platform and provides a high-level API as well.

Setup

npm install hermes-protocol

hermes-protocol uses a dynamic library generated by the hermes rust code under the hood. The installation process will automagically download the file if your os and architecture is supported.

⚠️ Unsupported platforms / architectures

If the setup could not infer the library file version, it will attempt to build it from the sources. Please note that rust and git are required in order to build the library!

If you want to force this behaviour, you can also define the HERMES_BUILD_FROM_SOURCES environment variable before running npm install.

env HERMES_BUILD_FROM_SOURCES=true npm install hermes-protocol

Usage

Minimal use case

const { withHermes } = require('hermes-protocol')

/*
    A small js context manager that sets up an infinite loop to prevent
    the process from exiting, and exposes an instance of the Hermes class.
*/
withHermes(hermes => {
    // Instantiate a dialog object
    const dialog = hermes.dialog()

    // Subscribes to intent 'myIntent'
    dialog.flow('myIntent', (msg, flow) => {
        // Log intent message
        console.log(JSON.stringify(msg))
        // End the session
        flow.end()
        // Use text to speech
        return `Received message for intent ${myIntent}`
    })
})

Expanded use case

const { withHermes } = require('hermes-protocol')

/*
    The 'withHermes' function exposes a done() function
    that can be called to clean up the context loop and exit.
*/
withHermes((hermes, done) => {
    // NB: Dialog is only one of the available API Subsets.
    const dialog = hermes.dialog()

    /*
        Every API Subset can publish and receive data based on a list of events.

        For the purpose of this example, we will only use the Dialog subset, and the
        events related to a dialog session.

        Note that more events are available for each subset.
    */

    // You can subscribe to an event triggered when the intent 'intentName' is detected like this:
    dialog.on('intent/intentName', message => {

        // The 'message' argument contain all the data you need to perform an action based on what the user said.

        // For instance, you can grab a slot and its value like this.
        const mySlot = msg.slots.find(slot => slot.slotName === 'slotName')
        const slotValue = mySlot.value.value
        // And here is how to grab the intent name.
        console.log('Received intent', message.intent.intentName)

        // Then, you can either:
        if(continueSession) {
            // 1 - Continue the dialog session if you expect another intent to be detected.
            dialog.publish('continue_session', {
                sessionId: message.sessionId,
                text: 'Session continued',
                // In this case, if you already set up a subscription for 'intent/nextIntent' then it will be triggered if the user speaks that intent.
                intentFilter: ['nextIntent']
            })
        } else {
            // 2 - Or end the dialog session.
            dialog.publish('end_session', {
                sessionId: message.sessionId,
                text: 'Session ended'
            })
        }
        // !! But not both !!
    })

    // You can also unsubscribe to a registered event.
    const handler = message => {
        // In this case, unsubscribe the first time this message is received.
        dialog.off('intent/someIntent', handler)
        // ...
    }
    dialog.on('intent/someIntent', handler)

    // Or process a subscription only once:
    dialog.once('intent/someIntent', message => {
        // ...
    })

    /*
        Now this is all for the basics, but for managing a dialog session
        using .on / .off / .once / .publish is actually not the best way!

        The Dialog API also exposes a small wrapper that make these operations much easier,
        and it is strongly recommended to use this wrapper instead!

        See below for an example on how to build a dialog flow tree using this API.
    */

    /*
        The goal is to register the following dialog paths:
        A
        ├── B
        │   └─ D
        └── C
        In plain words, intent 'A' starts the flow, then restrain the next intents to 'B' or 'C'.
        If 'B' is the next intent detected, then next intent must be 'D' (and end the flow after 'D').
        If it was 'C', end the flow.
    */
    dialog.flow('A', (msg, flow) => {

        console.log('Intent A received. Session started.')

        /*
            At each step of the dialog flow, you have the choice of
            registering the next intents, or end the flow.

            We then subscribe to both intent B or C so that the dialog
            flow will continue with either one or the other next.
        */

        // Mark intent 'B' as one of the next dialog intents. (A -> B)
        flow.continue('B', (msg, flow) => {
            console.log('Intent B received. Session continued.')

            // Mark intent 'D'. (A -> B -> D)
            flow.continue('D', (msg, flow) => {
                console.log('Intent D received. Session is ended.')
                flow.end()
                return 'Finished the session with intent D.'
            })

            // Make the TTS say that.
            return 'Continue with D.'
        })

        // Mark intent 'C' as one of the next dialog intents. (A -> C)
        flow.continue('C', (msg, flow) => {
            const slotValue = msg.slots[0].value.value
            console.log('Intent C received. Session is ended.')
            flow.end()
            return 'Finished the session with intent C having value ' + slotValue + ' .'
        })

        // The continue / end message options (basically text to speech)
        // If the return value is a string, then it is equivalent to { text: '...' }
        return 'Continue with B or C.'
    })

})

API

Sections:

• Context loop
• Hermes class
• Common ApiSubset methods
• Dialog Api Subset
• DialogFlow
• Injection Api Subset
• Feedback Api Subset
• Tts Api Subset

Context loop

Back ⬆️

An hermes client should implement a context loop that will prevent the program from exiting.

Using withHermes

const { withHermes } = require('hermes-protocol')

// Check the Hermes class documentation (next section) for available options.
const hermesOptions = { /* ... */ }

/*
The withHermes function automatically sets up the context loop.

Arguments:
   - hermes is a freshly created instance of the Hermes class
   - call done() to exit the loop and destroy() the hermes instance
*/
withHermes((hermes, done) => {
    /* ... */
}, hermesOptions)

Instantiate Hermes and use the keepAlive tool

In case you want to create and manage the lifetime of the Hermes instance yourself, you can use keepAlive and killKeepAlive to prevent the node.js process from exiting.

const { Hermes, tools: { keepAlive, killKeepAlive }} = require('hermes-protocol')

const hermes = new Hermes(/* options, see below (next section) */)

// Sleeps for 60000 miliseconds between each loop cycle to prevent heavy CPU usage
const keepAliveRef = keepAlive(60000)

// Call done to free the Hermes instance resources and stop the loop
function done () {
    hermes.destroy()
    killKeepAlive(keepAliveRef)
}

/* ... */

Hermes class

The Hermes class provides foreign function interface bindings to the Hermes protocol library.

⚠️ Important: Except for very specific use cases, you should have only a single instance of the Hermes class in your program. It can either be provided by the withHermes function OR created by calling new Hermes().

Just keep a single reference to the Hermes instance and pass it around.

The technical reason is that the shared hermes library is read and FFI bindings are created every time you call new Hermes or withHermes, which is really inefficient.

Back ⬆️

new Hermes({
    // The broker address (default localhost:1883)
    address: 'localhost:1883',
    // Enables or disables stdout logs (default true).
    // Use it in conjunction with the RUST_LOG environment variable. (env RUST_LOG=debug ...)
    logs: true,
    // Path to the hermes FFI dynamic library file.
    // Defaults to the hermes-protocol package folder, usually equivalent to:
    libraryPath: 'node_modules/hermes-protocol/libhermes_mqtt_ffi',
    // Username used when connecting to the broker.
    username: 'user name',
    // Password used when connecting to the broker
    password: 'password',
    // Hostname to use for the TLS configuration. If set, enables TLS.
    tls_hostname: 'hostname',
    // CA files to use if TLS is enabled.
    tls_ca_file: [ 'my-cert.cert' ],
    // CA paths to use if TLS is enabled.
    tls_ca_path: [ '/ca/path', '/ca/other/path' ],
    // Client key to use if TLS is enabled.
    tls_client_key: 'my-key.key',
    // Client cert to use if TLS is enabled.
    tls_client_cert: 'client-cert.cert',
    // Boolean indicating if the root store should be disabled if TLS is enabled.
    tls_disable_root_store: false
})

dialog()

Use the Dialog Api Subset.

const dialog = hermes.dialog()

injection()

Use the Injection Api Subset.

const injection = hermes.injection()

feedback()

Use the Sound Feedback Api Subset.

const feedback = hermes.feedback()

tts()

Use the text-to-speech Api Subset.

const tts = hermes.tts()

destroy()

Release all the resources associated with this Hermes instance.

hermes.destroy()

Common ApiSubset methods

Back ⬆️

Check out the hermes protocol documentation for more details on the event names.

on(eventName, listener)

Subscribes to an event on the bus.

const dialog = hermes.dialog()

dialog.on('session_started', message => {
    /* ... */
})

once(eventName, listener)

Subscribes to an event on the bus, then unsubscribes after the first event is received.

const dialog = hermes.dialog()

dialog.once('intent/myIntent', message => {
    /* ... */
})

off(eventName, listener)

Unsubscribe an already existing event.

const dialog = hermes.dialog()

const handler = message => {
    /* ... */
}

// Subscribes
dialog.on('intent/myIntent', handler)

// Unsubscribes
dialog.off('intent/myIntent', handler)

publish(eventName, message)

Publish an event programatically.

const { Enums } = require('hermes-protocol/types')

const dialog = hermes.dialog()

dialog.publish('start_session', {
    customData: 'some data',
    siteId: 'site Id',
    init: {
        type: Enums.initType.notification,
        text: 'hello world'
    }
})

Dialog Api Subset

Back ⬆️

The dialog manager.

Events available for publishing

• start_session

Start a new dialog session.

const { Enums } = require('hermes-protocol/types')

// Start a 'notification type' session that will say whatever is in the "text" field and terminate.

dialog.publish('start_session', {
    customData: /* string */,
    siteId: /* string */,
    init: {
        // An enumeration, either 'action' or 'notification'
        type: Enums.initType.notification,
        text: /* string */
    }
})

// Start an 'action type' session that will initiate a dialogue with the user.

dialog.publish('start_session', {
    customData: /* string */,
    siteId: /* string */,
    init: {
        // An enumeration, either 'action' or 'notification'
        type: Enums.initType.action,
        text: /* string */,
        intentFilter: /* string[] */,
        canBeEnqueued: /* boolean */,
        sendIntentNotRecognized: /* boolean */
    }
})

• continue_session

Continue a dialog session.

dialog.publish('continue_session', {
    sessionId: /* string */,
    text: /* string */,
    intentFilter: /* string[] */,
    customData: /* string */,
    sendIntentNotRecognized: /* boolean */,
    slot: /* string */
})

• end_session

Finish a dialog session.

dialog.publish('end_session', {
    sessionId: /* string */,
    text: /* string */
})

• configure

Configure intents that can trigger a session start.

dialog.publish('configure', {
    siteId: /* string */,
    intents: [{
        intentId: /* string */,
        enable: /* boolean */
    }]
})

Events available for subscribing

• intent/[intentName]

An intent was recognized.

• session_ended

A dialog session has ended.

• session_queued

A dialog session has been put in the queue.

• session_started

A dialog session has started.

• intent_not_recognized

No intents were recognized.

Note that the dialog session must have been started or continued with the sendIntentNotRecognized flag in order for this to work.

DialogFlow

Back ⬆️

The Dialog API Subset exposes a small API that makes managing complex dialog flows a breeze.

flow(intent, action)

Starts a new dialog flow.

const dialog = hermes.dialog()

dialog.flow('intentName', (message, flow) => {

    // Chain flow actions (continue / end)…

    // Return the text to speech if needed.
    return 'intentName recognized!'
})

// You can also return an object that will be used for
// the 'continue_session' or 'end_session' parameters.

dialog.flow('intentName', (message, flow) => {

    // Chain flow actions (continue / end)…

    return {
        text: 'intentName recognized!'
    }
})

// If you need to perform asynchronous calculations
// Just return a promise and the flow actions will
// be performed afterwards.

dialog.flow('intentName', async (message, flow) => {
    const json = await fetch('something').then(res => res.json())

    // Chain flow actions (continue / end)…

    return 'Fetched some stuff!'
})

flows([{ intent, action }])

Same as flow(), but with multiple starting intents.

Useful when designing speech patterns with loops ((intentOne or intentTwo) -> intentTwo -> intentOne -> intentTwo -> ..., so that the starting intents callbacks will not get called multiple times if a session is already in progress.

const intents = [
    {
        intent: 'intentOne',
        action: (msg, flow) => { /* ... */ }
    },
    {
        intent: 'intentTwo',
        action: (msg, flow) => { /* ... */ }
    }
]
dialog.flows(intents)

sessionFlow(id, action)

Advanced, for basic purposes use flow() or flows().

Creates a dialog flow that will trigger when the target session starts. Useful when initiating a session programmatically.

// The id should match the customData value specified on the start_session message.
dialog.sessionFlow('a_unique_id', (msg, flow) => {
    // ... //
})

flow.continue(intentName, action, { slotFiller })

Subscribes to an intent for the next dialog step.

dialog.flow('intentName', async (message, flow) => {

    flow.continue('otherIntent', (message, flow) => {
        /* ... */
    })

    flow.continue('andAnotherIntent', (message, flow) => {
        /* ... */
    })

    return 'Continue with either one of these 2 intents.'
})

About the slotFiller option

Set the slot filler for the current dialogue round with a given slot name.

Requires flow.continue() to be called exactly once in the current round.

If set, the dialogue engine will not run the the intent classification on the user response and go straight to slot filling, assuming the intent is the one passed in the continue, and searching the value of the given slot.

// The slot filler is called with value 'slotName' for intent 'myIntent'.
flow.continue('myIntent', (message, flow) => {
    // "message" will be an intent message ("myIntent") with confidence 1.
    // The "message.slots" field will either contain an array of "slotName" slots or an empty array,
    // depending on whether the platform recognized the slot.
}, { slotFiller: 'slotName' })

flow.notRecognized(action)

Add a callback that is going to be executed if the intents failed to be recognized.

dialog.flow('intentName', async (message, flow) => {

    /* Add continuations here ... */

    flow.notRecognized((message, flow) => {
        /* ... */
    })

    return 'If the dialog failed to understand the intents, notRecognized callback will be called.'
})

flow.end()

Ends the dialog flow.

dialog.flow('intentName', async (message, flow) => {
    flow.end()
    return 'Dialog ended.'
})

Injection Api Subset

Back ⬆️

Vocabulary injection for the speech recognition.

Events available for publishing

• injection_request

Requests custom payload to be injected.

const { Enums } = require('hermes-protocol/types')

injection.publish('injection_request', {
    // Id of the injection request, used to identify the injection when retrieving its status.
    id: /* string */,
    // An extra language to compute the pronunciations for.
    // Note: 'en' is the only options for now.
    crossLanguage: /* string */,
    // An array of operations objects
    operations: [
        // Each operation is a tuple (an array containing two elements)
        [
            // Enumeration: add or addFromVanilla
            // see documentation here: https://docs.snips.ai/guides/advanced-configuration/dynamic-vocabulary#3-inject-entity-values
            Enums.injectionKind.add,
            // An object, with entities as the key mapped with an array of string entries to inject.
            {
                films : [
                    'The Wolf of Wall Street',
                    'The Lord of the Rings'
                ]
            }
        ]
    ],
    // Custom pronunciations. Do not use if you don't know what this is about!
    // An object having string keys mapped with an array of string entries
    lexicon: {}
})

• injection_status_request

Will request that a new status message will be sent. Note that you should subscribe to injection_status beforehand in order to receive the message.

injection.publish('injection_status_request')

• injection_reset_request

Will clear all the previously injected values.

injection.publish('injection_reset_request', {
    // Identifies the request.
    // The id will be sent back in the injection reset completed message.
    requestId: /* string */
})

Events available for subscribing

• injection_status

Get the status of the last injection request.

• injection_complete

When an injection request completes.

• injection_reset_complete

When an injection reset request completes.

Feedback Api Subset

Back ⬆️

Control the sound feedback.

Events available for publishing

• notification_on

Turn the notification sound on.

feedback.publish('notification_on', {
    "siteId": /* string */,
    "sessionId": /* string */
})

• notification_off

Turn the notification sound off.

feedback.publish('notification_off', {
    "siteId": /* string */,
    "sessionId": /* string */
})

TTS Api Subset

Back ⬆️

Exposes text-to-speech options.

Events available for publishing

• register_sound

Register a sound file and makes the TTS able to play it in addition to pure speech.

You can interpolate a text-to-speech string with the following tag: [[sound:soundId]]

const wavBuffer = // A Buffer object containing a wav file.

tts.publish('register_sound', {
    soundId: /* the ID that is going to be used when telling the TTS to play the file */,
    wavSound: wavBuffer.toString('base64')
})

License

Apache 2.0/MIT

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)

• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

  at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.