Slang Payments Assistant

Try out the playground app for developers to understand the assistant.

Install the Slang Payments Assistant package

yarn setup

If you use yarn for install packages, run the below command,

yarn add @slanglabs/slang-conva-react-native-payments-assistant

npm setup

If you use npm for managing your packages, run the below command

npm install @slanglabs/slang-conva-react-native-payments-assistant --save

Because Slang uses native libraries, you need to link the package to your codebase to run the automatic linking steps

react-native link @slanglabs/slang-conva-react-native-payments-assistant

Android

Finally, add the path to the Slang maven repository (to download native library dependencies) to your top-level gradle file

# Add this to your top level gradle file

allprojects {  
    repositories {    
        …    
        maven { url "http://maven.slanglabs.in:8080/artifactory/gradle-release" }  
    }
}

Code Integration

Initialization

The next step is to initialize the SDK with the keys you obtained after creating the Assistant in the Slang console.

NOTE This should ideally be done in the componentDidMount of your main app component

import SlangPaymentsAssistant from '@slanglabs/slang-conva-react-native-payments-assistant';

SlangPaymentsAssistant.initialize({
    requestedLocales: ['en-IN', 'hi-IN'], // The languages to enable
    assistantId: '<assistant id>',        // The Assistant ID from the console
    apiKey: '<API Key>',                  // The API key from the console
})

Show the Trigger (microphone icon)

NOTE One can call "show" and "hide" methods as required to control the visibility of the Assistant

SlangPaymentsAssistant.ui.showTrigger() // There is a corresponding hideTrigger too if needed

NOTE The trigger is sticky, which means that it will show up on all Activities after it is made visible. To prevent the trigger from showing up on specific activities, you will need to call: SlangPaymentsAssistant.ui.hideTrigger()

Implement Actions

Last but not the least, the app needs to implement the Actions associated with the various User Journeys supported by the Assistant. This can be done as shown below

const actionHandler = {
  onTransaction: (transactionInfo, transactionUserJourney) => {
    // Handle the transaction info  
    // ...
    
    transactionUserJourney.setSuccess();
    return TransactionUserJourney.AppState.TRANSACTION;
  },

  onBillPayment: (billInfo, billPaymentUserJourney) => {
    // Handle the Bill info  
    // ...
    
    billPaymentUserJourney.setSuccess();
    return BillPaymentsUserJourney.AppState.PAYMENT;
  },

  onNavigation: (navigationInfo, navigationUserJourney) => {
    // Handle the navigation info  
    // ...
    
    navigationUserJourney.setSuccess();
    return NavigationUserJourney.AppState.NAVIGATION;
  },

  onAssistantError: errorCode => {
    // Handle errors that might have happened during the processing of the
    // Assistant
    
    // Error codes available 
    // FATAL_ERROR, SYSTEM_ERROR, ASSISTANT_DISABLED, INVALID_CREDENTIALS,    
  },
};

SlangPaymentsAssistant.setAction(actionHandler)

The following user journeys are currently supported by the Slang Payments Assistant:

1. Transactions
2. BillPayments
3. Navigation

The Action Handler interface has an explicit callback for each of the supported user journeys. Whenever the Assistant detects the user's journey (based on what they spoke), it invokes the callback associated with that user journey. When these callbacks are invoked, the Assistant also passes the parametric data corresponding to the user journey that the Assistant was able to gather. The app is then expected to:

1. Consume the parametric data as needed
2. Optionally launch appropriate UI actions
3. Set appropriate conditions in the Assistant based on the app's internal state
4. Return the AppState that the app transitioned to

Registering for events

The app can register with the Assistant to be notified of all interesting life-cycle events via the setLifeCycleObserver method

const paymentsAssistantLifeCycleObserver = {
  onAssistantInitSuccess: () => {
  },

  onAssistantInitFailure: error => {
  },

  onAssistantInvoked: () => {
  },

  onAssistantClosed: isCancelled => {
  },

  onAssistantLocaleChanged: locale => {
  },

  onUnrecognisedUtterance: utterance => {
  },

  onUtteranceDetected: utterance => {
  },

  onOnboardingSuccess: () => {
  },

  onOnboardingFailure: () => {
  },

  onMicPermissionDenied: () => {
  },

  onMicPermissionGranted: () => {
  },

};

SlangPaymentsAssistant.setLifecycleObserver(paymentsAssistantLifeCycleObserver);

As part of the Lifecycle Events API, an observer will be notified of the following events:

1. onAssistantInitSuccess Called as soon as the Assistant has initialized successfully and is ready to serve the app
2. onAssistantInitFailure Called when the Assistant failed to initialize successfully. The reason is passed as a parameter to this callback
3. onAssistantInvoked Called whenever the Assistant has been launched (this can be either as a result of the user clicking on the trigger or the app invoking the Assistant via the startConversation API)
4. onAssistantClosed Called whenever the Assistant has been dismissed. A boolean parameter isCanceled is passed to indicate whether this happened because the user canceled the session or if the Assistant was done with its job
5. onAssistantLocaleChanged Called whenever the user changes the locale of the Assistant
6. onUnrecognisedUtterance Called whenever the Assistant is not able to understand what the user spoke. The utterance that the user spoke is passed as a parameter.
7. onUtteranceDetected Called whenever the Assistant has detected the user uttearnce. The utterance that the user spoke is passed as a parameter.
8. onOnboardingSuccess Called when the Assistant onboarding process has completed successfully.
9. onOnboardingFailure Called when the Assistant onboarding process has failed.
10. onMicPermissionDenied Called when the mic permission for the assistant was denied by the user.
11. onMicPermissionGranted Called when the mic permission for the assistant was granted by the user.

The onTransaction callback

onTransaction: (transactionInfo, transactionUserJourney)

When this callback is invoked, the app is expected to:

1. Consume the details of the search request via the TransactionInfo parameter.
2. Fire the app's search request.
3. Finally, return the AppState along with the appropriate Condition corresponding to the state that the app transitioned into

For example, for a given onTransaction callback invocation, if the transaction completes successfully and the app transitions to a screen, the app would return theAppState as TransactionUserJourney.AppState.TRANSACTION along with condition SUCCESS , as shown below:

  onTransaction: (transactionInfo, transactionUserJourney) => {
    // Handle the transactionInfo 
    // ...
    
    transactionUserJourney.setSuccess();
    return TransactionUserJourney.AppState.TRANSACTION;
    
  }

Sample Utterances that could trigger the transaction Info

1. "send 200 rupees to Ankita"
2. "Request 300 rupees from Jagan"

TransactionInfo Parameter

The TransactionInfo contains the breakdown of the original transaction request. Its structure is as described below:

// When the user searches for something like 
// send 200 rupees to Ankita
// This is how the TransactionInfo parameter would be populated
{
   "amount": {
        "currency": "INR",
        "amount": 200
    },
   "transactionAction":"SEND",
   "name":"Ankita"
}

NOTE

TRANSACTION_ACTION includes :

1. "SEND"
2. "REQUEST"
3. "SCAN"
4. "UNKNOWN"

Supported AppStates

The following AppStates are supported:

1. TRANSACTION : To be returned when the app performs the transaction. To indicate whether the transaction was successful or not, with a greater level of detail, please use the appropriate conditions.

2. UNSUPPORTED : To be returned when the app is not ready to handle search yet. The Assistant will let the user know that the search is not yet supported by the app.

NOTE The Slang Payments Assistant provides a special WaitingAppState that is common across all UserJourney types for completing asynchronous operations within the callback.

Supported Conditions

1. TRANSACTION :
   1. setSuccess : The transaction journey was successful
   2. setFailure : The transaction journey was not successful
   3. setNameRequired : The name associated with the transaction is required
   4. setAmountRequired : The transaction amount is required
   5. setTransactionAction : The transaction action is required

The onBillPayment callback

onBillPayment: (billInfo, billPaymentJourney)

When this callback is invoked, the app is expected to:

1. Consume the details of the search request via the TransactionInfo parameter.
2. Fire the app's search request.
3. Finally, return the AppState along with the appropriate Condition corresponding to the state that the app transitioned into

For example, for a given onBillPayment callback invocation, if the bill payment completes successfully and the app transitions to a screen, the app would return theAppState as BillPaymentsUserJourney.AppState.PAYMENT along with condition SUCCESS , as shown below:

  onBillPayment: (billInfo, billPaymentJourney) => {
    // Handle the billInfo 
    // ...
    
    billPaymentJourney.setSuccess();
    return BillPaymentsUserJourney.AppState.PAYMENT;
    
  }

Sample Utterances that could trigger the bill payment Info

1. "pay my electricity bill"
2. "I want to pay my postpaid bill"

BillInfo Parameter

The BillInfo contains the breakdown of the original bill payment request. Its structure is as described below:

// When the user searches for something like 
// pay my electricity bill
// This is how the BillInfo parameter would be populated
{
   "type":"electricity bill"
}

Supported AppStates

The following AppStates are supported:

1. PAYMENT : To be returned when the app performs the payment. To indicate whether the payment was successful or not, with a greater level of detail, please use the appropriate conditions.

2. UNSUPPORTED : To be returned when the app is not ready to handle search yet. The Assistant will let the user know that the search is not yet supported by the app.

NOTE The Slang Payment Assistant provides a special WAITING that is common across all UserJourney types for completing asynchronous operations within the callback.

Supported Conditions

1. PAYMENT :
   1. setSuccess : The bill payment journey was successful
   2. setFailure : The bill payment journey was not successful
   3. setBillTypeRequired : The bill type associated with the bill payment is required

The onNavigation callback

onNavigation: (navigationInfo, navigationUserJourney)

When this callback is invoked, the app is expected to:

1. Consume the details of the search request via the TransactionInfo parameter.
2. Fire the app's search request.
3. Finally, return the AppState along with the appropriate Condition corresponding to the state that the app transitioned into

For example, for a given onNavigation callback invocation, if the navigation journey completes successfully and the app transitions to a screen, the app would return theAppState as NavigationUserJourney.AppState.NAVIGATION along with condition SUCCESS , as shown below:

  onNavigation: (navigationInfo, navigationUserJourney) => {
    // Handle the navigationInfo 
    // ...
    
    navigationUserJourney.setSuccess();
    return NavigationUserJourney.AppState.NAVIGATION;
    
  }

Sample Utterances that could trigger the navigation Info

1. "show my transaction history"
2. "I would like to check my balance"

NavigationInfo Parameter

The BillInfo contains the breakdown of the original bill payment request. Its structure is as described below:

// When the user searches for something like 
// pay my electricity bill
// This is how the BillInfo parameter would be populated
{
   "type":"electricity bill"
}

Supported AppStates

The following AppStates are supported:

1. NAVIGATION : To be returned when the app performs the navigation. To indicate whether the navigation was successful or not, with a greater level of detail, please use the appropriate conditions.

2. UNSUPPORTED : To be returned when the app is not ready to handle search yet. The Assistant will let the user know that the search is not yet supported by the app.

NOTE The Slang Payment Assistant provides a special WAITING that is common across all UserJourney types for completing asynchronous operations within the callback.

Supported Conditions

1. NAVIGATION :
   1. setSuccess : The navigation journey was successful
   2. setFailure : The navigation journey was not successful
   3. setTargetRequired : The target associated with the navigation is required

Asynchronous Action Handling

While handling a user journey, the app might not be able to identify the state or the condition of the journey in a synchronous way. For example, the app could open a new activity (or page) which could then fire up a search to the back-end and show the results. So the callback itself would not be aware of the result unless it blocks till the entire activity is loaded. This is not desirable and leads to poor UX. To solve this problem, the Slang Assistant programming model provides ways for the app to asynchronously signal the Assistant about the app state changes and conditions. This can be done by using the "wait" and "notify" semantics. The process is quite straightforward. In order to deal with asynchronous operations within the callback: The app must return the AppState WAITING to inform the Assistant that it's not yet ready to identify the state or the condition of the app. The Assistant goes into a "waiting" state without blocking the UI or other threads and continues to wait for further notification from the app. When the app has completed the asynchronous operation, it can then call notifyAppState to notify the Assistant to proceed further.

Returning WAITING AppState

The app should return WAITING from the user journey callback. This would keep the Assistant in a "processing" (which is what it would have been when it invoked the callback)

onTransaction: (transactionInfo, transactionUserJourney) => {
    // Fire an async transaction Info handling request  
    // ...
    
    return TransactionUserJourney.AppState.WAITING;
  }

Notifying the Assistant

Once the Assistant is asked to wait, it will continue to remain in the "processing" state until the app notifies it to stop waiting, or the user cancels the operation through the UI. For best results, the app should ensure that every path that returns WAITING has an accompanying call to notify the Assistant.

To notify the Assistant with the AppState, the app needs access to the transactionUserJourney object that was passed to the callback. (The App can store the transactionUserJourney object obtained in the callback to state and then use it once it needs to accessed later)

transactionUserJourney.setSuccess()
transactionUserJourney.notifyAppState(TransactionUserJourney.AppState.TRANSACTION);