Receptionist Bot (RB) using dialogflow - Part 1 (preparing)

Automation combine ai tech is easy for life.

In this first post, i will share step by step make bot from dialogflow on Android OS.

OVERVIEW

Bellow is block diagram of RB system.

Block 1: Android app with roles: recognize your voice, communication with dialogflow, speaker response text, display emotion and external infomation on screen.

Block 2: Dialogflow understand what u are saying, detect entity in voice and pass to your webhook.

Block 3: Webhook is center processing of RB. At here, we call some api, customize response with some condition. Pace for conduct business in your system.

Block 4 (Options): It's not necessary if your DB builtin webhook. Firebase database real time help your system more flexible. Your data is updated as soon as any changes occur.

Block 5: Notification builtin cashier machine's.

UNDERSTAND TECHNICAL POINT

In this part, i will introduction technical points using in RB system.

1. Speech to text on Android

    To achieve this goal, we have more options.

    + Google Cloud Speech API: good quality but need network for each request.
    + Snowboy: solution for hotword detection (base on Pocketsphinx library).

   + RecognizerIntent (Google Now): easy for using and communication by intent (call another app using deeplink). More options for recognize: time out listen, voice score, extra partial support, confidence scores, complete silent length support, ..)

   In above options, i will choice RecognizerIntent options because it's many advantages and easily integrated RB system.

2. Text to speech on Android

    We use TextToSpeech api for this technical point. TextToSpeech api will synthesizes speech from text for immediate playback or to create a sound file. Moreover, we can register start/end speech event in communication. 

3. Dialogflow

    Use machine learning to understand what users are saying. It's free and can integrated more another system: Google Assistant, Facebook Messager, Slack, Viber, ... 

    Basic concept: The process a Dialogflow agent follows from invocation to fulfillment is similar to someone answering a question, with some liberties taken of course. In the example scenario below, the same question is being asked, but we compare the "human to human" interaction with a conversation with an Dialogflow agent.

    Below is communication sample:

    

Dialogflow define 2 main concept: intent and entity.

    "Hi Shop. I want to order cup coffee" is intent. Intent is simple sentence includes S, V, O, adj, adv .. and make sense. 

    "want to", "order", "coffee" is entities. Entity is component, part of sentence and it has an important role within the meaning of the sentence.

    After analytic intent, dialogflow combine result into JSON response and pass to target (webhook).

4. Webhook

   We use Heroku to make webhook for RB system. This is free service for testing and deploy. 

  

  A webhook (also called a web callback or HTTP push API) is a way for an app to provide other applications with real-time information. A webhook delivers data to other applications as it happens, meaning you get data immediately. Unlike typical APIs where you would need to poll for data very frequently in order to get it real-time. This makes webhooks much more efficient for both provider and consumer. The only drawback to webhooks is the difficulty of initially setting them up.

   Webhooks are sometimes referred to as “Reverse APIs,” as they give you what amounts to an API spec, and you must design an API for the webhook to use. The webhook will make an HTTP request to your app (typically a POST), and you will then be charged with interpreting it.

   We using NodeJS languages for build webhook. Detail for build webhook i will introduction in next part.

5. Firebase database realtime

   The Firebase Realtime Database is a cloud-hosted database. Data is stored as JSON and synchronized in realtime to every connected client. When you build cross-platform apps with our iOS, Android, and JavaScript SDKs, all of your clients share one Realtime Database instance and automatically receive updates with the newest data.

   In RB system, we use Firebase database realtime for storage drink, food, promotions and user data. It's help immediate update price, promotions when the system is running

6. Notification on android system

   We use Firebase Cloud Messaging (FCM) for the notification function. 

   FCM is a cross-platform messaging solution that lets you reliably deliver messages at no cost.

   Using FCM, you can notify a client app that new email or other data is available to sync. You can send notification messages to drive user re-engagement and retention. For use cases such as instant messaging, a message can transfer a payload of up to 4KB to a client app.

PLANT FOR IMPLEMENT OCB SYSTEM

1. Understanding technical points

• Voice input/output on android.
• Dialogflow
• Webhook communication dialog flow

2. Dialogflow implement

• Make draft stories (small talk, order context, ..)
• Build intent and entities

3. Android implement

• Voice input/output (local languages support as vietnamese)
• Implement  dialogflow api for talk
• Interaction (emotion, display bill, display drink menu)

4. Webhook implement

• Prepare webhook
• Build DB structure (on Firebase database realtime)

5. Notification on cashier machine

6. Testing

In next part, i will start with make draft stories and build intent on dialogflow.
Thank for view and please ask me anything if u understand.