Receptionist Bot (RB) using Dialogflow (Part 0 - Introduction)

RECEPTIONIST CHATBOT

"new way for automated trading system"

What is this?

• Order ChatBot is chat bot running on Android OS
• This shop-assistant for coffee shop
• Emotions were displayed on Android screen
• Make conversation with customers
• Make order from customers

Target

• Build shop-assistant interact with customers:
• Emotion, conversation interface
• Recevice request from customer
• Make order
• Propose and provide promotinal information to customers
• Small talk
• Send bill notification to cashier

• Easy for update new context, not need to rebuild client app
• Online build, testing, integrations.
• Real-time updates

Overview system

Bot conversation

BOT display information

BOT emotions

Order scene

Customer: Hi. Could I have coffee no sugar
Bot: Sure. Do you need anything else?
Customer: Nope.

Bot: Yup. That'll be 25.000 vnđ in total

Bot:   Good morning. Can I take your order.

Customer: Yes. Could I have a coffee latte

Bot:   Sure. What size would you like?

Customer: Er... tall please. And I'll have a chocolate muffin.

Bot: OK. That'll be 32.000 vnđ in total

Ask promotion scene

Customer: Good morning. What promotions in shop today?

Bot: Nice to meet you. Today, shop have promotions:

* Discount 20% for each bill from 200.000 vnđ.

* Free 01 Coca cup for each Pizza Combo

Customer: Yes. Could I have a Pizza Combo

Bot:   Sure. That'll be 85.000 vnđ in total

Small talk scene

Customer: What's up?
Bot: I think you are surprising about me.

Customer: Goodmorning
Bot: Goodmorning to you.

Customer: What is your name?
Bot: My name is MariBot.

Customer: How old are you?
Bot: By the end of this year I was 1 year old.

Customer: How are you?
Bot: Wonderful as always. Thanks for asking.

Customer: You're so sweet

Bot: Thanks! The feeling is mutual.

Price

• Bot run on android OS (tablet): Available
• Webhook build on heroku: Free
• Firebase database: 1GB stored and 10 GB dowloaded/month Free.
• Dialogflow: text interfaction Free usage and unlimited requests. Quotas and limits: text requests 180/min

Timeline

Receptionist Bot (RB) using dialogflow - Part 3 (start with webhook)

Hi all. In this tutorials, we will work with webhook.
If you loss webhook define or any technical points then please check Part1 Part 2.

What need for build webhook?
+ Web server: local for testing
+ IDE tool: Visual Code (VS)
+ ngrok: for local testing

Let's go!


I. MAKE STRUCTURE FOR WEBHOOK

Step1. Setup NodeJS
You can skip this step if you have installed NodeJS berfore.
Follow steps for download and install NodeJS here

Step2. Create new project in VS and create package.json same below:

Open terminal on VS and typing: npm install

Step 3. Create app.js file with contains bellow:

Step 4. Run "node app.js" on terminal of VS and get result below.

II. CONNECT WEBHOOK TO DIALOGFLOW 

Currently, your webhook is running on localhost. Dialogflow is online so we can not connect to local webhook.
What solution for this problem? ngrok
We will push local to online by using ngrok.

You can download and install ngrok here
Next step, open VS terminal and run "ngrok http 8080"

Forwarding http.... localhost:8080 is online URL for your local.
Copy https URL to clipboard.
Open dash board of Dialogflow project  > Hit Fullfilment > Paste to Webhook URL* > Save

Typing anythings to "Try it now" of Dialogflow and you can see result on "Text response" session.
Now, I guest you can see:

"I didn't understand
I'm sorry, can you try again?"

That is done. We can connect your Dialogflow to webhook on your local machine.


In next post, we will build intent handle actions on webhook:
+ Find product
+ Order product

Receptionist Bot (RB) using dialogflow - Part 2 (build stories)

Hi. Welcome to next part of RB system.

If you don't know RB system then check before post here.

In this part. We will step by step build stories in coffee shop for order function and realtime testing.

What is stories? Stories are small communication between human and human, human and BOT.

Below is communication sample in coffee shop

Customer: Hello
Staff:         Hi. What would you like to drink?
Customer: I want a cup coffee
Staff:         Ok. Would you like more or less sugar?
Customer: more
Staff:         Yes. A cup coffee more sugar

We build a RB on Dialogflow and training to my RB can understand and response each request.

I. BUILD STORIES WITH SIMPLE INTENT

Step 1. Go to Dialogflow homepage and register new account.

Step 2. Go to console in dialogflow and create new bot.

 

Step 3. Set name and link RB to google project. This is importance, each bot in Dialogflow direct link to each google project for using service in google project (google assistant, ..). If select "create new google project" then new google project will create link to gmail account registered.

Step 4. After bot created, you have 2 intent default: 

• Default Fallback Intent: intent will called when you say anything not match intent available.
• Default Welcome Intent: handle for welcome sentence 

Try input anything in voice box and hit enter.

 

Some information display same above picture. 

"Can you say that again?" this is response from bot. It's define in Default Fallback Intent because text you enter not match anything intent.

Hit "SHOW JSON" to view response from bot in Json format.

Step 5. We create new Intent by hit plus at right of Intents menu. Enter name for intent "Ask Promotion". We will create ask promotion intent for handler question of customer.

Hit "Add Training Phrases" to add pattern we guess. Think some sentence customer can ask:

• What promotions today?
• Hi shop. Have any promotions today?
• Have promotions for VIP members?
• Have promotions discount?

We can see that "promotions" always exist in sentence but each another meaning. So we need group sentences same meaning.

Group 1: ask any promotions 

• What promotions today?
• Hi shop. Have any promotions today?

Group 2: ask only promotions for VIP member

• Have promotions for VIP members?

Group 3: ask only promitions discount

• Have promotions discount?

Each group we will build another intent. Starting build intent for group 1, we should rename intent "Ask Promotion" to "Ask any promotion". Enter full sentences in group 1 to Training Phrases. This pattern will compare when customer saying.

Hit "Add Responses" and enter some response you want response when this intent matched.

Hit "Save" to save to training bot.

Step 6. Testing new intent. Enter sentence in textbox "What promotions today?" and hit enter. What you see? In default response session display sentence you define in before step.  If you define more than 1 response then your system will random display. End this step, you can define more intent for group 2 and 3.

Next, try enter "promotions today?" and hit enter. What you see? Bot still recognizes correct Ask any promotion intent. This is ai of dialogflow, help build small pattern but recognizes and handler big request from user.

II. BUILD STORIES COMBINE INTENT AND ENTITY

In this part 1, we have build intent from sentence which we guess. But if intent relative more options then we need build sentence for each options. 

Example customer ask staff:  

• Does sell the capuchino?
• Does sell the coffee?
• Does sell the mocha?

Step 1. When product name in sentence is more options, we define this options is entity. Hit plus in Entities tab for add new entity.

Enter name for entity is "drink" and enter drink name into entry box. Each entry box is each product name. Each product can more name example: capuchino, hot capuchino, cream capuchino, .. we enter all name into entry box and separate by tab.

Step 2. Create new intent with name "Ask product"

In Training Phase session click to (") to change to (@). This is mode for entities references. 

Enter "Does sell the @drink:drink_entity ?" 

• @drink: reference to drink entity.
• drink_entity: anything name define param name response from dialogflow (Json format)

Enter response text and hit save.

Step 3. Testing with sentence "does sell the capuchino". What you see?

 

In PARAMETER session display drink_entity is capuchino. End step, you can knowledge dialogflow and build intent in dialogflow.

Try another product you not define and see. Bot not found drink_entity.

III. CHAT BOT TESTING

Dialogflow support more testing method. In which the easiest way is Web Demo. Hit to Integrations in left menu and turn on Web Demo. After, click to link in dialog for start testing.

  At this point. You can chat with your bot realtime.

In next part. We starting connect bot to webhook for realbot.

To view previous part, click here.

Receptionist Bot (RB) using dialogflow - Part 1 (preparing)

Automation combine ai tech is easy for life.

In this first post, i will share step by step make bot from dialogflow on Android OS.

OVERVIEW

Bellow is block diagram of RB system.

Block 1: Android app with roles: recognize your voice, communication with dialogflow, speaker response text, display emotion and external infomation on screen.

Block 2: Dialogflow understand what u are saying, detect entity in voice and pass to your webhook.

Block 3: Webhook is center processing of RB. At here, we call some api, customize response with some condition. Pace for conduct business in your system.

Block 4 (Options): It's not necessary if your DB builtin webhook. Firebase database real time help your system more flexible. Your data is updated as soon as any changes occur.

Block 5: Notification builtin cashier machine's.

UNDERSTAND TECHNICAL POINT

In this part, i will introduction technical points using in RB system.

1. Speech to text on Android

    To achieve this goal, we have more options.

    + Google Cloud Speech API: good quality but need network for each request.
    + Snowboy: solution for hotword detection (base on Pocketsphinx library).

   + RecognizerIntent (Google Now): easy for using and communication by intent (call another app using deeplink). More options for recognize: time out listen, voice score, extra partial support, confidence scores, complete silent length support, ..)

   In above options, i will choice RecognizerIntent options because it's many advantages and easily integrated RB system.

2. Text to speech on Android

    We use TextToSpeech api for this technical point. TextToSpeech api will synthesizes speech from text for immediate playback or to create a sound file. Moreover, we can register start/end speech event in communication. 

3. Dialogflow

    Use machine learning to understand what users are saying. It's free and can integrated more another system: Google Assistant, Facebook Messager, Slack, Viber, ... 

    Basic concept: The process a Dialogflow agent follows from invocation to fulfillment is similar to someone answering a question, with some liberties taken of course. In the example scenario below, the same question is being asked, but we compare the "human to human" interaction with a conversation with an Dialogflow agent.

    Below is communication sample:

    

Dialogflow define 2 main concept: intent and entity.

    "Hi Shop. I want to order cup coffee" is intent. Intent is simple sentence includes S, V, O, adj, adv .. and make sense. 

    "want to", "order", "coffee" is entities. Entity is component, part of sentence and it has an important role within the meaning of the sentence.

    After analytic intent, dialogflow combine result into JSON response and pass to target (webhook).

4. Webhook

   We use Heroku to make webhook for RB system. This is free service for testing and deploy. 

  

  A webhook (also called a web callback or HTTP push API) is a way for an app to provide other applications with real-time information. A webhook delivers data to other applications as it happens, meaning you get data immediately. Unlike typical APIs where you would need to poll for data very frequently in order to get it real-time. This makes webhooks much more efficient for both provider and consumer. The only drawback to webhooks is the difficulty of initially setting them up.

   Webhooks are sometimes referred to as “Reverse APIs,” as they give you what amounts to an API spec, and you must design an API for the webhook to use. The webhook will make an HTTP request to your app (typically a POST), and you will then be charged with interpreting it.

   We using NodeJS languages for build webhook. Detail for build webhook i will introduction in next part.

5. Firebase database realtime

   The Firebase Realtime Database is a cloud-hosted database. Data is stored as JSON and synchronized in realtime to every connected client. When you build cross-platform apps with our iOS, Android, and JavaScript SDKs, all of your clients share one Realtime Database instance and automatically receive updates with the newest data.

   In RB system, we use Firebase database realtime for storage drink, food, promotions and user data. It's help immediate update price, promotions when the system is running

6. Notification on android system

   We use Firebase Cloud Messaging (FCM) for the notification function. 

   FCM is a cross-platform messaging solution that lets you reliably deliver messages at no cost.

   Using FCM, you can notify a client app that new email or other data is available to sync. You can send notification messages to drive user re-engagement and retention. For use cases such as instant messaging, a message can transfer a payload of up to 4KB to a client app.

PLANT FOR IMPLEMENT OCB SYSTEM

1. Understanding technical points

• Voice input/output on android.
• Dialogflow
• Webhook communication dialog flow

2. Dialogflow implement

• Make draft stories (small talk, order context, ..)
• Build intent and entities

3. Android implement

• Voice input/output (local languages support as vietnamese)
• Implement  dialogflow api for talk
• Interaction (emotion, display bill, display drink menu)

4. Webhook implement

• Prepare webhook
• Build DB structure (on Firebase database realtime)

5. Notification on cashier machine

6. Testing

In next part, i will start with make draft stories and build intent on dialogflow.
Thank for view and please ask me anything if u understand.