Persona Form Sets for chatbots

Persona Form Sets for chatbots

Allow a single file to retain all the forms for a persona in one place, making them easy to save, modify and reload.

Role is added to each form to direct what it is doing in the persona, by default all are load unless the form type is action. Not all forms need to be used in the persona.

<form role=”main/background/action/load”

Once loaded you can see all the forms in the Web GUI. Change the role, view or remove them as needed to test.

Loading a form set

Select add form on a persona, select type=set, cut and paste the content in the window OR add a url to the location. Click ‘Add’ to load them and update the view.

Form Set Example

This example contains multiple types of bots and the can be found at: http://api.sedro.xyz/w/00_DEV_00/forms/formset_weather.xml

<doc>
 <!-- http://localhost:8081/forms/formset_weather.xml -->
 
 <!-- The DATA we get --> 
 <form name='location weather' data-type='dataform' data-link='location=possesses'>
 <input name="location" data-type="input" data-thought="no"/>
 <input name="sunrise" data-type="input" data-alias='sunrise time'/>
 <input name="sunset" data-type="input" data-alias='sunset time'/>
 <input name="precipitation" data-type="input"/>
 <input name="aqi" data-type="input" data-alias='Air Quality Index'/>
 <input name="temperature" data-type="input"/>
 <input name="visibility" data-type="input"/>
 <input name="wind" data-type="input"/>
 <input name="conditions" data-type="input"/> 
 </form>
 
 <!-- input -->
 <form name='location' data-type='dataform'>
 <input name="latitude" data-type="input" value="%caller/latitude%"/>
 <input name="longitude" data-type="input" value="%caller/longitude%"/>
 </form>
 
 <!-- FORM to show the weather -->
 <form name='_show_weather_' data-type='form' >
 <input name="weather sum" data-type="content" />
 <label for="weather sum" state='wake'>Currently it is %current:location weather/conditions% %current:location weather/temperature% and in %current:location weather/location%</label> 
 <input name="weather sum1" data-type="content" />
 <label for="weather sum1" state='wake'>Sunrise is at %current:location weather/sunrise%</label> 
 </form>
 
 <!-- Explicit get and show -->
 <form name='_getweather_' data-type='form'>
 <!-- for location weather a request form OR here with longitude/latitued could be added -->
 <input name="get_weather" data-type="post" data-req-formname='location' data-resp-formname='location weather' data-posturl='http://localhost:8080/api/1.0/test/weather'/> 
 <input name="show report" data-type="form" data-formname="_show_weather_" data-task='weather report'/> 
 </form>
 

 <!-- ACTION -->
 <form name='weather' role='action' data-type='action' data-object-type='thing' data-ref="caller's weather">
 <input name="status" data-type="input"/> 
 <input name="read" data-type="post"
 data-req-formname='location' data-resp-formname='location weather' 
 data-task='show weather' data-task-post='weather report'
 data-ability='is, get, tell' 
 data-posturl='http://localhost:8080/api/1.0/test/weather'/>
 </form>
 <!-- Wake from background to tell the weather -->
 <form name='_weather_invoke_' data-type='form' >
 <input name="listen" data-type="invoke" data-seq="the/my:% X:weather:%"/> 
 <input name="intro" data-type="content" />
 <label for="intro" state='wake'>Getting the current weather</label> 
 <input name="weather report" data-type="form" data-formname="_getweather_"/>
 <input name="background" data-type="event" data-event='background_reset' />
 </form>
 
 <!-- Bot (or sub element) that just gets the weather -->
 <form name='_weather_bot_' data-type='form'>
 <input name="intro" data-type="intro"/>
 <label for="intro" state='wake'>Getting the current weather</label> 
 
 <input name="weather report" data-type="form" data-formname="_getweather_"/>
 
 <input name="outro" data-type="outro" />
 <label for="outro" state='wake' data-pre-wait='10' data-post-wait='10'>Good bye</label>
 
 <input name="bye" data-type="event" data-event='bye'/>
 </form>
 

 <!-- Bot: Could do more but mostly is weather -->
 <form name='_weather_taskbot_' role='main' data-type='form'>
 <input name="intro" data-type="intro"/>
 <label for="intro" state='wake'>Good %_time_period_of_day% how can I help you?</label>
 

 <input name="taskset" data-type="tasks" data-task-show='weather report' /> 
 <label for="taskset" state='wake' data-pre-wait='10'>What can I help you with this %_time_period_of_day%?</label>
 <label for="taskset" state='wake' data-channel='sms'>What do you want this time?</label>
 <label for="taskset" state='verify' data-pre-wait='10'>Now that %_last_task% is %_last_outcome%, is there anything else I can help you with?</label>
 <label for="taskset" state='canceled' data-pre-wait='6'>Is there anything else I can help you with?</label>
 

 <input name="outro" data-type="outro"/>
 <label for="outro" state='wake' data-pre-wait='10'>Thank you, hope you have a great %_time_period_of_day%</label> 
 
 <input name="bye" data-type="event" data-event='bye' />
 </form>
 </doc> 

In this Form Set _weather_taskbot_ acts as a conversational bot if set to main (the default), _weather_bot_ is a structured chatbot when set to main and weather if left as an action will automatically interact even with no main to get the weather. And lastly _weather_invoke_ if set to background will act as an element of a Digital Assistant to wake and get the weather.

With all of these in an enabled there will be conflicts though!

Our Next Posting

Our next posting will cover some of the information in this formset in more detail to help you build Conversation AI’s with The Sedro Project.

Digital Assistants via Background Forms

Digital Assistants via Background Forms

Forms may also work in the background providing an invocation from the foreground (like Apple’s Siri or Amazon’s Alexa) . Or they may stay in the background, recording/retrieving information or performing actions based on the conversation content.

Background forms can be used to add simple note takers, personal assistance, inline censor/editor or for more complex tasks that leverage integration with your external servers.

Adding a Background Form

Background Rorms are added like the others but with type set to background. Try this example to see a Background Form with an invokation.

 <form name='_invoke_' data-type='form' >
 <input name="listen" data-type="invoke" data-opt='wait' data-seq="X:woolley:%"/>
 
 <input name="intro" data-type="content" />
 <label for="intro" state='wake'>This is Woolley!</label>
 
 <input name="background" data-type="event" data-event='background_reset' />
 </form> 

This form will listen in conversation and when the caller says ‘woolley’ will be invoked. It simply prints a message and completes by returning to the background. This method allows for background monitoring with actions on command as needed for a personal or task assistant.

Invoking

Invoking a background form makes it active and starts it’s processing, this could take over the conversation from any existing form or do so when no forms are running. The invocation is done with an invoke input

 <input name="listen" data-type="invoke" data-opt='wait' data-seq="X:woolley:%"/>

The data-seq= is used to specify the rule that must match for to invoke the form and is in the SEQ (Semantic Sequence) Language. In this example it matches current token case independent to the string “woolley”.

data-opt=’wait‘ is set to tell the system not to auto-run the invoke, it must first wait for data.

Returning to the background

Events are used to return to the background or background_reset depending if you want this form to be reset to initial state or not.

<input name="background" data-type="event" data-event='background_reset' />

In parallel

Background and foreground forms may run together in the same Persona. This allows from more many different possibly unrelated tasks to take place in a conversation and allows for UX to be very complex.

Some SEQ examples

Not to go into the syntax here in any detail a few more examples are added so you can modify the invocation

hey:% X:woolley:% - "hey Wolley"
invoke:% X:woolley:% now:% - "invoke wolley now"
%any%:%action% X:woolley:% - "run wolley!" OR "I hate WOLLEY!"

These are simple, but should get you started matching content… full documentation is forthcoming

To Be Continued…

In the next blog on Forms we will focus on Rest API integration via Forms http://blog.sedro.xyz/2020/01/29/persona-forms-restapi-integration/

Chatbot RestAPI Integration

Chatbot RestAPI Integration

Most Chatbots will require external integration to get or change information and perform actions. The Sedro Project provides 2 methods for external integration 1) Form based explicit REST API interactions 2) Action handler Forms (covered in a future blog).

In this Blog post we will cover how to integrate API calls to perform actions and provide you with a live example.

Adding a RestAPI Post to a Form

An http POST can be added as an input inline in any form. Common case would be to retrieve a users account information or retrieve records from an external Database. The format is

 <input name="get_account" type="post" data-resp-formname="account" data-posturl="http://api.sedro.xyz/api/1.0/test/testformgetpost"/>

The form will be POSTed to the URL that is defined when the form reaches this input line. A request with parameters will be sent to the server in JSON and a JSON request in a standard format will be expected in response.

Request Params

The standard params for every post include some high level information that will allow external servers to link the correct user and context when it is needed.

  • ctok – caller token passed into the wake
  • chid – channel id
  • chtype – channel type
  • type – ‘input’ in all posts
  • lang – the language of the chatbot

The input parameters will be all the inputs in the current form, or you can specify a different form to define those parameters if needed. with the following attribute in the definition.

data-req-formname=request_form

Response Params

data-resp-formname=account

Specify the name of the form that defines the data to retrieve from the post. This form will need to be defined with the correct information, see Object as Form section below.

For a well organized set of forms, having a single form to format output is helpful, it can be reused in any case that needs to display the information. Also using %iterator:xxx% notation allows labels to show single or a member of a set.

JSON Request Format

The form of the request is standard aside from the params listed, which come from the form.

The request elements are the params passed in they are taken from the request form defined for the post. The form under lists the forms elements that are being requested to be filled in and returned, these come from the response form defined for the post

 {
 "ctok": "TEST_TOKEN_SEDRO"
 "chid": "40caee93-66a4-44e7-b466-dd38494e30a3",
 "chtype": "chat",
 "persona": "jane",
 "type": "input",
 "lang": "english",
 "request": {
    "elements": [
       {
          "val": "849392",
          "name": "account number"
       }
    ],
    "name": "_getexternaccount_"
    },
    "form": {
       "elements": [
          {
              "name": "full name",
              "type": "input"
           },
           {
              "name": "address",
              "type": "input"
           },
           {
              "name": "home phone number",
              "type": "input"
           },
           {
              "name": "account number",
              "type": "input"
           }
       ],
       "name": "account",
       "id": 23
    }
 } 

JSON Response Format

The response format is mostly the same as the request, but with the values added for each param.

 {
 "chtype": "chat",
 "request": {
    "elements": [
        {
           "val": "849392",
           "name": "account number"
        }
    ],
    "name": "_getexternaccount_"
 },
 "form": {
    "elements": [
       {
            "val": "Bart Simpson",
            "name": "full name"
        },
        {
            "val": "123 N. Alphabet St., Los Angeles, Ca 97045",
            "name": "address"
        },
        {
            "val": "(555) 345-6789",
            "name": "home phone"
        },
        {
            "val": "849392",
           "name": "account number"
        }
    ],
    "name": "account",
    "id": 23
 },
 "type": "data",
 "chid": "3c5f5025-e560-42b7-b172-68456dcb8768"
 } 

Multiple form response for a list can include information for pagination as well.

{
 "chtype": "chat",
 "request": {
    "elements": [
        {
           "val": "849392",
           "name": "account number"
        }
    ],
    "name": "_getexternaccount_"
 },
 "forms": [
{
    "elements": [
       {
            "val": "Bart Simpson",
            "name": "full name"
        },
        {
            "val": "123 N. Alphabet St., Los Angeles, Ca 97045",
            "name": "address"
        },
        {
            "val": "(555) 345-6789",
            "name": "home phone"
        },
        {
            "val": "849392",
           "name": "account number"
        }
    ],
    "name": "account",
    "id": 1
 },
{
    "elements": [
       {
            "val": "Hommer Simpson",
            "name": "full name"
        },
        {
            "val": "123 N. Alphabet St., Los Angeles, Ca 97045",
            "name": "address"
        },
        {
            "val": "(555) 345-6789",
            "name": "home phone"
        },
        {
            "val": "849391",
           "name": "account number"
        }
    ],
    "name": "account",
    "id": 2
 }
],
 "type": "data",
 "count": "2",
 "start": "1",
 "end": "2",
 "chid": "3c5f5025-e560-42b7-b172-68456dcb8768"
 } 

Server Response

The server called in the post must respond with the JSON format expected, the data in the response should be in the format of the ‘form’ passed in. Each value should be set as expected. If more that one form are filled out by the server a list of forms should be returned in the response.

Objects as Forms

For integration, managing API params, retrieved data and saved data a form may be created to represent an Object. It simply has a set of inputs with names to represent the params or fields of data.

 <form name='account' data-type='dataform' data-link='caller=possesses'>
 <input name="full name" data-type="input" data-alias='full name, account holder, account holder name'/>
 <input name="address" data-type="input" data-alias='home address, address, mailing address'/>
 <input name="home phone" data-type="input" data-alias='home phone number, home phone, phone number'/>
 <input name="account number" data-type="input" data-alias='acctnum' data-fmt='^[0-9]{6}$'/>
 </form> 

This form can then be used to as the Request Param and Response Param Form. This would get back a completed form with all the values filed in, which could be saved and referenced.

The name of each input is important, it must be in english and specify what the datum is, as this is interpreted by Sedro NLU so that it can learn the information, the alias= allows for additional names that this datum may be called to be added for better learning. For instance in the above example name=address and data-alias=home address, mailing address, this allows the Persona to know that this is the callers address, home address, and mailing address.

Attribute data-fmt is optional, on an input it is used to validate the value is ok, it is in standard regx. If there is a specific format to the datum, it is recommended that you add it.

Typing Message

While waiting for the post response a message may be sent to the caller, a typing message is most useful. Adding a wake message to the post will have it sent to the caller before the post. setting data-msg-event=’typing‘ will send a typing message, also add a pre-wait to ensure it is show.

<label for="get_account" state='wake' data-msg-event='typing' data-pre-wait='5'>Getting Account...</label>

Full Example

This example gets an account number from the caller, then gets the callers account information from the external server and allows the caller to view it.

 <form name='_getexternaccount_' data-type='form'>
 <input name="account number" data-type="input" data-seq="X:%any%:(lx_number):sz[large]" data-fmt='^[0-9]{6}$'/>
 <label for="account number" state='wake'>What is your account number?</label>
 <label for="account number" state='incomplete'>Account number is 6 digits</label>
 <label for="account number" state='verify'>Is %account number% correct?</label>
 <label for="account number" state='complete'>Retrieving account information...</label>
  
   <input name="get_account" data-type="post" data-resp-formname="account" data-posturl="http://api.sedro.xyz/api/1.0/test/testformgetpost"/>
<label for="get_account" state='wake' data-msg-event='typing' data-pre-wait='5'>Adding Account...</label>
 <label for="get_account" state='incomplete'>Still waiting for account information</label>
 
 <input name="account holder" data-type="content" />
 <label for="account holder" state='wake'>Account Holder: %current:account/full name%</label> 
 
 <input name="account addr" data-type="content" />
 <label for="account addr" state='wake'>Address: %current:account/address%</label> 
 
 <input name="account phone" data-type="content" />
 <label for="account phone" state='wake'>Phone: %current:account/home phone%</label> 
 </form> 

The information is also learned by the Persona, allowing it to answer many questions about the information on its own.

The Other Method

In the next blog post we will cover a bit about the command and action forms and integration. These perform operations similar to what other services do with intent.

To Be Continued…

In the next blog on Forms we will focus on Action forms

Chatbot Controls with Action Forms

Chatbot Controls with Action Forms

Action forms allow for automatic mapping to controls in forms. For example you could use an action form to map controls for a light, to turn it on/off up and down. With an Action form you define an object and what it can do, Sedro then will call the corresponding controls when they are requested in the text or chat.

There are many ways in language to ask for something simple, such as control for a light, it can come as a question, command or be ambiguous. Some examples:

Turn the lights on. 
Turn the lights up. 
Close the light. 
Hit the lights. 
Lights on! 
Please lower the lights. 
lights on. 
Dim the lights please.
please get the lights.
If the lights are off, turn them on.
lights brighter.
lights brightest, now!
Make the lights lower.
Will you please lower the lights?

Here we will use a light example to show what can be done with Action forms

Adding a Action Form / Object

Action forms can be used to define an object, such as a light or a kitchen light in this case. They are the same as all other forms, but are added as an ‘action’ form in the Persona Manager.

 <form name='kitchen light' data-type='action'
 data-classifier='light'  
 data-object-type='thing' 
 > ....
</form>

The basic action form has data-type=’action’ so the system knows to map this to objects in the ontology. data-classifier=’light’ is set to map the object type of this form/object, this is a light and will be added to the ontology as a light. data-object-type=’thing’ tells Sedro that this is a physical object. There are some optional attributes covered later that allow more explicit mapping

Adding Controls

To get the Form to respond in a chat we must add a control, which is just an input like in all forms. Here we add a comment control to send a message when we ask to open or close the light.

 <input name="answer" data-crmd='comment' data-ability='open, close' data-type="comment" />
 <label for="answer" state='wake'>You want me to %current/cmd_action% the %current/cmd_object_instance%?</label> 

Control Input attributes

This has a few new attributes added to it to allow it to map the specific abilities.

data-ability=’open, close’ defines abilities or actions that the form/object is capable of. If added to a form defines what an object can do, if added to an input maps that input to the ability.

data-crmd=’comment’ is used to tell sedro that this control is a comment; the full set of options are create/read/modify/delete/comment. read is the default.

data-mod=’up, down, brighter’ defines modifications that the object is capable of, if added to the object. If added to an input, the input is then mapped to these modifications.

data-state=’on, off, broken’ defines states that the object can be in or set to, if added to an object. If added to an input, the input is then mapped to these modifications.

REST API Control

To call a REST API an input is added of type post, just as in all other forms. Here we define an input to turn the light on, it has a few states and abilities added to it to make it an easy example to work with.

 <input name="on" data-crmd='modify'  data-ability='turn, raise, flip, toggle, switch'  data-state='on, bright, open'  data-status-value='10' data-status-op='set'  data-type="post" data-posturl='http://api.sedro.xyz/api/1.0/test/testformgetpost'/>  <label for="on" state='done'>Updated, the light is now %current/status%</label> 

Standard Params

Each API Call will have a set of system parameters that will provide more context, use them or ignore them. In addition any inputs added to the form (that are not controls) will also be passed in with the value set.

  • cmd_action – the action turn / flip / switch
  • cmd_mod – the modification on / off
  • cmd_object – object and object class: kitchen light / light
  • cmd_object_instance – instance object ‘kitchen light’
  • lang – language
  • ctok – caller token passed into chat or sedro instance
  • chid – channel id
  • status – (next section)

The Status value

The status parameter is auto filled by sedro based on the information it finds in the question or command. It will automatically use the meaning of the words to determine what value to set based on the type of status you need as specified the status input via attributes

  • data-value-type – has options of range/list/none
  • data-value-min – should be set to the minimum value if range
  • data-value-max – should be set to the maximum value if range

For controlling a light a range is used (dimmable) with a min of 0 and a max of 10. This will send to the api a status with val=’set’, ‘increment’, or ‘decrement’ and val2=0-10

Overriding Status Value

An input/control can override the auto setting of the status value with the follow attributes on the input

  • data-status-value=<the value to send> – value or string
  • data-status-op=<operation to perform> – set/increment/decrement

Form Attributes detailed

There are 2 additional attributes that can be added to a an action form to provide it mappings to alternate names, that are not already know to the language or are not working correctly.

  • data-alias=‘my kitchen light’ – specifies other phrases or words that will be globally equal to the name of the form/object
  • data-ref=‘big kitchen light’ – specifies non-global but possible alternates in context for the name of the form/object
  • data-synonym=’xxx’ – specifies one or more objects that will get the same controls, that are synonyms for this object

Optional Verification

Prior to executing the input a verification will requests user to confirm, just as in other forms. Add this to any control input in the form.

 <label for="on" state='verify'>You would like to %current/cmd_action% the lights?</label> 

NOTE: be cautious with these in non-interactive content, they may block further progress.

Full Example

The following example Action form is for controlling the kitchen light (it has some unneeded elements to aid as examples)

 <form name='kitchen light' data-type='action'
 data-classifier='light'  
 data-object-type='thing' 
 data-seq='%action% X:lights:%'  
 data-alias='my kitchen light' data-ref='big kitchen light'
 >
 <input name="status" data-type="input" data-value-type='range' data-value-min='0' data-value-max='10'/>
 <input name="location" data-type="input" value='kitchen'/> 
 
 <input name="answer" data-crmd='comment' data-ability='touch' data-type="comment" />
 <label for="answer" state='wake'>You want me to %current/cmd_action% the %current/cmd_object_instance%?</label>
 
 <input name="read" data-crmd='read' 
 data-ability='status, level, brightness, is'
 data-type="post" data-posturl='http://api.sedro.xyz/api/1.0/test/testformgetpost'/>
 <label for="read" state='done'>The light is %current/status%</label>
 
 <input name="modify" data-crmd='modify'
 data-mod='up, down, brighter, darker, dimmer, lower, higher'
 data-ability='turn, shut, dim, raise, hit, get, flip, toggle, switch'
 data-state='bright, dark, dim, high, low'
 data-type="post" data-posturl='http://api.sedro.xyz/api/1.0/test/testformgetpost'/>
 <label for="modify" state='verify'>You would like to %current/cmd_action% the lights?</label>
 <label for="modify" state='done'>Updated, the light is now %current/status%</label>
 
 <input name="on" data-crmd='modify'
 data-ability='turn, raise, flip, toggle, switch'
 data-state='on, bright, open'
 data-status-value='10' data-status-op='set'
 data-type="post" data-posturl='http://api.sedro.xyz/api/1.0/test/testformgetpost'/>
 <label for="on" state='verify'>You would like to %current/cmd_action% the lights?</label>
 <label for="on" state='done'>Updated, the light is now %current/status%</label>
 
 <input name="off" data-crmd='modify'
 data-ability='turn, shut, dim, flip, toggle, switch'
 data-state='off, dark, closed'
 data-status-value='0' data-status-op='set'
 data-type="post" data-posturl='http://api.sedro.xyz/api/1.0/test/testformgetpost'/>
 <label for="off" state='verify'>You would like to %current/cmd_action% the lights?</label>
 <label for="off" state='done'>Updated, the light is now %current/status%</label>
 </form> 

This can be added to a persona like any other and the POST actions mapped. Make changes to the abilities and modifiers to see how it works in practice.

To Be Continued…

In the next blog on Forms we will focus on filter input that allows for lists of input parameters to be gathered, and modified by text.

Chatbot Control Events

Chatbot Control Events

Events are added to a Persona Form to perform specific inline tasks, such as resetting the form, closing the connection or saving the data. Here we cover the syntax for each and provide an example.

General syntax

Events are always <input> in a <form> and must have a name and data-type=’event’

 <input name="test_event" data-type="event data-event='bye' /> 

Additionally every event needs data-event set to one of the documented events.

  • data-event=event type
  • data-dbname=name of database
  • data-posturl=url

Most events have limits on the type of form they can be used in: load, background, or action. This will be noted.

bye

Closes the connection with a bye

<input name="test_event" data-type="event data-event='bye' data-opt='start' />

use in form: load

reset

resets the form back to start, clears inputs

<input name="test_event" data-type="event data-event='reset' data-opt='start' />

use in form: load / background / action

exit

reset and return to the caller form if any

<input name="test_event" data-type="event data-event='exit' data-opt='start' />

use in form: load / background / action

save

saves the form instance to the context, then resets form

<input name="test_event" data-type="event data-event='save' data-opt='start' />

use in form: load / background / action

save_db

Saves the form and its inputs as an instance of the object in the DB, then resets form

<input name="test_event" data-type="event data-event='save_db' data-opt='start' data-dbname='default'/>

use in form: load / background / action

save_post

POSTS the form and its inputs to defined URL, then resets form

<input name="test_event" data-type="event data-event='save_post' data-opt='start' data-posturl='http://api.sedro.xyz/api/depots/testformdatapost' />

use in form: load / background / action

background

switch Active form to background again

<input name="test_event" data-type="event data-event='background' data-opt='start' />

use in form: background

background_reset

switch Active from to background form, then resets form

<input name="test_event" data-type="event data-event='background_reset' data-opt='start' />

use in form: background

activate

for background Form, this will make it the active form.

<input name="test_event" data-type="event" data-event='activate' data-opt='start' />

use in form: background

To Be Continued…

In the next blog on Forms we will focus on Background Forms http://blog.sedro.xyz/2020/01/29/background-forms/

Chatbot Interaction with Forms & Sub-Forms

Chatbot Interaction with Forms & Sub-Forms

Forms can be added to other forms, create a tree with any level of depth… much like a DMV IVR. In this blog we go through the details of how to use Persona Sub-Forms in The Sedro Project.

Sub forms can be added in any other form, they simply run inline when the form gets to the sub-form input. This allows forms to be created for specific inputs and allow reuse.

Sub-Forms

The simplest is a form called in the processing flow of another form. In this case a Form to get the callers phone number is added to the persona

 <form name='_getphonenumber_' data-type='form'>
 <input name="phone number" data-type="input" data-seq="X:%any%:(lx_phone number):sz[large]"/>
 <label for="phone number" state='wake'>What is your phone number?</label> 
 <label for="phone number" state='incomplete'>Full phone number please, including area code</label>
 <label for="phone number" state='verify'>Is %phone number% correct?</label>
 <label for="phone number" state='complete'>got %phone number%</label>
 </form> 

Another Form is added to the Persona and set as Main Form, that includes an input of type ‘form’ that links to Sub-Form _getphonenumber_

 <form name='_getinfo_' data-type='form'>
       <input name="intro" data-type="content" />
       <label for="intro" state='wake'>We need a bit of information from you</label>
  
       <input name="home phone" data-type="form" data-formname="_getphonenumber_"/>           
       
       <input name="exit" data-type="content" />
       <label for="exit" state='wake'>Thank you for your information.</label>              
       <input name="bye" data-type="event" data-event='bye' />
 </form> 

Inline inclusion of a remote form works exactly the same but starts with a URL.

 <input name="test_form" data-type="form" data-formname='http://api.sedro.xyz/w/00_DEV_00/forms/testform.xml'/> 

Menu Sub-Forms

Menu’s with multiple option can also link to local or remote forms on the selection of an option. This allows your service to nest menu’s as is common in an IVR or ordering system. In this example we have 3 options, get phone number with the previous form, get the test form from remote (allowing personalization) or go back to the last form.

 <form name='_submenu_' data-type='menu'>
 <input name="intro" data-type="content"/>
 <label for="intro" state='wake'>Select an option from this list</label>
 
 <input name="select" data-type="menu"/>
 <label for="select" state='option' value='1' data-formname='_getphonenumber_'>1, Get Phone Number</label>
 <label for="select" state='option' value='2' data-formname='http://api.sedro.xyz/w/00_DEV_00/forms/testform.xml'>2, Test Form</label>
 <label for="select" state='option' value='3' data-formname='cancel'>Go back</label>
 <label for="select" state='incomplete'>Please select an option for the list</label>
 <label for="select" state='done'>You selected option %select% %select_txt%</label>
 </form> 

Accessing Form input values

Accessing values from the form can be done via input name of the form from the caller

 <label for="exit" state='wake'>Your Home Phone Number: %home phone%</label> 

Or before the form is reset you can access the value

 <label for="exit" state='wake'>Your Phone number: %current:_getphonenumber_/phone number%</label> 

If the form is reset you can access any of the inputs using the ‘current’ for the name of the instance, this will also work if only one has been saved, or will get the last one saved.

 <label for="exit" state='wake'>Your Phone number: %current:_getphonenumber_/phone number%</label> 

Accessing saved forms can be done using the form ID

 <label for="exit" state='wake'>Your Phone number: %9393:_getphonenumber_/phone number%</label> 

Accessing Database Values

Saving forms to the Data Base allows you to access the information during the session or beyond if you retain the information across chat sessions. This also allows for multiple copies of the same time of information, but each will need a distinct name.

 <label for="exit" state='wake'>Your Phone number: %db:my_number:phone number/phone number%</label> 

Complex User Interfaces

Sub-Forms can be used to develop complex and dynamic user interfaces for chat or voice allowing any level of control desired. Mixed with the ability of Sedro to answer questions on its own, you can quickly develop a functional and human like interface.

To Be Continued…

In the next blog on Forms we will focus on more complex events http://blog.sedro.xyz/2020/01/27/persona-form-events/

Getting started with Chatbots

Getting Started with Chatbots and Forms

Persona Forms are used by The Sedro Project as a tool to manage and map content to and conversations. Forms are written in simple XML much like an HTML form. They allow the definition of inputs, labels, content validation, events and a bit more.

Each Persona may have Forms added to it to manage input, output, or integrate with external APIs and databases. Form management can be done via the APIs or via the persona manager that is a thin layer on top of the same APIs, but a bit easier to work with http://api.sedro.xyz/extern/mgt

Since the format is much like HTML forms so it will be a bit familiar, additionally you can embedded in them directly HTML pages (style=’display:none’ perhaps), this allows a web page to provide visual interaction as well as chat or audio.

Hello World Chatbot

A simple Hello World Chatbot example provides the basic structure for the form: name, data-type with inputs and labels.

 <form name='_helloform_' data-type='form'>
       <input name="intro" data-type="content" />
       <label for="intro" state='wake'>Hello from your form!</label>
 </form> 

Form names should follow the convention of _name_ for interaction or action forms (so that your persona does not find it and think it is an object).

Main Form

The main form for a Persona is a form that runs at the time the persona wakes. To make a Form main, just set it as the main form.

Now you can add the Hello World example as main when you submit it. Then Try it by click ‘Chat’ on the persona from the Persona Manager.

Extending hello world form

Now the Form is extended to add another line to the introduction, this is done by adding another line with the same state. Then conversation closing text is added via another input and labels. After that a bye event is used to close the conversation.

They bye event is added as an input with data-type=event and data-event=bye this forces the chatbot to hangup.

 <form name='_helloworld2_' data-type='form'>
       <input name="intro" data-type="content" />
       <label for="intro" state='wake'>Hello World!</label>
       <input name="intro2" data-type="content" />
       <label for="intro2" state='wake'>(from your form)</label>
             
       <input name="exit" data-type="content" />
       <label for="exit" state='any'>That is all.</label>  
       <input name="exit2" data-type="content" />
       <label for="exit2" state='any'>Good Bye</label>     
                         
       <input name="bye" data-type="event" data-event='bye' />
 </form> 

Remote Forms

Forms may be in the persona or remote. Remote forms are fetched on demand via HTTP GET from the provided URL. This allows for simpler management of the chatbot flow and as well as per user / etc customization.

In the Persona Manager adding a remote form will add a main form that is remote. The URL provided will be pulled to the persona each time it wakes.

The URL to retrive the remote Form will contain some parameters to allow dynamic creation.

  • ctok = the caller_token passed into the wake API call
  • lang = the language of the conversation
  • chtype = the channel type for the conversation (chat=online chat)

Test a remote form, from the Persona Manager add a form and set the URL with our example http://api.sedro.xyz/w/00_DEV_00/testform.xml

Try it by clicking ‘Chat’ on the persona… It also has a few more features.

Inputs

In this Form an input is added to collect the caller/chatter’s title. Additionally it ask them to verify we have the correct answer and complains if they send junk. Then it uses it in the context of the conversation.

 <form name='_helloform_' data-type='form'>
       <input name="intro" data-type="content" />
       <label for="intro" state='wake'>Hello from the form!</label>
       
       <input name="title name" data-type="input" data-seq="X:%any%:(lx_role_name):sz[large]"/>
       <label for="title name" state='wake'>What is your Title?</label>
       <label for="title name" state='incomplete'>Your job title or personal title</label>
       <label for="title name" state='verify'>You are really a %title name%?</label>
       <label for="title name" state='nomatch'>%title name% isn't a title!</label>
       <label for="title name" state='complete'>Never met a %title name% before</label>
       
       <input name="exit" type='hidden' data-type="content" />
       <label for="exit" state='any'>That is all.</label>
       <input name="exit2" type='hidden' data-type="content" />
       <label for="exit2" state='any'>Good Bye</label>
  
       <input name="bye" data-type="event" data-event='bye' />
 </form>

The value of the input can be aded in phrases by wrapping the input name with a ‘%’’s example:  %title name%. This makes it easy to use the current values in follow on messages.

In the “title name” labels we have added some additional states that are available for inputs;

  • wake – starting state, use for opening label
  • incomplete – still waiting for a valid response
  • verify – check if the current response is what they want to use
  • nomatch – if content is not a match, question or command
  • complete – done with the input

This input has an additional data-seq= attribute, this is used to correctly identify and extract the information from the response they give us and uses the SEQ language (documented elsewhere)

Menus

Menus are the most common interaction for many voice systems (think phone IVR), they are often used in text based chatbots as well. The menu is always a complete form, it can be linked as the main form or as a sub-form from any other (more on that in documentation and later post)

 <form name='_testmenu_' data-type='form'>
 <input name="intro" data-type="content"/>
 <label for="intro" state='wake'>Select an option from this list</label>
 
 <input name="select" data-type="menu"/>
 <label for="select" state='option' value='1'>option 1</label>
 <label for="select" state='option' value='2'>option 2</label>
 <label for="select" state='option' value='3'>option 3</label>
 <label for="select" state='option' value='4' data-formname='cancel'>Go back</label>
 <label for="select" state='incomplete'>Please select an option or cancel</label>
 <label for="select" state='verify'>You would like option %select% %select_txt%?</label>
 <label for="select" state='done'>You selected option %select% %select_txt%</label>
 
 <input name="exit" data-type="content" />
 <label for="exit" state='wake'>Thank you</label>
 </form> 

The menu is created as a menu type input. Labels are then added for each option with the state=’option’ set and a value set for the simple option value.

 <label for="select" state='option' value='1'>option 1</label>

Matching the value= or the text for the label will match the option for the caller.

Once a response is selected the done label is displayed. The phrase can include the value and or label for the selected option using the % method and _txt% for the label. If a verify message is added (or multiple via conditional labels) the verification of the selection will take place, just as it does on a standard input.

Conditional labels

Many times the response for an input or other interaction needs to be dependent on the response data. A data-seq=’xxx’ option on a label allows selection from a set of responses based on the input using SEQ language definitions to match content.

 <input name="full name" data-type="input" data-seq="X:%any%:(lx_person_name):sz[large]"/>
 <label for="full name" state='wake'>What is your full name?</label>
 <label for="full name" state='incomplete'>Your full legal name please</label>
 <label for="full name" state='verify'>Is %full name% your full name?</label>
 <label for="full name" state='verify' data-seq='P:%any%:length[1]'>%full name%, you have only 1 name?</label>
 <label for="full name" state='complete'>nice to meet you %full name%.</label> 

The full name is desired in this example and verification is added if a single name is added instead of the expected first and last (and possibly more). A data-seq=”P:%any%:length[1]’ has been added that will display this message if the token length == 1 for the input (Parent in SEQ terms) .

Channel Type specific Labels

The content from a label without a channel type specified is the default, it will be used on all channel types. chat, sms, email, voice, etc. If you would like a specific channel type to have different content, just add another label with attribute data-channel-type=’xxx’ specified.

<label for="full name" state='complete'>nice to meet you %full name%.</label>
<label for="full name" state='complete' data-channel-type='sms'>Hello %full name%.</label>

In some cases you may want no content for a specific channel type, set the content to ‘%’ to accomplish that.

<label for="full name" state='complete' data-channel-type='sms'>%</label>

Message Delays

Waits before or after a message allow better control over the user experience, providing time to separate content as in a natural conversation. Each label may have a wait before the content data-pre-wait=”time” and after the content with data-post-wait=”time”. Time is in tenths of a second so 10 = 1 second.

<label for="full name" state='complete' data-pre-wait='5' data-post-wait='20'>nice to meet you %full name%.</label>

Get started

Now is a good time to get started with The Sedro Project to try Forms and Persona based chatbots. http://www.sedro.xyz

To Be Continued…

In the next blog on Forms we will focus on Sub-Forms http://blog.sedro.xyz/2020/01/26/persona-forms-sub-forms/

Person Manager start

Setting up a Persona

In order to use Chat, Ask, or Tell APIs you will need a persona to communicate with.

Setup is easy if you use the Persona Management web page provided. First you must subscribe to the API in RapidAPI, then you will need to copy your API Key. Open the Persona Manager at http://api.sedro.xyz/extern/mgt then past the key.

  1. Sedro Account Information – list of persona’s and subscription limits.
  2. Add a Persona – The added first name will be the persona name. Your subscription will limit how many you may have and how long they will be retained.
  3. Add some Knowledge – So that the Persona knows something. This information can be added multiple times here or added and extended via the tell API.
  4. Start a Chat – The Persona Chat button will start a test Chat

Chatbot Conversation Control and Integration

In The Sedro Project Persona Forms are used to manage conversations as well as to integrate commands and request with external services via RestAPIs.

More information on forms is provides in the API documentation, Form Documentation and will be covered in a later Blog Post.

Content Generation

The API for chat and ask respond with extended information to allow you to generate text on your own or use the response that we provided. Check the docs and the JSON in the response to get more information on this.

Security Concerns with your API Key?

Many of you may be concerned about submitting your Rapid API Key to some random site, understood. Alternate options

1) Use our REST APIs directly to do the same setup (see api documentation/Rapid API)

2) Download our Persona Manager and run it on your local machine or in your web service. You will need to modify the include paths in mgt.html to correspond with your .js placements.

Simple JavaScript API for the REST Apis:

http://api.sedro.xyz/w/00_DEV_00/js/extern/api_sedro.js

Implementation of the GUI

http://api.sedro.xyz/w/00_DEV_00/js/extern/mgt.js

The HTML for the GUI

http://api.sedro.xyz/w/00_DEV_00/extern/mgt.html

To Be Continued…

Continue reading our blog to about Persona Forms for more details and examples on how to build functional chatbots, personal assistants and Language Interface with The Sedro Project.

Chat with Fiona

NLU Question and Answer

NLU Question and Answer

Questions are automatically identified in all content (with or without proper punctuation) and evaluated to determine the proper answer. Much of the time the answer will not be known, some of the time it will be very generic (from ontology ancestry). But many of the questions will be answered simply and concisely.

The method used to get answers doesn’t need training, It just need to have the information. The easiest way is to provide the information in text to it, just as you would to a person so that they could answer the question.

Example:

Shrek is a big green ogre. He hates people and he lives in a swamp. Shrek likes eating. He once said "what are you doing in my swamp!". Is Shrek green? 
Shrek is big?
What is Shrek?
What does shrek like doing?
Does Shrek like eating?
What does Shrek do in a swamp?
What did Shrek say?

Scan down to the sentence for each question, you will see ‘Answer’ button in the sentence button bar. Click this to expand for the Answer information.

This displays the Answer type, Question we are answering (composited in cases of incomplete information) and the response options. Green means is very certain of the answer, Orange is undecided, Red means no idea.

Chat Question and Answer

The same learning occurs in chat conversations, the persona learns what you say, or what it gets from external APIs. The following is the same as the above example but with the Persona giving the answers to the questions after the information is provided.

As with the first example, information is provided and Sedro is able to use it to answer questions; in this case it is just via a persona and in a chat flow.

Chat with retained knowledge

In Sedro the information from text may be saved and loaded as a knowledge set, thus allowing plain text to provide information and be loaded for a chatbot or data mining later. The knowledge can be continually extended in the same way, with text additions and the /tell API.

In Analyze the same content is given then saved to a DB name, this DB name is added to the Persona ‘Fiona’. Using RapidAPI apis this is done by adding knowledge or using tell.

Some responses may be less detailed when from memory than in conversation (just as IRL). For example Shrek will be just ‘ogre’ instead of ‘big green ogre’ as comes from the conversation.

Try it for yourself

Try it via the live demo, or via the Apis at http://api.sedro.xyz/api-current

The 1st, 2nd and 3rd Person Perspective in NLP

The 1st, 2nd and 3rd Person Perspective in NLP

Natural Language Processors must identify the correct Grammatical person for words, objects and phrases so that it can correctly map references and utilize the information correctly. Here we are going to take a look at how they are mapped and how Sedro tags the information and exports it via API.

Generally personal pronouns are considered for grammatical person, such as in this case of 3rd person ‘He’.

   It is his dog

Person also comes into play for lists, objects and couples such as in this 3rd person use of ‘they’:

   Linus, Lucy and Charles said they want to go trick-or-treating

Sedro maps person simply, or at least for its ability to map information references. It tags tokens and objects as either inclusive of the speaker (includes the speaker) or exclusive of the speaker (does not include the speaker). This also allows reference matching to more complex objects such as lists, couples and named objects.

The 1st Person

First person items are marked as inclusive, and have a type set to person.

   usive=inc, rtype=person

The personal pronouns are well known some are:

   [I, me, my, us, we] => inclusive

Lists and couples may also be first person and will be marked the same

   Linus, Lucy and I => inclusive

3rd Person

Third person items are marked as exclusive, and have type set to person and set to thing for objects.

   usive=exc, rtype=person/thing

The personal pronouns are well known some are:

   [he, she, it, they, them, that] => exclusive

Lists and couples may also be first person and will be marked the same

   Linus, Lucy and Charles => exclusive

2nd Person

Second person like third person are marked as exclusive, have type set to person but then also have direct set.

   usive=exc, dir=dir, rtype=person

The personal pronouns are well known some are:

   [you, y’all, thee] => exclusive, direct

Quantity

The last bit of information needed for grammatical person is the quantity which differentiates between single, plural and all. This bit of information is exported via API as the quant and is set when known with one of the listed values.

   quant=[single, plural, dual, all]

The quant allows reference mapping decisions for lists, groups or across like quantity objects.

There are other uses for the inclusive / exclusive information as well, that don’t relate direction to person. More on that perhaps in a later post.