YAML general overview
NLP engine configuration using YAML
General YAML file format
For each NLP engine project, we define new set of flow configuration files, and the chatbot platform itself stays the same. All models and installations are made based on the configuration files.
One project can be defined by one or more YAML files with extension .yaml (or .yml). Yaml is a standardized format described in many sources such as:
General YAML syntax
In NLP engine, all YAMLs are structured in root level as dictionaries (mappings). The dictionary keys in NLP engine are always strings, the values in the root level are usually dictionaries.
Strings in YAML can be wrapped in quotes, preferred form for keys is without wrapping. Following entries are equivalent (and the first one recommended):
Comments
Strings
String values are recommended to be wrapped into double quotes.
Integers and floats
Numbers are used in configuration section. In other sections, string are used instead.
Lists
Dictionaries (mappings)
Ordered dictionaries (equal to list of lists)
Merging YAML files
If more YAML files are provided and they have same substructure with the same keys, the values are merged if possible.
Example for two files A and B:
File A:
File B:
Merged A and B:
Example for two incompatible files C and D:
File C:
File B:
Cannot merge two strings with different values.
Configuration
YAML configuration section
Following settings must be present at least in one YAML file of the given project.
All model settings are withing configuration key:
Example of filled values:
Parameters description: : 1. version (str): version of flow configuration. 2. start_node (str): The code of graph node in conversation flow in which the conversation starts. 3. allow_jumping (bool): If allowed, the user can leave conversation tree and starts new topic which is accessible from start_node. If forbidden, the user can visit only graph nodes allowed in conversation flow or nodes presented in allow_jumping_to_nodes. 4. allow_jumping_to_nodes: List of intents which can be jumped to from another intents (confidence of such intent prediction have some penalty to prefer intents in trees). 5. language: model is trained using data for given language code (tested for cs, sk and en). 6. description: notes about this project, not used in code 7. project: project code, will be used for folder naming in Linux 8. classifier: configuration of NLU module: > * 8.1. tokenizer (str): default value nist > * 8.2. aspell (replace/duplicate/whitelist/null): > one of values replace (replace misspelled words by their correct forms), > duplicate (add correct form of misspelled word to the end of utterance), > whitelist (use only whitelist corrections, ignore aspell, very good for speech channel) and > null (no correction and accentuation) > * 8.3. stopwords (bool): remove stopwords from text (e.g. the, be and and). 9. train: models for deep neural network, two parameters allowed in configuration: > * 9.1. epoch (int): how long will the training takes (recommended number between 3 and 15) > * 9.2. lr (float): learning rate, how fast will neural network learn, must be positive number 10. intent_threshold (float): if intent confidence is lower than this value, : return I_DONT_UNDERSTAND intent instead 11. intent_relative_threshold (float): if the first suggested intent confidence : is not higher than the second suggested intent confidence multiplied by this value, return I_DONT_UNDERSTAND intent instead 12. speed_coefficient (float): smaller value, smaller delay between messages sent by chatbot to : user, 1 gives enough time for average user to read everything before the next message is displayed. 13. max_delay_milliseconds (int): max. delay between messages sent by chatbot in milliseconds 14. debug (bool): if set for True, deployment will not fail on extractors which were not implemented yet (such : extractors will be replaced by one that return nothing)
How does the threshold work?
Test is performed only if more than one intent is considered.
Intent resolver predicts <0, n> pairs of original confidences and labels (oc1, label1), (oc2, label2), …. Original confidence is a number in range <0, 1> and the sum of original confidences is in range <0, inf).
Flow cuts off unreachable nodes and nodes not passing EXTRACTors entry conditions.
Also, we work with SSI (semantically same intent) groups. The rule is following:
while the label with the highest score is in the same SSI group as the second-best node, merge scores of these nodes and keep the label of winner
Example intent resolver scores:
is converted to:
and also normalized confidence is computed (the sum of normalized confidences is 1):
After merging SSI nodes and computing normalized score, the threshold tests follow:
original threshold test:
If the first label original confidence is less than original threshold, no intent is selected.
With
original_intent_threshold=0.001, the labelyes_plainpasses and test continues.With
original_intent_threshold=0.005, the labelyes_plainfails and no intent is selected.normalized threshold test:
If the first label normalized confidence is bigger than intent_threshold, the first label is selected, otherwise, the next test follows.
with
intent_threshold=0.8, the labelyes_plainpasses and is returned (intent is selected)with
intent_threshold=0.9, the labelyes_plainfails and next test followsrelative threshold test:
If the best label confidence is X times bigger than the second-best label confidence, the best label is accepted and intent is selected.
with
intent_relative_threshold=10, the labelyes_plainpasses (0.067 * 10 < 0.87), intent is selectedwith
intent_relative_threshold=20, the labelyes_plainfails (0.067 * 20 > 0.87), no intent is selected
Designing the conversation flow
Flow attribute
In flow attribute, the conversation flow is represented using YAML syntax. To express the conversation in YAML, the flow is simplified to the oriented graph with nodes and edges (actions, conditions, gotos).
Nodes are usually named with CAPITAL_ASCII letters and underscores.
Edges of the graphs are: : * intents/user actions
automatic transitions based on conditions (based on system state)
simple automatic transitions
The conversation graph always starts in node defined in
Nodes properties
Node properties order:
EXTRACT: entry check calls before entering NODE, enter only if all EXTRACT calls receive non-None response
CALL: system backend calls (interface agreements calls)
RESET: unset variables
SET: set variables
1, 2, …: produce outputs
CONDITIONS: try to change node to the first fulfilled condition
ACTIONS_SIGNALS: handle signals sent instead of user input
ACTIONS: wait for user utterance, after that, change node or continue
exclusive fallbacks (all variants are mutually exclusive):
GOTO: fallback
GOTO_REUSE_UTTERANCE: fallback leading to another intent resolving
GOTO_WAIT: fallback waiting for user action, signal or empty message, but do not consume the message
GOTO_BACK: fallback leading to already visited node
EXTRACT
Check if node can be accessed by calling existing_function_name_X. If node is selected by IntentResolver as the best candidate and at the same time, return value of call existing_function_name_X is not None (null), assign value to variable var2fill_no_X.
EXTRACT Roles
Extract attributes will be always designed by BornDigital team.
If EXTRACT attribute is added to YAML by BornDigital, all visual tools should be able to display it and preserve the information
EXTRACT Visualization example
CALL
Call some function from interface agreement (existing_function_name_X). Set the response to variable (var2fill_no_X).
CALL Roles
Not displayed in visualization or displayed for business users.
Will be added to YAML using interface agreement markdown.
CALL Visualization example
Extensions documentation
SET
Set system, local and other variables for current conversation_id (session id). To write
SET Roles
All sets should be present in logic.
SET Visualization example
RESET
Remove content of some variable, usually some help variable. Also used when skipping to another part of logic.
Remove content of variables a, b and c:
Delete all variables:
Delete all variables except a and b and c:
Approaches above cannot be combined.
RESET Roles
All resets should be present in logic.
RESET Visualization examples
1, 2, 3, …
Produces outputs.
Text output:
Speech output:
Mixed channels:
Extra data for frontend and/or widgets:
We can use placeholders to add variables:
1, 2, 3, … Roles
Defined in logic.
ACTIONS
Wait in the node until user produces next utterance (speech or text). After utterance is received, process it by IntentResolver. If some available intent matches the utterance, the state changes from current NODE_NAME_X to new NODE_NAME_Y.
ACTIONS Roles
All actions should be present in logic.
ACTIONS Visualization example
ACTIONS_SIGNALS
Signal can be sent in special cases like too much noise in the voice channel to inform the customer that he/she should repeat the sentence in the more calm environment. After the signal is received the state changes from current NODE_NAME to new NODE_NAME_A.
ACTIONS_SIGNALS Roles
All signal nodes should be present in logic.
no_input: Customer is silent (his/her microphone is not catching any sound). The recommended reaction is to check whether is customer still listening and ready to continue current conversation.
no_match: Customer speech is unintelligible (too much noise in the background; speaker is cold; speaker is foreigner with bad pronunciation; bad articulation). The recommended reaction is to ask customer to rephrase his utterance slowly and intelligibly.
call_end_customer_hangup: Customer ended conversation prematurely. After this, the data storage should be cleaned and extra follow-up actions planned.
ACTIONS_SIGNALS Visualization example
Generic intents documentation
CONDITIONS
Try conditions from top to bottom, the first fulfilled condition is activated and the state is changed to the node assigned to the condition. The last condition can be True which replaces GOTO fallback.
Version with fallback:
CONDITIONS Roles
All conditions should be present in logic.
CONDITIONS Visualization example
Version with fallback:
GOTO
Fallback, if there is no other option, travel from current node to GOTO destination.
GOTO Roles
All gotos should be present in logic.
GOTO Visualization example
GOTO_REUSE_UTTERANCE
Fallback, if there is no other option, travel from current node to GOTO_REUSE_UTTERANCE destination. Also, the utterance written/said by user can be modified reused later. Please be mindful of the fact that input utterance is also processed. That means corrector and transformer changes are reapplied to it.
GOTO_REUSE_UTTERANCE Python code
It can operate with one variable original_utterance and can use regex Python library re. It must have non-empty value (empty value would make GOTO_REUSE_UTTERANCE same as GOTO), it can use newlines (the return value will be used from last line).
GOTO_REUSE_UTTERANCE Roles
All tags should be present in logic.
GOTO_REUSE_UTTERANCE Visualization example
GOTO_WAIT
Fallback, wait here until next API call. The API call can contain user utterance, signal or be empty.
Can be used for situations we:
need to ignore user input (user is waiting, music is playing; waiting for backend call response; etc.):
want to use only regular expressions and variables for routing and need to turn intents off for this decision:
GOTO_STOP
Fallback, stops user and sends response from interaction with preset timeout variable set to 0.
Can be used identically to GOTO_WAIT above, however when working in tandem with voice server a new interaction is initiated due to timeout value.
Interaction with GOTO_REUSE_UTTERANCE
GOTO_WAITis activated by utterance,GOTO_REUSE_UTTERANCEkeeps that utterance
GOTO_WAITis activated by signal or empty message,GOTO_REUSE_UTTERANCEkeeps utterance empty (None)
GOTO_WAIT Visualization example
Clever fallback for I_DONT_UNDERSTAND and ERROR
To modify your flow to support fallbacks, several changes need to be implemented. Firstly, you need to find all nodes needing fallbacks:
“I do not understand” fallbacks should be present for every node that: : 1. Contains property ACTIONS and do not contain property GOTO 2. Contains property ACTIONS and contains property GOTO
“Error” fallback should be present for every node that: : 1. Contains property CONDITIONS and do not contain property GOTO
Examples of situations and solutions:
is transformed to:
is transformed to:
is transformed to:
Now we need to add newly mentioned nodes:
For actions (we will use extractor counter which automatically increment variable by one each time we enter the node):
For conditions:
Handling meaningless inputs with signals
In some conversations, GoodBot can receive a short meaningless inputs. Commonly a speech engine catches noises in the background (music, car engine, other people are talking) or user writes a typo (a cat walks over the keyboard, children seize the phone, etc.). In such cases, the GoodBot receives input such as pp, hjk or mm.
This feature allows to recognize such noises and replace them by a signal defined in the configuration yaml in the section:
To turn off this feature, set
falseornullin the configuration section:
Of course, it is necessary to have signals set up in GoodBot. Let’s have look at an example of a node named ARE_YOU_CONTENT with a definition for the signal catching meaningless inputs:
And example of a node in which the feature won’t work:
Last updated
