diff --git a/docs/docs/concepts/policies.mdx b/docs/docs/concepts/policies.mdx index 0cee35039d98..c01ad9cb4f12 100644 --- a/docs/docs/concepts/policies.mdx +++ b/docs/docs/concepts/policies.mdx @@ -10,10 +10,19 @@ import RasaProLabel from "@theme/RasaProLabel"; import RasaProBanner from "@theme/RasaProBanner"; In Rasa, policies are the components responsible for dialogue management. -In NLU-based assistants, dialogue policies play a larger role. -You can read more about those policies and other advanced topics [here](../nlu-based-assistants/policies.mdx). - In a CALM-based assistant, the `FlowPolicy` is responsible for executing your business logic. +If you're building an NLU-based assistant, you can read about the relevant +policies [here](../nlu-based-assistants/policies.mdx). + + +:::note Policies in CALM + +Because the [Dialogue Understanding](./dialogue-understanding.mdx) component in CALM already accounts +for the context of a conversation, the role of the dialogue manager is simpler than it is an +an NLU-based assistant. + +::: + You can customize the policies your assistant uses by specifying the `policies` key in your project's `config.yml`. @@ -36,7 +45,7 @@ policies: At every turn, each policy defined in your configuration gets a chance to predict a next [action](./actions.mdx) with a certain confidence level. -A policy can also not predict an action. +A policy can also decide not to predict any action. The policy that predicts with the highest confidence decides the assistant's next action. :::note Maximum number of predictions @@ -47,7 +56,7 @@ to the desired number of maximum predictions. ::: -### Flow Policy +## Flow Policy :::info New in 3.7 @@ -56,99 +65,73 @@ The _Flow Policy_ is part of Rasa's new with version `3.7.0`. ::: -The Flow Policy is a state machine that allows you to manage and control your -assistant's conversational flows. It facilitates the integration of your business -logic with machine learning capabilities. +The Flow Policy is a state machine that deterministically executes +the business logic defined in your [flows](./flows.mdx). The Flow Policy oversees your assistant's state, handles state transitions, and -triggers new flows when needed. The "Dialogue Stack" represents all started but -incomplete flows and refers to steps defined in the -[flows specification](../concepts/flows.mdx). - -Using the Flow Policy, you can encapsulate your logic into flows, thereby -guiding the conversation based on user input and system response. Other -classifiers adjust the underlying state machine to handle edge cases. - -For more detailed information on defining flows, refer to the -[flows specification](../concepts/flows.mdx). +starts new flows when required for [Conversation Repair](./conversation-repair.mdx). -## Adding the FlowPolicy to your bot +### Adding the Flow Policy to your assistant -The Flow Policy can be integrated like [any other policy](../nlu-based-assistants/policies.mdx). -It involves two steps: +To use the Flow Policy, add it to the list of policies in your `config.yml`. -1. Configure the `FlowPolicy` in your `config.yml` file: - - ```yaml-rasa title="config.yml" - policies: - - name: rasa.core.policies.flow_policy.FlowPolicy - ``` +```yaml-rasa title="config.yml" +# ... -2. Add flows to your assistant as per the [flows specification](../concepts/flows.mdx). This - step involves defining the states and transitions that your bot should - recognize and react to. +policies: + - name: FlowPolicy +``` -Once added, the `FlowPolicy` will read all defined flows from your training data -and apply them. +The Flow Policy does not have any additional configuration parameters. -## How does it work? +### How does it work? The `FlowPolicy` employs a dialogue stack structure (Last In First Out) along with -internal slots to manage your bot's state. +internal slots to manage the state of a conversation. ### Managing the State -Your bot's state is managed using a "dialogue stack". The dialogue stack stores the -current position in each started flow. The stack follows a "last in, first out" -sequence. The last started flow will be completed first, and then earlier -inserted flows will continue. The dialogue stack is stored in the `dialogue_stack` slot. +The Flow Policy manages state using a "dialogue stack". +Whenever a flow is started, it is pushed on to the dialogue stack, and this stack +keeps track of the current position in each of those flows. +The dialogue stack follows a "last in, first out" sequence, meaning that the flow +that was started most recently will be completed first. +Once it has completed, the next most recently started flow will continue. -Here's an example flow and a conversation currently using it: +Consider the `transfer_money` flow from the [tutorial](../tutorial.mdx): ```rasa-yaml title="flows.yml" flows: transfer_money: - description: This flow enables users to send money to friends and family. - + description: | + This flow lets users send money to friends + and family, in US Dollars. steps: - - id: "ask_recipient" - collect_information: transfer_money_recipient - next: "ask_amount" - - id: "ask_amount" - collect_information: transfer_money_amount_of_money - next: "check_transfer_funds" - # - ... + - collect: recipient + - collect: amount + - action: utter_transfer_complete ``` -The first step in the flow is `ask_recipient`. A correction by the user, _"oh -sorry I meant John"_, initiates a correction flow. This correction is not -explicitly modeled in the `transfer_money` flow; it's handled by a -conversational pattern: - -The correction flow only executes when the user wants to correct a previous -input. Once triggered, it's placed on top of the dialogue stack for execution. - -For the `transfer_money` flow, the correction flow is an interruption. Once -completed, the `transfer_money` flow resumes. Before its res umption, a -`resume_flow` flow is activated. You can customize what happens in `resume_flow` -and `correction` flows in [Conversation Repair](./conversation-repair.mdx). - -Upon completion of the `resume_flow` flow, the `transfer_money` flow resumes, -and the bot returns to the state before the correction flow started. The -`transfer_money` flow will re-ask the interrupted question of a -`collect_information` step. - -The `resume_flow` and `correction` flows are conversational patterns not -explicitly modeled in the `transfer_money` flow. The `FlowPolicy` automatically -triggers them when needed. For more details on defining conversational patterns, -refer to [Conversation Repair](./conversation-repair.mdx). - -### Transitioning Between States - -The `FlowPolicy` transitions between states either in response to user inputs or -after an action's completion. Initially, it awaits user input. Once received, it -advances the state of the machine until an action prediction is made. After -completing the action and incorporating its result, it moves to the next state. +When the conversation reaches a `collect` step, your assistant will ask the user for +information to help it fill the corresponding slot. In the first step, your assistant +will say *"Who would you like to send money to?"* and then wait for a response. + +When the user responds, the [Dialogue Understanding](./dialogue-understanding.mdx) component +will generate a sequence of commands which will determine what happens next. +In the simplest case, the user says something like *"to Jen"* and the command +`SetSlot("recipient", "Jen")` is generated. + +The flow has collected the information it needed and the flow policy proceeds to the next step. +If instead the user says something like *"I want to send 100 dollars to Jen"*, the commands +`SetSlot("recipient", "Jen"), SetSlot("amount", 100)` will be generated, and the flow +policy will skip directly to the final step in the flow. + +There are many things a user might say *other than* providing the value of the slot your +assistant has requested. +They may clarify that they didn't want to send money after all, ask a clarifying question, +or change their mind about something they said earlier. +Those cases are handled by +[Conversation Repair](./conversation-repair.mdx). ### Starting New Flows @@ -161,27 +144,9 @@ A flow can be started in several ways: - One flow can ["link" to another flow](../concepts/flows.mdx#link), which will initiate the linked flow and return to the original flow once the linked flow completes. -- Flows, which are conversational patterns (e.g. correction, resume flow), are - automatically initiated by the `FlowPolicy` when needed. - -### Interruptions and Digressions - -Flows and their specifications should ideally model the happy path of a -conversation. The `FlowPolicy` depends on other components to identify -digressions and interruptions and interrupt the flow accordingly. For details on -which digressions and interruptions the flow policy can handle, refer to -[Conversation Repair](./conversation-repair.mdx). - -For example, the user message _"oh sorry I meant John"_ is a correction to a -prior input. The `FlowPolicy` depends on another component, like the -[LLM Command Generator](../concepts/dialogue-understanding.mdx), to identify it as a -correction and set the corresponding slots. Based on this, the `FlowPolicy` -initiates the correction flow. +- In the case of [Conversation Repair](./conversation-repair.mdx), default flows can be + automatically added by the `FlowPolicy`. -## Configuration - -No additional configuration is required for the FlowPolicy beyond what's already -outlined above. ## FAQs @@ -191,7 +156,7 @@ Currently, we recommend not using the `FlowPolicy` in conjunction with the `RulePolicy`. The `RulePolicy` isn't aware of the dialogue stack and may interfere with the `FlowPolicy`. -### DocSearch Policy +## DocSearch Policy @@ -206,7 +171,7 @@ generate a response based on the prompt. --TODO-- -## How to Use Docsearch in Your Bot +### How to Use Docsearch in Your Bot To use docsearch, add the following lines to your `config.yml` file: @@ -222,7 +187,7 @@ The `source` parameter specifies the directory containing your documentation. The `DocsearchPolicy` will automatically index all files with a `.txt` extension in this directory (recursively) and uses them to generate responses. -## Customization +### Customization You can customize the LLM by modifying the following parameters in the `config.yml` file. @@ -368,7 +333,7 @@ The following threat vectors should be considered: More detailed information can be found in Rasa's webinar on [LLM Security in the Enterprise](https://info.rasa.com/webinars/llm-security-in-the-enterprise-replay). -### Intentless Policy +## Intentless Policy @@ -378,41 +343,46 @@ The _Intentless Policy_ is part of Rasa's new Conversational AI with Language Models (CALM) approach and available starting with version `3.7.0`. ::: -The new intentless policy leverages large language models (LLMs) to complement -existing rasa components and make it easier: - -- to build assistants without needing to define a lot of intent examples -- to handle conversations where messages - [don't fit into intents](https://rasa.com/blog/were-a-step-closer-to-getting-rid-of-intents/) - and conversation context is necessary to choose a course of action. - -Using the `IntentlessPolicy`, a -question-answering bot can already understanding many different ways -that users could phrase their questions - even across a series of user messages: - -This only requires appropriate responses to be defined in the domain file. - -To eliminate hallucinations, the policy only chooses which response from -your domain file to send. It does not generate new text. - -In addition, you can control the LLM by: - -- providing example conversations (end-to-end stories) which will be used in the prompt. -- setting the confidence threshold to determine when the intentless policy should kick in. - -[This repository](https://github.com/RasaHQ/starter-pack-intentless-policy) contains a starter pack with a bot that uses the -`IntentlessPolicy`. It's a good starting point for trying out the policy and for -extending it. - -## Demo - -[Webinar demo](https://hubs.ly/Q01CLhyG0) showing that this policy can already -handle some advanced linguistic phenomena out of the box. - -The examples in the webinar recording are also part of the end-to-end tests -defined in the [example repository](https://github.com/RasaHQ/starter-pack-intentless-policy) in (`tests/e2e_test_stories.yml`). - -## Adding the Intentless Policy to your bot +The Intentless Policy is used to send `responses` which +are not part of a flow. +This can be helpful for handling chitchat, contextual questions, +and high-stakes topics. + +### Chitchat +In an enterprise setting it may not be appropriate to use +a purely generative model to handle chitchat. +Teams want to ensure that the assistant is always on-brand and on-message. +Using the Intentless Policy, you can define vetted, human-authored `responses` +that your assistant can send. +Because the Intentless Policy leverages LLMs and considers the whole context of the conversation +when selecting an appropriate `response`, it is much more powerful +than simply predicting an intent and triggering a fixed response to it. + +### High-stakes Topics +In addition to chitchat, another common use case for the Intentless Policy is +providing answers on high-stakes topics. +For example, if users have questions about policies, legal terms, or guarantees, +like: _"My situation is X, I need Y. Is that covered by my policy?"_ +In these cases the DocSearchPolicy's RAG approach is risky. +Even with the relevant content present in the prompt, a RAG approach allows an +LLM to make interpretations of documents. +The answer users get will vary depending on the exact phrasing of their question, +and may change when the underlying model changes. + +For high-stakes topics, it is safest to send a self-contained, vetted answer +rather than relying on a generative model. +The Intentless Policy provides that capability in a CALM assistant. +Note that conversation design is crucial in these cases, as you want your responses +to be self-contained and unambiguous rather than just "yes" or "no". + +### Interjections +When a flow reaches a `collect` step and your assistant asks the user for information, +your user might ask a clarifying question, refuse to answer, or otherwise interject in +the continuation of the flow. +In these cases, the Intentless Policy can contextually select appropriate `responses`, +while afterwards allowing the flow to continue. + +### Adding the Intentless Policy to your bot The `IntentlessPolicy` is part of the `rasa_plus` package. To add it to your bot, add it to your `config.yml`: @@ -423,163 +393,21 @@ policies: - name: rasa_plus.ml.IntentlessPolicy ``` -## Customization - -### Combining with NLU predictions - -The intentless policy can be combined with NLU components which predict -intents. This is useful if you want to use the intentless policy for -some parts of your bot, but still want to use the traditional NLU components for -other intents. - -The `nlu_abstention_threshold` can be set to a value between 0 and 1. If -the NLU prediction confidence is below this threshold, the intentless policy -will be used if it's confidence is higher than the NLU prediction. Above the -threshold, the NLU prediction will always be used. - -The following example shows the default configuration in the `config.yml`: - -```yaml-rasa title="config.yml" -policies: - # ... any other policies you have - - name: rasa_plus.ml.IntentlessPolicy - nlu_abstention_threshold: 0.9 -``` - -If unset, `nlu_abstention_threshold` defaults to `0.9`. - -### LLM / Embeddings - -You can customize the openai models used for generation and embedding. - -#### Embedding Model - -By default, OpenAI will be used for embeddings. You can configure the -`embeddings.model_name` property in the `config.yml` file to change the used -embedding model: - -```yaml-rasa title="config.yml" -policies: - # ... any other policies you have - - name: rasa_plus.ml.IntentlessPolicy - embeddings: - model_name: text-embedding-ada-002 -``` - -Defaults to `text-embedding-ada-002`. The model name needs to be set to an -[available embedidng model.](https://platform.openai.com/docs/guides/embeddings/embedding-models). - -#### LLM Model - -By default, OpenAI is used for LLM generation. You can configure the -`llm.model_name` property in the `config.yml` file to specify which -OpenAI model to use: - -```yaml-rasa title="config.yml" -policies: - # ... any other policies you have - - name: rasa_plus.ml.IntentlessPolicy - llm: - model_name: text-davinci-003 -``` - -Defaults to `text-davinci-003`. The model name needs to be set to an -[available GPT-3 LLM model](https://platform.openai.com/docs/models/gpt-3). - -If you want to use Azure OpenAI Service, you can configure the necessary -parameters as described in the -[Azure OpenAI Service](./components/llm-configuration.mdx#additional-configuration-for-azure-openai-service) -section. - -#### Other LLMs / Embeddings - -By default, OpenAI is used as the underlying LLM and embedding provider. - -The used LLM provider and embeddings provider can be configured in the -`config.yml` file to use another provider, e.g. `cohere`: - -```yaml-rasa title="config.yml" -policies: - # ... any other policies you have - - name: rasa_plus.ml.IntentlessPolicy - llm: - type: "cohere" - embeddings: - type: "cohere" -``` - -For more information, see the -[LLM setup page on llms and embeddings](./components/llm-configuration.mdx#other-llmsembeddings). - -### Other Policies - -For any rule-based policies in your pipeline, set -`use_nlu_confidence_as_score: True`. Otherwise, the rule-based policies will -always make predictions with confidence value 1.0, ignoring any uncertainty from -the NLU prediction: - -```yaml-rasa title="config.yml" -policies: - - name: MemoizationPolicy - max_history: 5 - use_nlu_confidence_as_score: True - - name: RulePolicy - use_nlu_confidence_as_score: True - - name: rasa_plus.ml.IntentlessPolicy -``` - -This is important because the intentless policy kicks in only if the other -policies are uncertain: +As with all components which make use of LLMs, you can configure which provider +and model to use, as well as other parameters. +All of those are described [here](./components/llm-configuration.mdx). -- If there is a high-confidence NLU prediction and a matching story/rule, the - `RulePolicy` or `MemoizationPolicy` will be used. -- If there is a high-confidence NLU prediction but no matching story/ rule, the - `IntentlessPolicy` will kick in. +### Steering the Intentless Policy -- If the NLU prediction has low confidence, the `IntentlessPolicy` will kick in. +The Intentless Policy can often choose the correct response in a zero-shot fashion. +That is, without providing any example conversations to the LLM. -- If the `IntentlessPolicy` prediction has low confidence, the `RulePolicy` will - trigger fallback based on the `core_fallback_threshold`. - -**What about TED?** - -There is no reason why you can't also have TED in your configuration. However, - -- TED frequently makes predictions with very high confidence values (~0.99) so - will often override what the `IntentlessPolicy` is doing. -- TED and the `IntentlessPolicy` are trying to solve similar problems, so your - system is easier to reason about if you just use one or the other. - -## Steering the Intentless Policy - -The first step to steering the intentless policy is adding and editing responses -in the domain file. Any response in the domain file can be chosen as an response -by the intentless policy. This whitelisting ensures that your assistant can -never utter any inappropriate responses. - -```yaml-rasa title="domain.yml" -utter_faq_4: - - text: - We currently offer 24 currencies, including USD, EUR, GBP, JPY, CAD, AUD, - and more! -utter_faq_5: - - text: - Absolutely! We offer a feature that allows you to set up automatic - transfers to your account while you're away. Would you like to learn more - about this feature? -utter_faq_6: - - text: - You can contact our customer service team to have your PIN unblocked. You - can reach them by calling our toll-free number at 1-800-555-1234. -``` - -Beyond having the `utter_` prefix, the naming of the utterances is not relevant. - -The second step is to add +However, you can improve the performance of the policy by adding example conversations. +To do this, add [end-to-end stories](../nlu-based-assistants/training-data-format.mdx#end-to-end-training) -to `data/e2e_stories.yml`. These stories teach the LLM about your domain, so it -can figure out when to say what. +to `data/e2e_stories.yml` to your training data. +These conversations will be used as examples to help the Intentless Policy learn. ```yaml title="data/e2e_stories.yml" - story: currencies @@ -600,85 +428,8 @@ can figure out when to say what. - action: utter_faq_11 ``` -The stories and utterances in combination are used to steer the LLM. The -difference here to the existing policies is, that you don't need to add a lot of -intent examples to get this system going. - -## Testing - -The policy is a usual Rasa Policy and can be tested in the same way as any other -policy. - -### Testing interactively - -Once trained, you can test your assistant interactively by running the following -command: - -```bash -rasa shell -``` - -If a flow you'd like to implement doesn't already work out of the box, you can -add try to change the examples for the intentless policy. Don't forget that you -can also add and edit the traditional Rasa primitives like intents, entities, -slots, rules, etc. as you normally would. The `IntentlessPolicy` will kick in -only when the traditional primitives have low confidence. - -### End-to-End stories - -As part of the beta, we're also releasing a beta version of a new End-To-End -testing framework. The `rasa test e2e` command allows you to test your bot -end-to-end, i.e. from the user's perspective. You can use it to test your bot in -a variety of ways, including testing the `IntentlessPolicy`. - -To use the new testing framework, you need to define a set of test cases in a -test folder, e.g. `tests/e2e_test_stories.yml`. The test cases are defined in a -similar format as stories are, but contain the user's messages and the bot's -responses. Here's an example: - -```yaml title="tests/e2e_test_stories.yml" -test_cases: - - test_case: transfer charge - steps: - - user: how can I send money without getting charged? - - utter: utter_faq_0 - - user: not zelle. a normal transfer - - utter: utter_faq_7 -``` - -**Please ensure all your test stories have unique names!** After setting the -beta feature flag for E2E testing in your current shell with -`export RASA_PRO_BETA_E2E=true`, you can run the tests with -`rasa test e2e -f tests/e2e_test_stories.yml` - -## Security Considerations - -The intentless policy uses the OpenAI API to create responses. -This means that your users conversations are sent to OpenAI's servers. - -The response generated by OpenAI is not send back to the bot's user. However, -the user can craft messages that will misslead the intentless policy. These -cases are handled gracefully and fallbacks are triggered. - -The prompt used for classification won't be exposed to the user using prompt -injection. This is because the generated response from the LLM is mapped to -one of the existing responses from the domain, -preventing any leakage of the prompt to the user. - -More detailed information can be found in Rasa's webinar on -[LLM Security in the Enterprise](https://info.rasa.com/webinars/llm-security-in-the-enterprise-replay). - -## FAQ - -### What about entities? - -Entities are currently not handled by the intentless policy. They have to still -be dealt with using the traditional NLU approaches and slots. - -### What about custom actions? -At this point, the intentless policy can only predict utterances but not custom -actions. Triggering custom actions needs to be done by traditional policies, -such as the rule- or memoization policy. +### Testing -## Configuring Policies +Writing [end-to-end tests](../production/testing-your-assistant.mdx#end-to-end-testing) +allows you to evaluate the performance of the Intentless Policy and to guard against regressions. \ No newline at end of file