Add Docs on Creating and Using External Actions

Author: srhinos

docs/mint.json

@@ -72,6 +72,7 @@
         "open-source/langchain-agent",
         "open-source/action-agents",
         "open-source/action-phrase-triggers",
+        "open-source/external-action",
         "open-source/local-conversation",
         "open-source/events-manager",
         "open-source/using-synthesizers",

docs/open-source/external-action.mdx

@@ -0,0 +1,250 @@
+---
+title: "External Actions"
+description: "Have your agent communicate with an External API"
+---
+
+External Actions allow Vocode agents to take actions outside the realm of a phone call. In particular, Vocode agents can decide to _push_ information to external systems via an API request, and _pull_ information from the API response in order to:
+
+1. change the agent’s behavior based on the pulled information
+2. give the agent context to inform the rest of the phone call
+
+## How it Works
+
+### Configuring an External Action with `ExecuteExternalActionVocodeActionConfig`
+
+The Vocode Agent will determine after each turn of conversation if its the ideal time to interact with the External API based primarily on the configured External Action's `description` and `input_schema`!
+
+#### Overview
+
+- `processing_mode`: while it only supports `muted` right now, the intent is for this field to (in the future) enable hold music while processing the request, or to enable the agent to continue speaking until the request returns.
+
+- `name`: the name of the external action -- this has no impact on the functionality of action itself.
+
+- `description`: Used by the Function Calling API to determine when to make the External Action call itself. See the [`description` field section below](/open-source/external-action#description-field) for more info!
+
+- `url`: The API request is sent to this URL in the format
+  defined below in [Responding to External Action API Requests](/open-source/external-action#input-schema-field)
+
+- `input_schema`: A [JSON Schema](https://json-schema.org/) object that instructs how to properly form a payload to send to the External API. See the [`input_schema` field section below](/open-source/external-action#responding-to-external-action-api-requests) for more info!
+
+- `speak_on_send`: if `True`, then the underlying LLM will generate a message to be spoken into the phone call as the
+  API request is being sent.
+
+- `speak_on_receive`: if `True`, then the Vocode Agent will invoke the underlying
+  LLM to respond based on the result from the API Response or the Error encountered.
+
+- `signature_secret`: a base64 encoded string to enable request validation, see the [Signature Validation section](/open-source/external-action#signature-validation) below for more info
+
+#### `input_schema` Field
+
+The `input_schema` field is a [JSON Schema](https://json-schema.org/) object that instructs how to properly form a payload to send to the External API.
+
+For example, in the [Meeting Assistant Example](/open-source/external-action#meeting-assistant-example) below we formed the following JSON payload:
+
+```json
+{
+  "type": "object",
+  "properties": {
+    "length": {
+      "type": "string",
+      "enum": ["30m", "1hr"]
+    },
+    "time": {
+      "type": "string",
+      "pattern": "^d{2}:d0[ap]m$"
+    }
+  }
+}
+```
+
+This is stating the External API is expecting:
+
+- Two fields
+  - `length` (string): either "30m" or "1hr"
+  - `time` (string): a regex pattern defining a time ending in a zero with `am`/`pm` on the end ie: `10:30am`
+
+<Card title="💡 Note" color="#ca8b04">
+    If you’re noticing that this looks very familiar to OpenAI function calling, it is! Vocode treats OpenAI Function Calling as a first-class standard when the agent uses an OpenAI LLM.
+
+    The lone difference is that the top level `input_schema` JSON schema must be an `object` - this is so we can use JSON to send over parameters to the user’s API.
+
+</Card>
+
+#### `description` Field
+
+The `description` is best used to descibe your External Action's purpose. As its passed through directly to the LLM, its the best way to convey instructions to the underlying Vocode Agent.
+
+For example, in the [Meeting Assistant Example](/open-source/external-action#meeting-assistant-example) below we want to book a meeting for 30 minutes to an hour so we set the description as `Book a meeting for a 30 minute or 1 hour call.`
+
+<Card title="💡 Note" color="#ca8b04">
+  The `description` field is passed through and heavily affects how we do our
+  function decisioning so we recommend treating it in the same way you would a
+  prompt to an LLM!
+</Card>
+
+### Responding to External Action API Requests
+
+Once an External Action has been created, the Vocode Agent will issue API requests to the defined `url` during the course of a phone call based on the [configuration noted above](/open-source/external-action#configuring-the-external-action)
+The Vocode API will wait a maximum of _10 seconds_ before timing out the request.
+
+In particular, Vocode will issue a POST request to `url` with a JSON payload that matches `input_schema` , specifically (using the [Meeting Assistant Example](/open-source/external-action#meeting-assistant-example) below):
+
+```bash
+POST url HTTP/1.1
+Accept: application/json
+Content-Type: application/json
+x-vocode-signature: <encoded_signature>
+
+{
+	"payload": {
+	  "length": "30m",
+	  "time": "10:30am"
+  }
+}
+```
+
+#### Signature Validation
+
+A cryptographically signed signature of the request body and a randomly generated byte hash is included as a header (under `x-vocode-signature`) in the outbound request so that the user’s API can validate the identity of the incoming request. The signature secret is used to sign the request and is used to ensure the validity of the `x-vocode-signature` field.
+
+This should be set as a base64-encoded string and we recommend a longer length as well, using the following snippet as an example:
+
+```python
+import os
+import base64
+
+signature_secret = base64.b64encode(os.urandom(32)).decode()
+```
+
+Use the following code snippet to check the signature in an inbound request:
+
+```python
+import base64
+import hashlib
+import hmac
+
+async def test_requester_encodes_signature(
+request_signature_value: str, signature_secret: str, payload: dict
+):
+"""
+Asynchronous function to check if the request signature is encoded correctly.
+
+    Args:
+        request_signature_value (str): The request signature to be decoded.
+        signature_secret (str): The signature to be decoded and used for comparison.
+        payload (dict): The payload to be used for digest calculation.
+
+    Returns:
+        None
+    """
+    signature_secret_as_bytes = base64.b64decode(signature_secret)
+    decoded_digest = base64.b64decode(request_signature_value)
+    calculated_digest = hmac.new(signature_secret_as_bytes, payload, hashlib.sha256).digest()
+    assert hmac.compare_digest(decoded_digest, calculated_digest) is True
+
+```
+
+#### Response Formatting
+
+Vocode expects responses from the user’s API in JSON in the following format:
+
+```python
+Response {
+	result: Any
+	agent_message: Optional[str] = None
+}
+```
+
+- `result` is a payload containing the result of the action on the user’s side, and can be in any format
+- `agent_message` optionally contains a message that will be synthesized into audio and sent back to the phone call (see [Configuring the External Action](/open-source/external-action#configuring-the-external-action) above for more info)
+
+In the [Meeting Assistant Example](/open-source/external-action#meeting-assistant-example) below, the user’s API could return back a JSON response that looks like:
+
+```json
+{
+  "result": {
+    "success": true
+  },
+  "agent_message": "I've set up a calendar appointment at 10:30am tomorrow for 30 minutes"
+}
+```
+
+## EA Local Response Server Example:
+
+The following is an example of a quick start to enable testing external actions locally.
+
+Running `fastapi dev app.py` will run the server @ `http://127.0.0.1:8000` and can be used for external actions locally!
+
+<CodeGroup>
+```python app.py
+import time
+
+from fastapi import FastAPI
+from pydantic import BaseModel
+
+app = FastAPI()
+
+class ExternalActionRequest(BaseModel):
+class Payload(BaseModel):
+length: str
+time: str
+
+    call_id: str
+    payload: Payload
+
+@app.post("/external_action")
+def update_item(external_action_request: ExternalActionRequest):
+print(f"Received request:\n{external_action_request.model_dump_json(indent=2)}")
+time.sleep(3)
+return {"result": {"success": True}}
+
+````
+```txt requirements.txt
+pydantic==2.*
+fastapi==0.111.*
+````
+
+</CodeGroup>
+
+## Meeting Assistant Example:
+
+This is an example of how to configure a Meeting Assistant Action which will attempt to book a meeting for 30 minutes or an hour at any time ending in a zero (ie 10:30am is okay but 10:35am is not)
+
+```python Python
+import json
+import base64
+import os
+
+from vocode.streaming.action.execute_external_action import (
+    ExecuteExternalAction,
+    ExecuteExternalActionVocodeActionConfig,
+)
+
+ACTION_INPUT_SCHEMA = {
+    "type": "object",
+    "properties": {
+        "length": {
+            "type": "string",
+            "enum": ["30m", "1hr"],
+        },
+        "time": {
+            "type": "string",
+            "pattern": "^\d{2}:\d0[ap]m$",
+        },
+    },
+}
+
+action_config = {
+    "name": "Meeting_Booking_Assistant",
+    "description": "Book a meeting for a 30 minute or 1 hour call.",
+    "url": "http://example.com/booking",
+    "speak_on_send": True,
+    "speak_on_receive": True,
+    "input_schema": json.dumps(ACTION_INPUT_SCHEMA),
+    "signature_secret": base64.b64encode(os.urandom(32)).decode(),
+}
+
+action = ExecuteExternalAction(
+    action_config=ExecuteExternalActionVocodeActionConfig(**action_config),
+)
+```