From 39752368b6ffa79d77f147d6ace26a0ecf41161c Mon Sep 17 00:00:00 2001 From: ChengZi Date: Mon, 13 Jan 2025 15:09:02 +0800 Subject: [PATCH] integrate_with_phidata Signed-off-by: ChengZi --- .../integration/integrate_with_phidata.ipynb | 302 ++++++++++++++++++ 1 file changed, 302 insertions(+) create mode 100644 bootcamp/tutorials/integration/integrate_with_phidata.ipynb diff --git a/bootcamp/tutorials/integration/integrate_with_phidata.ipynb b/bootcamp/tutorials/integration/integrate_with_phidata.ipynb new file mode 100644 index 000000000..89ec4511f --- /dev/null +++ b/bootcamp/tutorials/integration/integrate_with_phidata.ipynb @@ -0,0 +1,302 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Integrate Milvus with Phidata\n", + "\n", + "[Phidata](https://github.com/phidatahq/phidata/tree/main) is a powerful framework for building intelligent agents and workflows. It allows you to create multi-modal agents that can understand text, images, audio, and video, and leverage various tools and knowledge sources to accomplish complex tasks. Phidata supports multi-agent orchestration, enabling teams of agents to collaborate and solve problems together. It also provides a beautiful Agent UI for interacting with your agents.\n", + "\n", + "Milvus vector database enable efficient storage and retrieval of information as embeddings. With Milvus and Phidata, you can easily integrate your knowledge into your Agent workflows. This document is a basic guide on how to use Milvus integration with Phidata." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Preparation\n", + "Install the necessary dependencies:" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "vscode": { + "languageId": "shellscript" + } + }, + "outputs": [], + "source": [ + "! pip install --upgrade phidata pymilvus openai" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "> If you are using Google Colab, to enable dependencies just installed, you may need to **restart the runtime** (click on the \"Runtime\" menu at the top of the screen, and select \"Restart session\" from the dropdown menu)." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We will use OpenAI as the LLM in this example. You should prepare the [api key](https://platform.openai.com/docs/quickstart) `OPENAI_API_KEY` as an environment variable." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "\n", + "os.environ[\"OPENAI_API_KEY\"] = \"sk-***********\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Initalize Milvus\n", + "\n", + "Import the packages and initialize the Milvus vector database instance." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "from phi.agent import Agent\n", + "from phi.knowledge.pdf import PDFUrlKnowledgeBase\n", + "from phi.vectordb.milvus import Milvus\n", + "\n", + "# Initialize Milvus\n", + "vector_db = Milvus(\n", + " collection=\"recipes\",\n", + " uri=\"./milvus.db\",\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Specify the collection name and the uri and token(optinal) for your Milvus server.\n", + "\n", + "Here is how to set the uri and token:\n", + "\n", + "> - If you only need a local vector database for small scale data or prototyping, setting the uri as a local file, e.g.`./milvus.db`, is the most convenient method, as it automatically utilizes [Milvus Lite](https://milvus.io/docs/milvus_lite.md) to store all data in this file.\n", + "> - If you have large scale of data, say more than a million vectors, you can set up a more performant Milvus server on [Docker or Kubernetes](https://milvus.io/docs/quickstart.md). In this setup, please use the server address and port as your uri, e.g.`http://localhost:19530`. If you enable the authentication feature on Milvus, use \":\" as the token, otherwise don't set the token.\n", + "> - If you use [Zilliz Cloud](https://zilliz.com/cloud), the fully managed cloud service for Milvus, adjust the `uri` and `token`, which correspond to the [Public Endpoint and API key](https://docs.zilliz.com/docs/on-zilliz-cloud-console#cluster-details) in Zilliz Cloud.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load data\n", + "\n", + "Create a PDF url knowledage base instance and load the data into the instance. We use a public recipe pdf data as an example." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
INFO     Creating collection                                                                                       \n",
+       "
\n" + ], + "text/plain": [ + "\u001b[34mINFO \u001b[0m Creating collection \n" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
INFO     Loading knowledge base                                                                                    \n",
+       "
\n" + ], + "text/plain": [ + "\u001b[34mINFO \u001b[0m Loading knowledge base \n" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
INFO     Reading: https://phi-public.s3.amazonaws.com/recipes/ThaiRecipes.pdf                                      \n",
+       "
\n" + ], + "text/plain": [ + "\u001b[34mINFO \u001b[0m Reading: \u001b[4;94mhttps://phi-public.s3.amazonaws.com/recipes/ThaiRecipes.pdf\u001b[0m \n" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
INFO     Added 0 documents to knowledge base                                                                       \n",
+       "
\n" + ], + "text/plain": [ + "\u001b[34mINFO \u001b[0m Added \u001b[1;36m0\u001b[0m documents to knowledge base \n" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Create knowledge base\n", + "knowledge_base = PDFUrlKnowledgeBase(\n", + " urls=[\"https://phi-public.s3.amazonaws.com/recipes/ThaiRecipes.pdf\"],\n", + " vector_db=vector_db,\n", + ")\n", + "\n", + "knowledge_base.load(recreate=False) # Comment out after first run" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Use agent to response to a question\n", + "Integrate the knowledge base into an agent, then we can ask the agent a question and get a response." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "4b528ef9435248688320a110c3aab0e4", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Output()" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n"
+      ],
+      "text/plain": []
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "# Create and use the agent\n",
+    "agent = Agent(knowledge_base=knowledge_base, use_tools=True, show_tool_calls=True)\n",
+    "\n",
+    "# Query the agent\n",
+    "agent.print_response(\"How to make Tom Kha Gai\", markdown=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "┏━ Message ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
+    "┃                                                                                                                                                             ┃\n",
+    "┃ How to make Tom Kha Gai                                                                                                                                     ┃\n",
+    "┃                                                                                                                                                             ┃\n",
+    "┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛\n",
+    "┏━ Response (6.9s) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
+    "┃                                                                                                                                                             ┃\n",
+    "┃ Running:                                                                                                                                                    ┃\n",
+    "┃                                                                                                                                                             ┃\n",
+    "┃  • search_knowledge_base(query=Tom Kha Gai recipe)                                                                                                          ┃\n",
+    "┃                                                                                                                                                             ┃\n",
+    "┃ Here's a recipe for Tom Kha Gai, a delicious Thai chicken and galangal soup made with coconut milk:                                                         ┃\n",
+    "┃                                                                                                                                                             ┃\n",
+    "┃ Ingredients (One serving):                                                                                                                                  ┃\n",
+    "┃                                                                                                                                                             ┃\n",
+    "┃  • 150 grams chicken, cut into bite-size pieces                                                                                                             ┃\n",
+    "┃  • 50 grams sliced young galangal                                                                                                                           ┃\n",
+    "┃  • 100 grams lightly crushed lemongrass, julienned                                                                                                          ┃\n",
+    "┃  • 100 grams straw mushrooms                                                                                                                                ┃\n",
+    "┃  • 250 grams coconut milk                                                                                                                                   ┃\n",
+    "┃  • 100 grams chicken stock                                                                                                                                  ┃\n",
+    "┃  • 3 tbsp lime juice                                                                                                                                        ┃\n",
+    "┃  • 3 tbsp fish sauce                                                                                                                                        ┃\n",
+    "┃  • 2 leaves kaffir lime, shredded                                                                                                                           ┃\n",
+    "┃  • 1-2 bird’s eye chilies, pounded                                                                                                                          ┃\n",
+    "┃  • 3 leaves coriander                                                                                                                                       ┃\n",
+    "┃                                                                                                                                                             ┃\n",
+    "┃ Directions:                                                                                                                                                 ┃\n",
+    "┃                                                                                                                                                             ┃\n",
+    "┃  1 Bring the chicken stock and coconut milk to a slow boil.                                                                                                 ┃\n",
+    "┃  2 Add galangal, lemongrass, chicken, and mushrooms. Once the soup returns to a boil, season it with fish sauce.                                            ┃\n",
+    "┃  3 Wait until the chicken is cooked, then add the kaffir lime leaves and bird’s eye chilies.                                                                ┃\n",
+    "┃  4 Remove the pot from heat and add lime juice.                                                                                                             ┃\n",
+    "┃  5 Garnish with coriander leaves.                                                                                                                           ┃\n",
+    "┃                                                                                                                                                             ┃\n",
+    "┃ Tips:                                                                                                                                                       ┃\n",
+    "┃                                                                                                                                                             ┃\n",
+    "┃  • Keep the heat low throughout the cooking process to prevent the oil in the coconut milk from separating.                                                 ┃\n",
+    "┃  • If using mature galangal, reduce the amount.                                                                                                             ┃\n",
+    "┃  • Adding lime juice after removing the pot from heat makes it more aromatic.                                                                               ┃\n",
+    "┃  • Reduce the number of chilies for a milder taste.                                                                                                         ┃\n",
+    "┃                                                                                                                                                             ┃\n",
+    "┃ Enjoy making and savoring this flavorful Thai soup!                                                                                                         ┃\n",
+    "┃                                                                                                                                                             ┃\n",
+    "┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Congratulations, you have learned the basics of using Milvus in Phidata. If you want to know more about how to use Phidata, please refer to the [official documentation](https://docs.phidata.com/introduction).\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "py310",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}