enginesz

zou-group · Sep 29, 2024 · 891f74c · 891f74c
1 parent 2e5c8ac
commit 891f74c
Show file tree

Hide file tree

Showing 6 changed files with 645 additions and 3 deletions.
diff --git a/README.md b/README.md
@@ -29,6 +29,47 @@ This API is similar to the Pytorch API, making it simple to adapt to your usecas
 
 ![Analogy with Torch](assets/analogy.png)
 
+
+### Updates:
+
+**29th Sept 2024**:
+
+We are introducing a new engine based on [litellm](https://github.com/BerriAI/litellm). This should allow
+you to use any model you like, as long as it is supported by litellm. This means that now
+**Bedrock, Together, Gemini and even more** are all supported by TextGrad!
+
+In addition to this, with the new engines it should be easy to enable and disable caching.  
+
+We are in the process of testing these new engines and deprecating the old engines. If you have any issues, please let us know!
+
+The new litellm engines can be loaded with the following code: 
+
+An example of loading a litellm engine:
+```python
+engine = get_engine("experimental:gpt-4o", cache=False)
+
+# this also works with
+
+set_backward_engine("experimental:gpt-4o", cache=False)
+```
+
+Be sure to set the relevant environment variables for the new engines!
+
+An example of forward pass:
+```python
+
+import httpx
+from textgrad.engine_experimental.litellm import LiteLLMEngine
+
+LiteLLMEngine("gpt-4o", cache=True).generate(content="hello, what's 3+4", system_prompt="you are an assistant")
+
+image_url = "https://upload.wikimedia.org/wikipedia/commons/a/a7/Camponotus_flavomarginatus_ant.jpg"
+image_data = httpx.get(image_url).content
+```
+
+In the examples folder you will find two new notebooks that show how to use the new engines.
+
+
 ## QuickStart
 If you know PyTorch, you know 80% of TextGrad.
 Let's walk through the key components with a simple example. Say we want to use GPT-4o to solve a simple
@@ -96,7 +137,42 @@ answer
 > :white_check_mark: **answer: It will still take 1 hour to dry 30 shirts under the sun,**
 > **assuming they are all laid out properly to receive equal sunlight.**
 
+### Updates:
+
+**29th Sept 2024**:
+
+We are introducing a new engine based on [litellm](https://github.com/BerriAI/litellm). This should allow
+you to use any model you like, as long as it is supported by litellm. This means that now
+**Bedrock, Together, Gemini and even more** are all supported by TextGrad!
+
+In addition to this, with the new engines it should be easy to enable and disable caching.  
+
+We are in the process of testing these new engines and deprecating the old engines. If you have any issues, please let us know!
 
+The new litellm engines can be loaded with the following code: 
+
+An example of loading a litellm engine:
+```python
+engine = get_engine("experimental:gpt-4o", cache=False)
+
+# this also works with
+
+set_backward_engine("experimental:gpt-4o", cache=False)
+```
+
+An example of forward pass:
+```python
+
+import httpx
+from textgrad.engine_experimental.litellm import LiteLLMEngine
+
+LiteLLMEngine("gpt-4o", cache=True).generate(content="hello, what's 3+4", system_prompt="you are an assistant")
+
+image_url = "https://upload.wikimedia.org/wikipedia/commons/a/a7/Camponotus_flavomarginatus_ant.jpg"
+image_data = httpx.get(image_url).content
+```
+
+In the examples folder you will find two new notebooks that show how to use the new engines.
 
 
 We have many more examples around how TextGrad can optimize all kinds of variables -- code, solutions to problems, molecules, prompts, and all that!
@@ -119,6 +195,8 @@ you need an OpenAI/Anthropic key to run the LLMs).
 
 </div>
 
+
+
 ### Installation
 
 You can install TextGrad using any of the following methods.

diff --git a/examples/notebooks/experimental_engines/Tutorial-ExperimentalEngines.ipynb b/examples/notebooks/experimental_engines/Tutorial-ExperimentalEngines.ipynb
@@ -0,0 +1,133 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2661032c-2d9b-43f5-b074-28c119b57a14",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import textgrad\n",
+    "import os\n",
+    "from textgrad.engine_experimental.openai import OpenAIEngine"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "5e74c489-c47e-413c-adae-cd1201f6f94f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "os.environ[\"OPENAI_API_KEY\"] = \"SOMETHING\"\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "50c02746-dd33-4cb0-896e-4771e4b76ed7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "OpenAIEngine(\"gpt-4o-mini\", cache=True).generate(content=\"hello, what's 3+4\", system_prompt=\"you are an assistant\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f0ac186e-5c34-4115-aeda-bbd301be2667",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b09f7e36-ae4f-4746-8548-c2c189827435",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2ccce563-3d11-4d20-9c72-05d43cce4f6c",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3053630a-dbf3-4d3e-b553-5c8ea73e2ccd",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "58db7bb4-7f0f-4517-bba2-60a51b85908b",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7ee586ac-473e-4807-b66d-8524d08dc236",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import httpx\n",
+    "from textgrad.engine_experimental.litellm import LiteLLMEngine\n",
+    "\n",
+    "LiteLLMEngine(\"gpt-4o\", cache=True).generate(content=\"hello, what's 3+4\", system_prompt=\"you are an assistant\")\n",
+    "\n",
+    "image_url = \"https://upload.wikimedia.org/wikipedia/commons/a/a7/Camponotus_flavomarginatus_ant.jpg\"\n",
+    "image_data = httpx.get(image_url).content"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e5b1f1d5-8971-4dee-9958-c708ba807921",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "LiteLLMEngine(\"gpt-4o\", cache=True).generate(content=[image_data, \"what is this my boy\"], system_prompt=\"you are an assistant\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "43d9b703-4488-4222-a7fb-773293c13514",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}