diff --git a/README.md b/README.md index 4a4e8a3..86e25ff 100644 --- a/README.md +++ b/README.md @@ -3,30 +3,65 @@ ___ ## About -`llm-wrapper` is a versatile and powerful library designed to streamline the process of querying large language models, offering a user-friendly experience. -The` llm-wrapper` package is designed to simplify interactions with the underlying models by providing the following features: +llm-wrapper is a versatile and powerful library designed to streamline the process of querying Large Language Models +(LLMs) 🤖💬 -* **Simple and User-Friendly Interface**: The module offers an intuitive and easy-to-use interface, making it straightforward to work with the model. +Developed by the Allegro engineers, llm-wrapper is based on popular libraries like transformers, pydantic, and langchain. It takes care +of the boring boiler-plate code you write around your LLM applications, quickly enabling you to prototype ideas, and eventually helping you to scale up +for production use-cases! -* **Asynchronous Querying**: Requests to the model are processed asynchronously by default, ensuring efficient and non-blocking interactions. +Among the llm-wrapper most notable features, you will find: -* **Automatic Retrying Mechanism**: The module includes an automatic retrying mechanism, which helps handle transient errors and ensures that queries to the model are robust. +* **😊 Simple and User-Friendly Interface**: The module offers an intuitive and easy-to-use interface, making it straightforward to work with the model. -* **Error Handling and Management**: Errors that may occur during interactions with the model are handled and managed gracefully, providing informative error messages and potential recovery options. +* **🔀 Asynchronous Querying**: Requests to the model are processed asynchronously by default, ensuring efficient and non-blocking interactions. -* **Output Parsing**: The module simplifies the process of defining the model's output format as well as parsing and working with it, allowing you to easily extract the information you need. +* **🔄 Automatic Retrying Mechanism** : The module includes an automatic retrying mechanism, which helps handle transient errors and ensures that queries to the model are robust. + +* **🛠️ Error Handling and Management**: Errors that may occur during interactions with the model are handled and managed gracefully, providing informative error messages and potential recovery options. + +* **⚙️ Output Parsing**: The module simplifies the process of defining the model's output format as well as parsing and working with it, allowing you to easily extract the information you need. ___ ## Documentation -Are you interested in using `llm-wrapper` in your project? Consult the [Official Documentation](# TODO: open-source)! +Full documentation available at **[llm-wrapper.allegro.tech](https://llm-wrapper.allegro.tech/)** + +Get familiar with llm-wrapper 🚀: [introductory jupyter notebook](https://github.com/allegro/llm-wrapper/blob/main/examples/introduction.ipynb) ___ +## Quickstart + +Install the package via pip: + +``` +pip install llm-wrapper +``` + +Configure endpoint credentials and start querying the model! + +```python +from llm_wrapper.models import AzureOpenAIModel +from llm_wrapper.domain.configuration import AzureOpenAIConfiguration + +configuration = AzureOpenAIConfiguration( + api_key="", + base_url="", + api_version="", + deployment="", + model_name="" +) + +gpt_model = AzureOpenAIModel(config=configuration) +gpt_response = gpt_model.generate("Plan me a 3-day holiday trip to Italy") +``` +___ + ## Local Development -### Installation +### Installation from the source We assume that you have python `3.10.*` installed on your machine. You can set it up using [pyenv](https://github.com/pyenv/pyenv#installationbrew) @@ -59,6 +94,13 @@ In order to execute tests, run: make tests ``` +### Updating the documentation + +Run `mkdocs serve` to serve a local instance of the documentation. + +Modify the content of `docs` directory to update the documentation. The updated content will be deployed +via the github action `.github/workflows/docs.yml` + ### Make a new release When a new version of `llm-wrapper` is ready to be released, do the following operations: diff --git a/docs/tutorial/index.md b/docs/tutorial/index.md deleted file mode 100644 index ac376e9..0000000 --- a/docs/tutorial/index.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -layout: default -title: Tutorial -nav_order: 3 -has_children: true ---- - -## Unleash the Library's Potential - -Dive into our tutorial to fully grasp the library's capabilities. From beginners to experts, it's designed to accommodate all levels of experience. - -### What You'll Gain - -- **Discover Core Features**: Learn how to harness the library's power for your projects. -- **Efficient Learning**: Save time with hands-on examples, tips, and best practices. -- **Hidden Gems**: Unlock advanced functions that can enhance your work. -- **Troubleshooting**: Find solutions to common issues. -- **Community Support**: Join a helpful user community. - -### Access the Tutorial - -Visit our documentation to access the tutorial. It's your key to maximizing the potential of the library, whether you're using it for research, development, or any other application. Start your journey now and witness the limitless possibilities it offers! - - -## Before You Begin - -All the examples presented in the tutorials assume that you pass a configuration object to the model. How to do -it is described in detail in the Quick Start section. For example, for the Azure GPT model it's done like this: -```python -from llm_wrapper.models.azure_openai import AzureOpenAIModel -from llm_wrapper.domain.configuration import AzureOpenAIConfiguration - -configuration = AzureOpenAIConfiguration( - api_key="", - base_url="", - api_version="", - deployment="", - model_name="" -) - -model = AzureOpenAIModel(config=configuration) -``` - -------- -_*Documentation Powered by GPT-3.5-turbo_ \ No newline at end of file diff --git a/docs/tutorial/batch_query.md b/docs/usage/batch_query.md similarity index 100% rename from docs/tutorial/batch_query.md rename to docs/usage/batch_query.md diff --git a/docs/tutorial/deploy_llama2_on_azure.md b/docs/usage/deploy_llama2_on_azure.md similarity index 100% rename from docs/tutorial/deploy_llama2_on_azure.md rename to docs/usage/deploy_llama2_on_azure.md diff --git a/docs/tutorial/error_handling.md b/docs/usage/error_handling.md similarity index 100% rename from docs/tutorial/error_handling.md rename to docs/usage/error_handling.md diff --git a/docs/tutorial/forcing_response_format.md b/docs/usage/forcing_response_format.md similarity index 100% rename from docs/tutorial/forcing_response_format.md rename to docs/usage/forcing_response_format.md diff --git a/docs/tutorial/single_query.md b/docs/usage/single_query.md similarity index 100% rename from docs/tutorial/single_query.md rename to docs/usage/single_query.md diff --git a/examples/introduction.ipynb b/examples/introduction.ipynb new file mode 100644 index 0000000..5a4b29a --- /dev/null +++ b/examples/introduction.ipynb @@ -0,0 +1,443 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "source": [ + "# Introduction\n", + "\n", + "Follow this tutorial to get to know the most important features of llm-wrapper!\n", + "\n" + ], + "metadata": { + "collapsed": false + }, + "id": "d6cb6b8c8fdca3cd" + }, + { + "cell_type": "markdown", + "id": "7bcb1d86-2487-4ca1-9d03-19bd3ad1a097", + "metadata": {}, + "source": [ + "# Import and utils" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "6e0a9b56-8099-4b2e-a881-01af966ed59d", + "metadata": { + "ExecuteTime": { + "end_time": "2024-01-04T16:03:35.407204Z", + "start_time": "2024-01-04T16:03:35.401117Z" + } + }, + "outputs": [], + "source": [ + "# This allows to run asynchronous code in a Jupyter notebook\n", + "import nest_asyncio\n", + "\n", + "nest_asyncio.apply()" + ] + }, + { + "cell_type": "markdown", + "id": "3b81480b-06ad-4f7e-9fe6-731baf9c80ef", + "metadata": {}, + "source": [ + "## Setting up your LLM\n", + "\n", + "To start working with `llm-wrapper` you need to import one of the supported models and configure it. Make sure to have access to an Azure OpenAI endpoint and dispose of the needed information. In this tutorial we are going to use a GPT model." + ] + }, + { + "cell_type": "markdown", + "id": "9c0fba84-c906-4c40-9fcb-15f7fefd2b82", + "metadata": {}, + "source": [] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "cecb6d45-52bb-4530-bfd9-99848b40e106", + "metadata": { + "ExecuteTime": { + "end_time": "2024-01-04T16:03:39.700051Z", + "start_time": "2024-01-04T16:03:35.414123Z" + } + }, + "outputs": [], + "source": [ + "from llm_wrapper.models import AzureOpenAIModel\n", + "from llm_wrapper.domain.configuration import AzureOpenAIConfiguration\n", + "\n", + "configuration = AzureOpenAIConfiguration(\n", + " api_key=\"\",\n", + " base_url=\"\",\n", + " api_version=\"\",\n", + " deployment=\"\",\n", + " model_name=\"\"\n", + ")\n", + "\n", + "model = AzureOpenAIModel(config=configuration)" + ] + }, + { + "cell_type": "markdown", + "id": "d4afb572-c2a8-4e00-95a7-d7f7bdf2dc84", + "metadata": {}, + "source": [ + "## Basic usage" + ] + }, + { + "cell_type": "markdown", + "id": "4278435d-a259-408c-85dc-329b38e617d5", + "metadata": {}, + "source": [ + "The model has a `generate()` method that is responsible for running the generations. In the most basic case, you can simply provide it with a prompt and it’ll return generated content. " + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "c2c243b4-51af-4bfd-a0a0-d9787f4d19e5", + "metadata": { + "ExecuteTime": { + "end_time": "2024-01-04T16:03:47.961341Z", + "start_time": "2024-01-04T16:03:47.030222Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": "[ResponseData(response='The capital of Poland is Warsaw.', input_data=None, number_of_prompt_tokens=7, number_of_generated_tokens=7, error=None)]" + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model.generate(\"What is the capital of Poland?\")" + ] + }, + { + "cell_type": "markdown", + "id": "e3997f4f-c26f-4af2-b1b9-3c6f35217aa5", + "metadata": {}, + "source": [ + "This was an example of the most basic usage. But what if you wanted to run a single prompt multiple times, but with slightly changed data? For example, you have a dataset of reviews and you want to classify each of them as positive or negative. You can use batch mode to do this." + ] + }, + { + "cell_type": "markdown", + "id": "c6ab1163-bc77-4234-8095-31f0592af3bc", + "metadata": {}, + "source": [ + "## Batch mode" + ] + }, + { + "cell_type": "markdown", + "id": "a3d3beb8-0b94-40fd-bca7-2b213362ef50", + "metadata": {}, + "source": [ + "Let's say you have a dataset with 3 reviews and you want to classify each of them as positive or negative. To do so:\n", + "- create a `prompt` and inside it use symbolic variable `{review}`, which will later be replaced by actual reviews coming from the dataset.\n", + "- create `input_data`. `input_data` is simply a list of `InputData`, where each `InputData` is a single example and it's a dataclass with two fields:\n", + " - `input_mappings` - a dictionary mapping symbolic variables used in the prompt to the actual review.\n", + " - `id` - is needed because requests are made asynchronously, so the output order will not always be the same as the input order.\n", + "- run the generation by calling the `generate()` method with the `prompt` and `input_data` as arguments. \n", + "\n", + "This will automatically run the generation in async mode, so it'll be much faster than a normal, sequential calling. Additionally, it'll automatically retry requests in case of failure. " + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "eeb8c703-8203-46fb-b3c2-c0836ad2c349", + "metadata": { + "ExecuteTime": { + "end_time": "2024-01-04T16:03:48.720192Z", + "start_time": "2024-01-04T16:03:48.234700Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": "{'review_id=0': 'The review is positive.',\n 'review_id=1': 'The review is positive.',\n 'review_id=2': 'The review is negative.'}" + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "from llm_wrapper.domain.input_data import InputData\n", + "\n", + "\n", + "positive_review_0 = \"Very good coffee, lightly roasted, with good aroma and taste. The taste of sourness is barely noticeable (which is good because I don't like sour coffees). After grinding, the aroma spreads throughout the room. I recommend it to all those who do not like strongly roasted and pitch-black coffees. A very good solution is to close the package with string, which allows you to preserve the aroma and freshness.\"\n", + "positive_review_1 = \"Delicious coffee!! Delicate, just the way I like it, and the smell after opening is amazing. It smells freshly roasted. Faithful to Lavazza coffee for years, I decided to look for other flavors. Based on the reviews, I blindly bought it and it was a 10-shot, it outperformed Lavazze in taste. For me the best.\"\n", + "negative_review = \"Marketing is doing its job and I was tempted too, but this coffee is nothing above the level of coffees from the supermarket. And the method of brewing or grinding does not help here. The coffee is simply weak - both in terms of strength and taste. I do not recommend.\"\n", + "\n", + "prompt = \"You'll be provided with a review of a coffe. Decide if the review is positive or negative. Review: {review}\"\n", + "input_data = [\n", + " InputData(input_mappings={\"review\": positive_review_0}, id=\"0\"),\n", + " InputData(input_mappings={\"review\": positive_review_1}, id=\"1\"),\n", + " InputData(input_mappings={\"review\": negative_review}, id=\"2\")\n", + "]\n", + "\n", + "responses = model.generate(prompt=prompt, input_data=input_data)\n", + "\n", + "{f\"review_id={response.input_data.id}\": response.response for response in responses}" + ] + }, + { + "cell_type": "markdown", + "id": "52cc821b-1d30-4220-be2e-7d4464c3d605", + "metadata": {}, + "source": [ + "### Multiple symbolic variables" + ] + }, + { + "cell_type": "markdown", + "id": "82d20f2a-cbf3-4b15-a4d0-07c7a6cbf771", + "metadata": {}, + "source": [ + "The example above showed a prompt with only one symbolic variable used in it. But you can use as many of them as you want.\n", + "\n", + "Let’s say you have two reviews: one positive and one negative, and you want the model to tell which one of them is positive. To do so:\n", + "- create a prompt as shown in the cell below. Two symbolic variables are used inside it: `{first_review}` and `{second_review}`.\n", + "- create `input_data`. It looks similar to the example above - it's a list of `InputData`, but here the `input_mappings` fields have two entries, one per single symbolic variable used in the prompt.\n", + "- same as above, generation is ran by calling the `generate()` method." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "c4ddfdcd-21c1-43d5-9f88-97868da710cb", + "metadata": { + "ExecuteTime": { + "end_time": "2024-01-04T16:03:50.249349Z", + "start_time": "2024-01-04T16:03:49.587683Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": "{'example_id=0': 'The first review is positive.',\n 'example_id=1': 'The second review is positive.'}" + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "prompt = \"\"\"You'll be provided with two reviews of a coffee. Decide which one is positive.\n", + "\n", + "First review: {first_review}\n", + "Second review: {second_review}\"\"\"\n", + "input_data = [\n", + " InputData(input_mappings={\"first_review\": positive_review_0, \"second_review\": negative_review}, id=\"0\"),\n", + " InputData(input_mappings={\"first_review\": negative_review, \"second_review\": positive_review_1}, id=\"1\"),\n", + "]\n", + "\n", + "responses = model.generate(prompt=prompt, input_data=input_data)\n", + "{f\"example_id={response.input_data.id}\": response.response for response in responses}" + ] + }, + { + "cell_type": "markdown", + "id": "1a64a6a8-28c8-4ce3-9c7c-bd1ffc56ed24", + "metadata": {}, + "source": [ + "## Forcing model response format" + ] + }, + { + "cell_type": "markdown", + "id": "c16211a8-c937-4f9e-8c61-286b1278f004", + "metadata": {}, + "source": [ + "This is one of the most interesting features of our library. In a production setup, it's often the case that we want the model to return generated content in a format that will later be easy to ingest by the rest of our pipeline - for example, json with some predefined fields. With our library it’s really easy to achieve." + ] + }, + { + "cell_type": "markdown", + "id": "85da7d66-7b05-4b78-838f-82ab1c8968c6", + "metadata": {}, + "source": [ + "Let’s say that again you have a review of a coffee, and you want the model to generate information that might be interesting for you, and additionally you want it to return them in the format provided by you. To do so, first you have to create a dataclass that defines the output format and the information you want the model to generate. Each field of this dataclass must have a type defined and also a description provided that describes what given field means. The better the description, the better the model will understand what it should generate for a given field." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "62fcaf9d-b5c5-4e6c-a8b6-077a8bc9288f", + "metadata": { + "ExecuteTime": { + "end_time": "2024-01-04T16:03:52.193625Z", + "start_time": "2024-01-04T16:03:52.187122Z" + } + }, + "outputs": [], + "source": [ + "import typing\n", + " \n", + "from pydantic import BaseModel, Field\n", + " \n", + "class ReviewOutputDataModel(BaseModel):\n", + " summary: str = Field(description=\"Summary of a product description\")\n", + " should_buy: bool = Field(description=\"Recommendation whether I should buy the product or not\")\n", + " brand_name: str = Field(description=\"Brand of the coffee\")\n", + " aroma:str = Field(description=\"Description of the coffee aroma\")\n", + " cons: typing.List[str] = Field(description=\"List of cons of the coffee\")" + ] + }, + { + "cell_type": "markdown", + "id": "19c9635d-93aa-4549-87db-366ef914acb6", + "metadata": {}, + "source": [ + "The next thing is to create a prompt, which can be pretty simple as shown in the cell below, and the `input_data` for the model. To force the model to generate a response in a given format, you have to call the `generate()` method with `prompt`, `input_data` and with one additional argument called `output_data_model_class`. The `ReviewOutputDataModel` class defined above should be provided to this argument. This automatically tells the model to output predictions in the format defined by this dataclass." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "dd633a75-ce33-4bad-a298-61ed6c4e8de4", + "metadata": { + "ExecuteTime": { + "end_time": "2024-01-04T16:03:54.571172Z", + "start_time": "2024-01-04T16:03:53.648911Z" + } + }, + "outputs": [], + "source": [ + "review = \"Marketing is doing its job and I was tempted too, but this Blue Orca coffee is nothing above the level of coffees from the supermarket. And the method of brewing or grinding does not help here. The coffee is simply weak - both in terms of strength and taste. I do not recommend.\"\n", + " \n", + "prompt = \"Summarize review of the coffee. Review: {review}\"\n", + "input_data = [\n", + " InputData(input_mappings={\"review\": review}, id=\"0\")\n", + "]\n", + "\n", + "responses = model.generate(\n", + " prompt=prompt, \n", + " input_data=input_data,\n", + " output_data_model_class=ReviewOutputDataModel\n", + ")\n", + "response = responses[0].response" + ] + }, + { + "cell_type": "markdown", + "id": "3a322c89-c2ca-462f-ba77-8120b5e0945a", + "metadata": {}, + "source": [ + "The results below show that the predictions are indeed returned in the format defined above. " + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "9f9996ec-141f-45a6-a900-71efd5fe3a96", + "metadata": { + "ExecuteTime": { + "end_time": "2024-01-04T16:03:55.095035Z", + "start_time": "2024-01-04T16:03:55.078664Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": "__main__.ReviewOutputDataModel" + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "type(response)" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "01bb6d5c-80bc-43e7-97ac-252b98b45262", + "metadata": { + "ExecuteTime": { + "end_time": "2024-01-04T16:03:55.867763Z", + "start_time": "2024-01-04T16:03:55.854264Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": "{'summary': 'The Blue Orca coffee is nothing above the level of coffees from the supermarket. It is weak in terms of strength and taste.',\n 'should_buy': False,\n 'brand_name': 'Blue Orca',\n 'aroma': 'Not mentioned in the review',\n 'cons': ['Weak in terms of strength', 'Weak in terms of taste']}" + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "response.dict()" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "767ff534-85a6-4d2c-b71b-c869f9343623", + "metadata": { + "ExecuteTime": { + "end_time": "2024-01-04T16:03:56.968171Z", + "start_time": "2024-01-04T16:03:56.958045Z" + } + }, + "outputs": [ + { + "data": { + "text/plain": "ReviewOutputDataModel(summary='The Blue Orca coffee is nothing above the level of coffees from the supermarket. It is weak in terms of strength and taste.', should_buy=False, brand_name='Blue Orca', aroma='Not mentioned in the review', cons=['Weak in terms of strength', 'Weak in terms of taste'])" + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "response" + ] + }, + { + "cell_type": "markdown", + "id": "64fef78d-1ae4-4477-8096-4b52fa4e8d16", + "metadata": {}, + "source": [ + "This is really interesting feature, because it gives the possibility to do several tasks at once. In the above example, there was summarization, classification, entity extraction and so on. To add another one, simply add a new field to the dataclass. For example, if you'd like to know the pros of the coffee, you just need to add one additional field `pros` to the dataclass, describe it properly, re-run everything and you'll get the results. So as you can see, it significantly reduces the need to do extensive prompt engineering. You just define it in the code as an additional field and you’re done." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}