From f3db605285342b6d51e111350db84bc86156965d Mon Sep 17 00:00:00 2001 From: Riccardo Belluzzo Date: Tue, 20 Feb 2024 12:16:14 +0100 Subject: [PATCH] nav tab added to mkdocs.yml --- docs/api/input_output_dataclasses.md | 8 ----- docs/api/models/azure_llama2_model.md | 8 ----- docs/api/models/azure_mistral_model.md | 8 ----- docs/api/models/azure_openai_model.md | 8 ----- docs/api/models/vertexai_gemini_model.md | 8 ----- docs/api/models/vertexai_palm_model.md | 8 ----- docs/faq.md | 6 +--- docs/index.md | 6 ---- docs/installation_and_quick_start.md | 6 ---- docs/usage/{batch_query.md => advanced.md} | 27 +++++++++++----- docs/usage/basic.md | 32 +++++++++++++++++++ ..._azure.md => deploy_open_source_models.md} | 7 +--- docs/usage/error_handling.md | 8 +---- docs/usage/forcing_response_format.md | 13 +++----- docs/usage/single_query.md | 17 ---------- mkdocs.yml | 19 ++++++++++- 16 files changed, 77 insertions(+), 112 deletions(-) rename docs/usage/{batch_query.md => advanced.md} (88%) create mode 100644 docs/usage/basic.md rename docs/usage/{deploy_llama2_on_azure.md => deploy_open_source_models.md} (92%) delete mode 100644 docs/usage/single_query.md diff --git a/docs/api/input_output_dataclasses.md b/docs/api/input_output_dataclasses.md index 81cb083..777bea7 100644 --- a/docs/api/input_output_dataclasses.md +++ b/docs/api/input_output_dataclasses.md @@ -1,11 +1,3 @@ ---- -layout: default -title: Input/Output dataclasses -nav_order: 0 -parent: API ---- - - ## `class llm_wrapper.domain.input_data.InputData` dataclass ```python @dataclass diff --git a/docs/api/models/azure_llama2_model.md b/docs/api/models/azure_llama2_model.md index 40fdf3f..090708d 100644 --- a/docs/api/models/azure_llama2_model.md +++ b/docs/api/models/azure_llama2_model.md @@ -1,11 +1,3 @@ ---- -layout: default -title: AzureLlama2Model -parent: Models -grand_parent: API -nav_order: 3 ---- - ## `class llm_wrapper.models.AzureLlama2Model` API ### Methods ```python diff --git a/docs/api/models/azure_mistral_model.md b/docs/api/models/azure_mistral_model.md index 6485c77..e250314 100644 --- a/docs/api/models/azure_mistral_model.md +++ b/docs/api/models/azure_mistral_model.md @@ -1,11 +1,3 @@ ---- -layout: default -title: AzureMistralModel -parent: Models -grand_parent: API -nav_order: 5 ---- - ## `class llm_wrapper.models.AzureMistralModel` API ### Methods ```python diff --git a/docs/api/models/azure_openai_model.md b/docs/api/models/azure_openai_model.md index 8b9238d..0a8bbf2 100644 --- a/docs/api/models/azure_openai_model.md +++ b/docs/api/models/azure_openai_model.md @@ -1,11 +1,3 @@ ---- -layout: default -title: AzureOpenAIModel -parent: Models -grand_parent: API -nav_order: 1 ---- - ## `class llm_wrapper.models.AzureOpenAIModel` API ### Methods ```python diff --git a/docs/api/models/vertexai_gemini_model.md b/docs/api/models/vertexai_gemini_model.md index 03476e7..401fa0c 100644 --- a/docs/api/models/vertexai_gemini_model.md +++ b/docs/api/models/vertexai_gemini_model.md @@ -1,11 +1,3 @@ ---- -layout: default -title: VertexAIGeminiModel -parent: Models -grand_parent: API -nav_order: 4 ---- - ## `class llm_wrapper.models.VertexAIGeminiModel` API ### Methods ```python diff --git a/docs/api/models/vertexai_palm_model.md b/docs/api/models/vertexai_palm_model.md index ed83283..f366edd 100644 --- a/docs/api/models/vertexai_palm_model.md +++ b/docs/api/models/vertexai_palm_model.md @@ -1,11 +1,3 @@ ---- -layout: default -title: VertexAIPalmModel -parent: Models -grand_parent: API -nav_order: 2 ---- - ## `class llm_wrapper.models.VertexAIPalmModel` API ### Methods ```python diff --git a/docs/faq.md b/docs/faq.md index a3e4551..b2b3d20 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -1,9 +1,5 @@ ---- -layout: default -title: FAQ -nav_order: 2 ---- +# Frequently Asked Questions ### 1. How to use the llm-wrapper in a python notebook? When using the `llm-wrapper` library, which utilizes asynchronous programming under the hood, you must install the `nest-asyncio` library to use it in a Jupyter notebook environment. diff --git a/docs/index.md b/docs/index.md index 46512c8..1c5b0e8 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,9 +1,3 @@ ---- -layout: default -title: Introduction -nav_order: 0 ---- -

LLM-Wrapper Logo

diff --git a/docs/installation_and_quick_start.md b/docs/installation_and_quick_start.md index 8b1b39c..85fb4a1 100644 --- a/docs/installation_and_quick_start.md +++ b/docs/installation_and_quick_start.md @@ -1,9 +1,3 @@ ---- -layout: default -title: Installation & Quick start -nav_order: 1 ---- - # Installation Install the package via pip: diff --git a/docs/usage/batch_query.md b/docs/usage/advanced.md similarity index 88% rename from docs/usage/batch_query.md rename to docs/usage/advanced.md index 01346d6..9516d83 100644 --- a/docs/usage/batch_query.md +++ b/docs/usage/advanced.md @@ -1,9 +1,6 @@ ---- -layout: default -title: Batch query -nav_order: 1 -parent: Tutorial ---- +# Advanced Usage + +## Symbolic Variables and Batch Mode If you want to generate responses for a batch of examples, you can achieve this by preparing a prompt with symbolic variables and providing input data that will be injected into this prompt. `llm-wrapper` will automatically make these @@ -11,6 +8,20 @@ requests in an async mode and retry them in case of any API error. Let's say we want to classify reviews of coffee as positive or negative. Here's how to do it: ```python +from llm_wrapper.models import AzureOpenAIModel +from llm_wrapper.domain.configuration import AzureOpenAIConfiguration +from llm_wrapper.domain.input_data import InputData + +configuration = AzureOpenAIConfiguration( + api_key="", + base_url="", + api_version="", + deployment="", + model_name="" +) + +model = AzureOpenAIModel(config=configuration) + positive_review_0 = "Very good coffee, lightly roasted, with good aroma and taste. The taste of sourness is barely noticeable (which is good because I don't like sour coffees). After grinding, the aroma spreads throughout the room. I recommend it to all those who do not like strongly roasted and pitch-black coffees. A very good solution is to close the package with string, which allows you to preserve the aroma and freshness." positive_review_1 = "Delicious coffee!! Delicate, just the way I like it, and the smell after opening is amazing. It smells freshly roasted. Faithful to Lavazza coffee for years, I decided to look for other flavors. Based on the reviews, I blindly bought it and it was a 10-shot, it outperformed Lavazze in taste. For me the best." negative_review = "Marketing is doing its job and I was tempted too, but this coffee is nothing above the level of coffees from the supermarket. And the method of brewing or grinding does not help here. The coffee is simply weak - both in terms of strength and taste. I do not recommend." @@ -68,12 +79,12 @@ And the results: } ``` -## Control the number of concurrent requests +## Controlling the Number of Concurrent Requests As it's written above, `llm-wrapper` automatically makes requests in an async mode. By default, the maximum number of concurrent requests is set to 1000. You can control this value by setting the `max_concurrency` parameter when initializing the model. Set it to a value that is appropriate for your model endpoint. -## Use a common asyncio event loop +## Using a common asyncio event loop By default, each model instance has its own event loop for handling the execution of async tasks. If you want to use a common loop for multiple models or to have a custom loop, it's possible to specify it in the model constructor: diff --git a/docs/usage/basic.md b/docs/usage/basic.md new file mode 100644 index 0000000..1a6c0e0 --- /dev/null +++ b/docs/usage/basic.md @@ -0,0 +1,32 @@ +# Basic Usage + +## Single Query + +In the simplest approach you just need to pass a prompt and the model will provide a response for it. + +```python +from llm_wrapper.models import AzureOpenAIModel +from llm_wrapper.domain.configuration import AzureOpenAIConfiguration +from llm_wrapper.domain.response import ResponseData + +configuration = AzureOpenAIConfiguration( + api_key="", + base_url="", + api_version="", + deployment="", + model_name="" +) + +model = AzureOpenAIModel(config=configuration) + +response = model.generate("What is the capital of Poland?") +print(response) + +#[ResponseData(response='The capital of Poland is Warsaw.', input_data=None, number_of_prompt_tokens=7, number_of_generated_tokens=7, error=None)] +``` + +As a response you'll get `List[ResponseData]`, where the first element will contain response from the model in the +`ResponseData.response` field and also information about number of prompt and generated tokens. If any error occurred +also `ResponseData.error` field will be filled with the actual exception. + + diff --git a/docs/usage/deploy_llama2_on_azure.md b/docs/usage/deploy_open_source_models.md similarity index 92% rename from docs/usage/deploy_llama2_on_azure.md rename to docs/usage/deploy_open_source_models.md index ee12cb8..6ed1bda 100644 --- a/docs/usage/deploy_llama2_on_azure.md +++ b/docs/usage/deploy_open_source_models.md @@ -1,9 +1,4 @@ ---- -layout: default -title: Azure Open-source Models Deployment -nav_order: 4 -parent: Tutorial ---- +# How to Deploy Azure Open-source LLMs? To use Open-source models like Llama or Mistral with llm-wrapper, first you have to deploy it on your own on Azure as a ML Online Endpoint. diff --git a/docs/usage/error_handling.md b/docs/usage/error_handling.md index 2f68ed1..b1a0056 100644 --- a/docs/usage/error_handling.md +++ b/docs/usage/error_handling.md @@ -1,10 +1,4 @@ ---- -layout: default -title: Error handling -nav_order: 3 -parent: Tutorial ---- - +# Error Handling ## Too long prompt Each LLM has its own context size defined. This is the maximum number of input plus output tokens that the model is able diff --git a/docs/usage/forcing_response_format.md b/docs/usage/forcing_response_format.md index 0b50206..094ecef 100644 --- a/docs/usage/forcing_response_format.md +++ b/docs/usage/forcing_response_format.md @@ -1,9 +1,6 @@ ---- -layout: default -title: Forcing model response format -nav_order: 2 -parent: Tutorial ---- +# Forcing Structured Response Format + +## Pydantic BaseModels Integration If you want to force the model to output the response in a given JSON schema, `llm-wrapper` provides an easy way to do it. You just need to provide a data model that describes the desired output format and the package does all the rest. @@ -64,7 +61,7 @@ False ['Weak in terms of strength', 'Weak in terms of taste'] ``` -### What to do when output formatting doesn't work? +## What to do when output formatting doesn't work? The feature described above works best with advanced proprietary models like GPT and PaLM/Gemini. Less capable models like Llama2 or Mistral may not able to understand instructions passed as output_dataclasses, and in most cases the returned response won't be compatible @@ -74,7 +71,7 @@ In such cases, we recommend to address the issue by specifying in the prompt how few-shot learning techniques is also advisable. In the case of JSON-like output, use double curly brackets to escape them in order to use them in the JSON example. -### How forcing response format works under the hood? +## How forcing response format works under the hood? To force the model to provide output in a desired format, under the hood `llm-wrapper` automatically adds a description of the desired output format. For example, for the `ReviewOutputDataModel` the description looks like this: ````text diff --git a/docs/usage/single_query.md b/docs/usage/single_query.md deleted file mode 100644 index 4481648..0000000 --- a/docs/usage/single_query.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -layout: default -title: Single query -nav_order: 0 -parent: Tutorial ---- - -In the simplest approach you just need to pass a prompt and the model will provide a response for it. - -```python ->>> model.generate("What is the capital of Poland?") -[ResponseData(response='The capital of Poland is Warsaw.', input_data=None, number_of_prompt_tokens=7, number_of_generated_tokens=7, error=None)] -``` - -As a response you'll get `List[ResponseData]`, where the first element will contain response from the model in the -`ResponseData.response` field and also information about number of prompt and generated tokens. If any error occurred -also `ResponseData.error` field will be filled with the actual exception. diff --git a/mkdocs.yml b/mkdocs.yml index 68ef2d3..184c972 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,2 +1,19 @@ site_name: llm-wrapper -theme: readthedocs \ No newline at end of file +theme: readthedocs +nav: + - Introduction: index.md + - Installation & Quickstart: installation_and_quick_start.md + - F.A.Q: faq.md + - User guide: + - Basic: usage/basic.md + - Advanced: usage/advanced.md + - Forcing Structured Output Format: usage/forcing_response_format.md + - Azure Open-source Models Deployment: usage/deploy_open_source_models.md + - API: + - Domain: api/input_output_dataclasses.md + - Models: + - Azure Llama2: api/models/azure_llama2_model.md + - Azure Mistral: api/models/azure_mistral_model.md + - Azure OpenAI GPT: api/models/azure_openai_model.md + - VertexAI PaLM2: api/models/vertexai_palm_model.md + - VertexAI Gemini: api/models/vertexai_gemini_model.md \ No newline at end of file