From 4398bcfdf8944281b65dc2f3ec827b81c3496faf Mon Sep 17 00:00:00 2001 From: caitlinwheeless Date: Mon, 29 Jul 2024 21:46:11 -0500 Subject: [PATCH 1/3] docs: Small updates to the Ragas ML backend readme --- .../examples/rag_quickstart/README.md | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/label_studio_ml/examples/rag_quickstart/README.md b/label_studio_ml/examples/rag_quickstart/README.md index c6392da63..b918064c9 100644 --- a/label_studio_ml/examples/rag_quickstart/README.md +++ b/label_studio_ml/examples/rag_quickstart/README.md @@ -3,7 +3,7 @@ title: Question answering with RAG using Label Studio type: guide tier: all -order: 5 +order: 5 hide_menu: true hide_frontmatter_title: true meta_title: RAG labeling with OpenAI using Label Studio @@ -12,27 +12,26 @@ categories: - Generative AI - Large Language Model - OpenAI - - Azure - - Ollama - ChatGPT - RAG - - LangChain + - Ragas - Embeddings -image: "/tutorials/llm-interactive.png" +image: "/tutorials/ragas.png" --- --> # RAG Quickstart Labeling -This example server connects Label Studio to [OpenAI](https://platform.openai.com/), to interact with chat and embedding models. It supports question answering and evaluation using RAG, given a list of questions as tasks, and a folder containing documentation (eg, a `/docs` path within a Github repository that has been cloned on your computer.) +This example server connects Label Studio to [OpenAI](https://platform.openai.com/), to interact with chat and embedding models. It supports question answering and evaluation using RAG, given a list of questions as tasks, and a folder containing documentation (e.g. a `/docs` path within a Github repository that has been cloned on your computer.) ## Starting the ML Backend -1. Make your reference documentation available to the backend +1. Make your reference documentation available to the backend. -Create a `docker-compose.override.yml` file alongside `docker-compose.yml`, and use it to mount a folder containing your documentation into the filesystem of the ML Backend's image. This example will mount the folder at `/host/path/to/your/documentation` on your computer, to the path /data/documentation inside the ML Backend docker image. The `DOCUMENTATION_PATH` and `DOCUMENTATION_GLOB` settings given below will match all `.md` files within `/data/documentation` (or its subfolders). +Create a `docker-compose.override.yml` file alongside `docker-compose.yml`, and use it to mount a folder containing your documentation into the filesystem of the ML backend's image. This example will mount the folder at `/host/path/to/your/documentation` on your computer, to the path /data/documentation inside the ML backend Docker image. The `DOCUMENTATION_PATH` and `DOCUMENTATION_GLOB` settings given below will match all `.md` files within `/data/documentation` (or its subfolders). ``` +services: rag_quickstart: volumes: - /host/path/to/your/documentation:/data/documentation @@ -58,8 +57,7 @@ $ curl http://localhost:9090/health Ensure the **Interactive preannotations** toggle is enabled and click **Validate and Save**. 5. Use the label config below. The config and backend can be customized to fit your needs. 6. Open a task and ensure the **Auto-Annotation** toggle is enabled (it is located at the bottom of the labeling interface). -7. Enter a prompt in the prompt input field and press `Shift+Enter`. The LLM response will be generated and displayed in - the response field. +7. The text fields should be auto-completed by the LLM. However, you can provide additional instructions with the field provided. To submit, press `Shift+Enter`. The LLM response will be generated and displayed in the response field. 8. If you want to apply LLM auto-annotation to multiple tasks at once, go to the [Data Manager](https://labelstud.io/guide/manage_data), select a group of tasks and then select **Actions > Retrieve Predictions** (or **Batch Predictions** in Label Studio Enterprise). ## Label Config @@ -154,6 +152,8 @@ $ curl http://localhost:9090/health ``` +For more information on this labeling config, see the [Evaluate RAG with Ragas](https://labelstud.io/templates/llm_ragas) template documentation. + **Example data input:** Tip: when generating questions for your project, it may be helpful to pass this snippet to ChatGPT etc to give it an example of Label Studio's tasks format to work from. @@ -173,4 +173,4 @@ Tip: when generating questions for your project, it may be helpful to pass this } } ] -``` +``` \ No newline at end of file From 230361c314a57a8be0379e14d7c41aa1f0cf1349 Mon Sep 17 00:00:00 2001 From: caitlinwheeless Date: Mon, 29 Jul 2024 21:49:46 -0500 Subject: [PATCH 2/3] fixing some clunky wording --- label_studio_ml/examples/rag_quickstart/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/label_studio_ml/examples/rag_quickstart/README.md b/label_studio_ml/examples/rag_quickstart/README.md index b918064c9..f603f903c 100644 --- a/label_studio_ml/examples/rag_quickstart/README.md +++ b/label_studio_ml/examples/rag_quickstart/README.md @@ -57,7 +57,7 @@ $ curl http://localhost:9090/health Ensure the **Interactive preannotations** toggle is enabled and click **Validate and Save**. 5. Use the label config below. The config and backend can be customized to fit your needs. 6. Open a task and ensure the **Auto-Annotation** toggle is enabled (it is located at the bottom of the labeling interface). -7. The text fields should be auto-completed by the LLM. However, you can provide additional instructions with the field provided. To submit, press `Shift+Enter`. The LLM response will be generated and displayed in the response field. +7. The text fields should be auto-completed by the LLM. However, you can provide additional instructions in the empty text area field. To submit, press `Shift+Enter`. 8. If you want to apply LLM auto-annotation to multiple tasks at once, go to the [Data Manager](https://labelstud.io/guide/manage_data), select a group of tasks and then select **Actions > Retrieve Predictions** (or **Batch Predictions** in Label Studio Enterprise). ## Label Config From 19b8f84fdde55fce96d6f25891c05b0b0f98922a Mon Sep 17 00:00:00 2001 From: caitlinwheeless Date: Tue, 30 Jul 2024 08:07:00 -0500 Subject: [PATCH 3/3] Updating template placeholders with Ragas instead of RAGAS --- label_studio_ml/examples/rag_quickstart/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/label_studio_ml/examples/rag_quickstart/README.md b/label_studio_ml/examples/rag_quickstart/README.md index f603f903c..0d54b53c5 100644 --- a/label_studio_ml/examples/rag_quickstart/README.md +++ b/label_studio_ml/examples/rag_quickstart/README.md @@ -114,7 +114,7 @@ $ curl http://localhost:9090/health /> -
+