Cookbooks showcasing various applications of Cleanlab, as well as code shared for education, reproducibility, transparency.
| Example | Description |
|---|---|
| TLM-Demo-Notebook | Demo-ing various applications of the Trustworthy Language Model, particularly in customer support |
| tlm_call_api_directly | Call the TLM REST API directly. You can use any programming language (eg. Typescript) with http lib/tools by providing the necessary payload and headers. |
| TLM-PII-Detection | Find and mask PII with the Trustworthy Language Model |
| Detecting GDPR Violations with TLM | Analyze application logs using TLM to detect GDPR violations |
| Customer Support AI Agent with NeMo Guardrails | Reliable customer support AI Agent with Guardrails and trustworthiness scoring (Nvidia Blogpost) |
| Better LLM Evals in MLFlow | Automatically find the bad LLM responses lurking in your production logs/traces via trustworthiness scoring in MLFlow |
| Trustworthy RAG with Cleanlab in LlamaIndex | Run Cleanlab in RAG apps built with LlamaIndex for real-time detection of incorrect responses and root cause analysis. |
| TLM-Record-Matching | Using the Trustworthy Language Model to reliably match records between two different data tables |
| fine_tuning_data_curation | Use Cleanlab TLM and Cleanlab Studio to detect bad data in instruction tuning LLM datasets |
| tlm_vpc_demo | How to use the VPC deployment of TLM with OpenAI ChatCompletions API |
| Example | Description |
|---|---|
| few_shot_prompt_selection | Clean the pool of few-shot examples to improve prompt template for OpenAI LLM |
| fine_tuning_classification | Use Cleanlab Studio to improve the accuracy of fine-tuned LLMs for classification tasks |
| fine_tuning_mistral_beavertails | Analyze human annotated AI-safety-related labels (like toxicity) using Cleanlab Studio, and thus generate safer responses from LLMs |
| Evaluating_Toxicity_Datasets_Large_Language_Models | Analyze toxicity annotations in the Jigsaw dataset using Cleanlab Studio |
| time_series_automl | Model time series data in a tabular format and use Cleanlab Studio AutoML to achieve high prediction accuracy |
| Example | Description |
|---|---|
| TLM-SimpleQA-Benchmark | Benchmarking TLM and OpenAI LLMs on the SimpleQA dataset |
| benchmarking_hallucination_metrics | Evaluate the performance of popular real-time hallucination detection methods on RAG benchmarks |
| benchmarking_hallucination_model | Evaluate the performance of popular hallucination detection models on RAG benchmarks |
| gpt4-rag-logprobs | Obtaining logprobs from a GPT-4 based RAG system |
| generate_llm_response | Generate LLM responses for customer service requests using Llama 2 and OpenAI's API |