UseCases/Fine_Tuning_OpenAI_Model/Fine_Tuning_OpenAI_Model_Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b6cd7ebc-fc77-4a2c-b17f-325921ece851",
   "metadata": {},
   "source": [
    "<header>\n",
    "   <p  style='font-size:36px;font-family:Arial; color:#F0F0F0; background-color: #00233c; padding-left: 20pt; padding-top: 20pt;padding-bottom: 10pt; padding-right: 20pt;'>\n",
    "       Fine-tune OpenAI GPT-3.5 Turbo Model in Teradata Vantage\n",
    "  <br>\n",
    "       <img id=\"teradata-logo\" src=\"https://storage.googleapis.com/clearscape_analytics_demo_data/DEMO_Logo/teradata.svg\" alt=\"Teradata\" style=\"width: 125px; height: auto; margin-top: 20pt;\">\n",
    "    </p>\n",
    "</header>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be6c633b-b136-403b-9828-0736a4ba079e",
   "metadata": {},
   "source": [
    "<p style = 'font-size:20px;font-family:Arial;color:#00233c'><b>Introduction</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The large Language Model (LLM) is a general-purpose model designed for a broad range of NLP tasks, including providing information on various topics, answering questions, offering suggestions, and even helping us in our work.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>However, to use LLM in a highly specialized business use case scenario like finance or healthcare, we need to train the model using a specific dataset to refine its capabilities and improve its performance. Fine-tuning can achieve this.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Fine-tuning</b> is training a pretrained model on a small and targeted dataset to achieve a specific task. By doing this, users can improve their performance on that task while preserving their general language knowledge.</p>\n",
    "\n",
    "<center><img src=\"images/fine-tuning.png\" alt=\"Fine_tuning_process\"  width=600 height=480/></center>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In this demo notebook, we train the GPT-3.5 Turbo model using OpenAI API on the \"mental health\" dataset. By doing so, the model becomes a specialist in answering mental health-related questions and can provide responses on how to tackle mental health and related issues.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>OpenAI allows users to create their own custom GPT-3.5 model tuned towards a particular dataset. As per the use case, we can teach GPT-3.5 the language and terminology of our niche domain, such as medicine or finance. The ChatGPT models are available via API, and in the example below, we used GPT-3.5 Turbo.</p>\n",
    "\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>Business Value</b></p>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Enhanced model performance for specific tasks.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Saves cost and time in model development.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Able to address real-world business problem in an efficient way. </li>\n",
    "</p> \n",
    "\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>Why Vantage? </b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Vantage provides a platform for storing data required for fine-tuning. This data is then cleaned, formatted, and validated to meet the standards necessary for fine-tuning GPT-3.5 Turbo model. Once it is ready, the data is stored as training and validation JSONL files.</p> \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The new data files saved in Vantage are then used to fine-tune the GPT-3.5 Turbo model with OpenAI API. Once the model is fine-tuned, it can be tested on Vantage.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fc8c433-7761-421b-8f35-c9eef22b9271",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233c'><b>Steps in the analysis:</b></p>\n",
    "<ol style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li>Configuring the environment</li>\n",
    "    <li>Connection to Vantage and OpenAI</li>\n",
    "    <li>Data Exploration</li>\n",
    "    <li>Data errors and cost estimation</li>\n",
    "    <li>Fine-tuning the model</li>\n",
    "    <li>Testing the fine-tuned model</li>\n",
    "    <li>Cleanup</li>\n",
    "</ol>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40af4879-c6b9-4e51-be98-15521cc4ac02",
   "metadata": {},
   "source": [
    "<hr style='height:2px;border:none;background-color:#00233C;'>\n",
    "<b style = 'font-size:20px;font-family:Arial;color:#00233c'>1. Configuring the environment</b>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>1.1 Install the required libraries</b></p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ca49fee0-9c37-466a-a98e-b55b4915faab",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "# '%%capture' suppresses the display of installation steps of the following packages\n",
    "\n",
    "!pip install -r requirements.txt --quiet"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdb01564-532d-4b64-82a7-23efdf467c52",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <i>*The above statements will install the required libraries to run this demo. To gain access to installed libraries after running this, restart the kernel.</i></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2fc33b9-a8ae-4b96-9302-b24cf78eaf1b",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "    <p style = 'font-size:16px;font-family:Arial;color:#00233C'><i><b>Note:</b> The above statements may need to be uncommented if you run the notebooks on a platform other than ClearScape Analytics Experience that does not have the libraries installed. If you uncomment those installs, be sure to restart the kernel after executing those lines to bring the installed libraries into memory. The simplest way to restart the Kernel is by typing zero zero: <b>0 0</b></i></p></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a29a6d38-f22f-403b-8348-5204722b682e",
   "metadata": {},
   "source": [
    "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>1.2 Import the required libraries</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Here, we import the required libraries, set environment variables and environment paths (if required).</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "319766f0-3826-4784-bf47-38637ed7e544",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import os\n",
    "import tiktoken # for token counting\n",
    "import openai\n",
    "from time import sleep\n",
    "\n",
    "# teradata lib\n",
    "from teradataml import *\n",
    "\n",
    "# Suppress warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "display.print_sqlmr_query = False\n",
    "display.suppress_vantage_runtime_warnings = True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f96eb4ba-ae01-493c-a0af-604641c66d01",
   "metadata": {},
   "source": [
    "<hr style='height:2px;border:none;background-color:#00233C;'>\n",
    "<b style = 'font-size:20px;font-family:Arial;color:#00233c'>2. Connection to Vantage</b>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c2e1bf5-9954-4c04-aa65-f34c041049c5",
   "metadata": {},
   "source": [
    "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>2.1 Connect to Vantage</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>When prompted to provide the password. Enter valid password, press the Enter key, and then use the down arrow to go to the next cell.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42d1f942-f0b2-4f56-a01e-b8b89d013621",
   "metadata": {},
   "outputs": [],
   "source": [
    "%run -i ../startup.ipynb\n",
    "eng = create_context(host = 'host.docker.internal', username='demo_user', password = password)\n",
    "print(eng)\n",
    "execute_sql('''SET query_band='DEMO=Fine_Tuning_OpenAI_Model_Python.ipynb;' UPDATE FOR SESSION;''')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2d55cba-326a-4ecd-85ec-a267b07a50f5",
   "metadata": {},
   "source": [
    "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>2.2 Getting Data for This Demo</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>For this demo, the data has been provided on cloud storage. We can run the demo using foreign tables to access the data without any storage in our environment.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "531a322d-7269-4adf-a454-52826c1e227e",
   "metadata": {},
   "outputs": [],
   "source": [
    "%run -i ../run_procedure.py \"call get_data('DEMO_LLM_FineTuning_cloud');\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33be9780-789e-426c-90d0-49a7de6d2f1d",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Next is an optional step. If you want, you can see the status of databases/tables created and space used.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "011f18c0-ac31-4c8a-a203-ce772b38d208",
   "metadata": {},
   "outputs": [],
   "source": [
    "%run -i ../run_procedure.py \"call space_report();\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91f3a9dd-214c-4c33-946f-51912d8ef02f",
   "metadata": {},
   "source": [
    "<hr style='height:2px;border:none;background-color:#00233C;'>\n",
    "<b style = 'font-size:20px;font-family:Arial;color:#00233c'>3. Data Exploration</b>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The data used in the demonstration is a mental health chatbot dataset containing questions and answers about mental health issues. The complete information on the data can be found <a href=\"https://huggingface.co/datasets/heliosbrahma/mental_health_chatbot_dataset\">here</a>.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In our database, we have one table: the <b>Mental_Health</b> table. This table contains text conversations between a human and an assistant about mental health issues. The human asks a question, and the assistant helps the human with an answer.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec55f30d-44e8-4317-b3f9-6431c230947d",
   "metadata": {},
   "source": [
    "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>3.1 Examine the Mental Health table</b></p>    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Let's look at the data in the table.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c4f950a0-ecb4-42cc-ad8b-ac7bb6654f2e",
   "metadata": {},
   "outputs": [],
   "source": [
    "tdf_mental_health = DataFrame(in_schema(\"DEMO_LLM_FineTuning\", \"Mental_Health\"))\n",
    "print(\"Data information: \\n\", tdf_mental_health.shape)\n",
    "\n",
    "pd_mental_health = tdf_mental_health.to_pandas()\n",
    "pd_mental_health.head"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4ba2e28-54e0-46b8-89aa-8d7725d23374",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The table has 1 column: <b>text</b> and 172 rows. Each row contains a pair of a question and an answer related to mental health.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "680ccdb2-dcbd-4015-a3a6-3d7d791fa885",
   "metadata": {},
   "source": [
    "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>3.2 Converting to correct format</b></p>    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The data in the table needs to be converted into a format suitable for fine-tuning OpenAI GPT-3.5 Turbo model.</p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The format required is:</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35e0e5a6-f05d-443c-84a9-48f617d190f9",
   "metadata": {},
   "source": [
    "```json\n",
    "{\"messages\": [{\"role\": \"system\", \"content\": \"text\"}, {\"role\": \"user\", \"content\": \"text\"}, {\"role\": \"assistant\", \"content\": \"text\"}]}\n",
    "{\"messages\": [{\"role\": \"system\", \"content\": \"text\"}, {\"role\": \"user\", \"content\": \"text\"}, {\"role\": \"assistant\", \"content\": \"text\"}]}\n",
    "{\"messages\": [{\"role\": \"system\", \"content\": \"text\"}, {\"role\": \"user\", \"content\": \"text\"}, {\"role\": \"assistant\", \"content\": \"text\"}]}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd8083d5-67e1-4ebc-9aa4-5f5f7922492f",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The \"messages\" is a dictionary with two keys:</p>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'><li> <b>roles</b>: <i>system</i>, <i>user</i>, or <i>assistant</i> tells us from where the content came from</li></ul>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'><li> <b>content</b>: the text content of the message</li></ul>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'></p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8374b392-08c6-407b-91a9-401164a86938",
   "metadata": {},
   "outputs": [],
   "source": [
    "def conversation_converter(conversation_data, system_message=None):\n",
    "    \n",
    "    #Splitting the conversation string into individual lines\n",
    "    lines = conversation_data.split('\\n<')\n",
    "    \n",
    "    messages = []\n",
    "    \n",
    "    # Including the system message if provided\n",
    "    if system_message:\n",
    "        messages.append({\n",
    "            \"role\": \"system\",\n",
    "            \"content\": system_message\n",
    "        })\n",
    "    \n",
    "    for l in lines:\n",
    "        parts = l.split(': ', 1)\n",
    "        # print(parts)\n",
    "        \n",
    "        if parts[0] == \"<HUMAN>\":\n",
    "            role = \"user\"\n",
    "        else:\n",
    "            role = \"assistant\"\n",
    "        \n",
    "        message = {\n",
    "            \"role\":role,\n",
    "            \"content\":parts[1]\n",
    "        }\n",
    "        messages.append(message)\n",
    "    \n",
    "    output = {\n",
    "        \"messages\":messages\n",
    "    }\n",
    "    return output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "549ce438-c473-44b7-bf5c-30d9d1c85a8b",
   "metadata": {},
   "outputs": [],
   "source": [
    "system_message = \"\"\"You are a helpful and understanding assistant who can help with mental health issues. You are friendly and polite\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f2ac99a-ef07-4977-8ace-ad51612e7cfc",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_mental_health = []\n",
    "\n",
    "for i in pd_mental_health.index:\n",
    "    data = pd_mental_health[\"text\"][i]\n",
    "    data_mental_health.append(data)\n",
    "\n",
    "dataset = []\n",
    "\n",
    "for data in data_mental_health:\n",
    "    data = conversation_converter(data, system_message=system_message)\n",
    "    dataset.append(data)\n",
    "\n",
    "print(\"The length of data:\", len(dataset))\n",
    "print(\"An example from dataset:\")\n",
    "for message in dataset[0][\"messages\"]:\n",
    "    print(message)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "590d238f-a210-4074-acba-cb48e0ac2948",
   "metadata": {},
   "source": [
    "<hr style='height:2px;border:none;background-color:#00233C;'>\n",
    "<b style = 'font-size:20px;font-family:Arial;color:#00233c'>4. Data errors and cost estimation</b>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Once we have compiled the dataset and before creating a fine-tuning job, it is important to perform following tasks:</p>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'><li>Format validation</li></ul>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'><li>Token counting</li></ul>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'><li>Cost estimation</li></ul>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>This part is taken from <a href=\"https://github.com/openai/openai-cookbook\">openai_cookbook</a>.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24bb8bdd",
   "metadata": {},
   "source": [
    "<div class=\\\"alert alert-block alert-warning\\>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><i><b>Note: Using OpenAI for fine-tuning is going to cost some money.</b></i></p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b41b45f5-6a19-4b38-b5ee-2ccb5ec169cf",
   "metadata": {},
   "source": [
    "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>4.1 Format validation</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>It is important to validate that each conversation in the dataset adheres to the format expected by the fine-tuning API. The below code cell finds out the error related to format of the dataset.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a6453dca-cc14-4101-a738-52daaa726ab2",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Format validation\n",
    "format_errors = defaultdict(int)\n",
    "\n",
    "for ex in dataset:\n",
    "    if not isinstance(ex, dict):\n",
    "        format_errors[\"data_type\"] += 1\n",
    "        continue\n",
    "\n",
    "    messages = ex.get(\"messages\", None)\n",
    "    if not messages:\n",
    "        format_errors[\"missing_messages_list\"] += 1\n",
    "        continue\n",
    "\n",
    "    for message in messages:\n",
    "        if \"role\" not in message or \"content\" not in message:\n",
    "            format_errors[\"message_missing_key\"] += 1\n",
    "\n",
    "        if any(k not in (\"role\", \"content\", \"name\") for k in message):\n",
    "            format_errors[\"message_unrecognized_key\"] += 1\n",
    "\n",
    "        if message.get(\"role\", None) not in (\"system\", \"user\", \"assistant\"):\n",
    "            format_errors[\"unrecognized_role\"] += 1\n",
    "\n",
    "        content = message.get(\"content\", None)\n",
    "        if not content or not isinstance(content, str):\n",
    "            format_errors[\"missing_content\"] += 1\n",
    "\n",
    "    if not any(message.get(\"role\", None) == \"assistant\" for message in messages):\n",
    "        format_errors[\"example_missing_assistant_message\"] += 1\n",
    "\n",
    "if format_errors:\n",
    "    print(\"Found errors:\")\n",
    "    for k, v in format_errors.items():\n",
    "        print(f\"{k}: {v}\")\n",
    "else:\n",
    "    print(\"No errors found\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4664775a-db2f-4c6a-8915-ae52fd8b6508",
   "metadata": {},
   "source": [
    "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>4.2 Token Counting Utilities</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Here the helpful utilities are defined which are to be used in the rest of the Notebook.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8b3a0c07-e004-4b38-ac71-a2c06bbe9585",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Token counting functions\n",
    "encoding = tiktoken.get_encoding(\"cl100k_base\")\n",
    "\n",
    "# not exact!\n",
    "# simplified from https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb\n",
    "def num_tokens_from_messages(messages, tokens_per_message=3, tokens_per_name=1):\n",
    "    num_tokens = 0\n",
    "    for message in messages:\n",
    "        num_tokens += tokens_per_message\n",
    "        for key, value in message.items():\n",
    "            num_tokens += len(encoding.encode(value))\n",
    "            if key == \"name\":\n",
    "                num_tokens += tokens_per_name\n",
    "    num_tokens += 3\n",
    "    return num_tokens\n",
    "\n",
    "def num_assistant_tokens_from_messages(messages):\n",
    "    num_tokens = 0\n",
    "    for message in messages:\n",
    "        if message[\"role\"] == \"assistant\":\n",
    "            num_tokens += len(encoding.encode(message[\"content\"]))\n",
    "    return num_tokens\n",
    "\n",
    "def print_distribution(values, name):\n",
    "    print(f\"\\n#### Distribution of {name}:\")\n",
    "    print(f\"min / max: {min(values)}, {max(values)}\")\n",
    "    print(f\"mean / median: {np.mean(values)}, {np.median(values)}\")\n",
    "    print(f\"p5 / p95: {np.quantile(values, 0.1)}, {np.quantile(values, 0.9)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a13afad9-4118-4060-99de-c617e2619ccb",
   "metadata": {},
   "source": [
    "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>4.3 Data Warnings and Token Counts</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Potential issues like missing messages in the dataset can be identified with some lightweight analysis. We can also find statistical insight into the message and token counts.</p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65a77b25-7aa9-41b4-896e-e473232675d0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Warnings and tokens counts\n",
    "n_missing_system = 0\n",
    "n_missing_user = 0\n",
    "n_messages = []\n",
    "convo_lens = []\n",
    "assistant_message_lens = []\n",
    "\n",
    "for ex in dataset:\n",
    "    messages = ex[\"messages\"]\n",
    "    if not any(message[\"role\"] == \"system\" for message in messages):\n",
    "        n_missing_system += 1\n",
    "    if not any(message[\"role\"] == \"user\" for message in messages):\n",
    "        n_missing_user += 1\n",
    "    n_messages.append(len(messages))\n",
    "    convo_lens.append(num_tokens_from_messages(messages))\n",
    "    assistant_message_lens.append(num_assistant_tokens_from_messages(messages))\n",
    "\n",
    "print(\"Num examples missing system message:\", n_missing_system)\n",
    "print(\"Num examples missing user message:\", n_missing_user)\n",
    "print_distribution(n_messages, \"num_messages_per_example\")\n",
    "print_distribution(convo_lens, \"num_total_tokens_per_example\")\n",
    "print_distribution(assistant_message_lens, \"num_assistant_tokens_per_example\")\n",
    "n_too_long = sum(l > 4096 for l in convo_lens)\n",
    "print(f\"\\n{n_too_long} examples may be over the 4096 token limit, they will be truncated during fine-tuning\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "decdbd4b-ea5c-47dd-ae94-fdffc7af90e9",
   "metadata": {},
   "source": [
    "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>4.4 Cost Estimation</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In this section, we estimate the total number of tokens that will be used for fine-tuning, this allows us to approximate the cost. It is worth noting that the duration of the fine-tuning jobs will also increase with the token count.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45d589aa-68e0-40c6-ad64-c1a49a153cdf",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pricing and default n_epochs estimate\n",
    "MAX_TOKENS_PER_EXAMPLE = 4096\n",
    "\n",
    "TARGET_EPOCHS = 3\n",
    "MIN_TARGET_EXAMPLES = 100\n",
    "MAX_TARGET_EXAMPLES = 25000\n",
    "MIN_DEFAULT_EPOCHS = 1\n",
    "MAX_DEFAULT_EPOCHS = 25\n",
    "\n",
    "n_epochs = TARGET_EPOCHS\n",
    "n_train_examples = len(dataset)\n",
    "if n_train_examples * TARGET_EPOCHS < MIN_TARGET_EXAMPLES:\n",
    "    n_epochs = min(MAX_DEFAULT_EPOCHS, MIN_TARGET_EXAMPLES // n_train_examples)\n",
    "elif n_train_examples * TARGET_EPOCHS > MAX_TARGET_EXAMPLES:\n",
    "    n_epochs = max(MIN_DEFAULT_EPOCHS, MAX_TARGET_EXAMPLES // n_train_examples)\n",
    "\n",
    "n_billing_tokens_in_dataset = sum(min(MAX_TOKENS_PER_EXAMPLE, length) for length in convo_lens)\n",
    "print(f\"Dataset has ~{n_billing_tokens_in_dataset} tokens that will be charged for during training\")\n",
    "print(f\"By default, you'll train for {n_epochs} epochs on this dataset\")\n",
    "print(f\"By default, you'll be charged for ~{n_epochs * n_billing_tokens_in_dataset} tokens\")\n",
    "print(\"See pricing page to estimate total costs\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61f2be44-a288-4e5f-ad74-6d9af72f6e55",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><i><b>Note:</b> See the pricing page to check the cost as per the analysis result of the above step.</i></p></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "171d2ec1-8163-419a-b5d4-a063dafec7c5",
   "metadata": {},
   "source": [
    "<hr style='height:2px;border:none;background-color:#00233C;'>\n",
    "<b style = 'font-size:20px;font-family:Arial;color:#00233c'>5. Fine-tuning the model</b>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>After validating the data format, fine-tuning the model comes next.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5eceffe",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-warning\">\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><i><b>Note: Using OpenAI for fine-tuning is going to cost some money.</b></i></p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4cc11aa0-09e2-4e07-ac86-366ada12186f",
   "metadata": {},
   "source": [
    "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>5.1 Get the OpenAI API key</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "398a4c74-3ff8-427a-8d6a-17a434501cc8",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In order to fine-tune the model, we will need an OpenAI API key. If do not have one, please refer to the instructions provided in this guide to obtain OpenAI API key:  </p>\n",
    "\n",
    "[Openai_setup_api_key_guide](..//Openai_setup_api_key/Openai_setup_api_key.md)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bcf51c55-433a-45e6-8ce0-6c1fd205cb5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import getpass\n",
    "\n",
    "# enter your openai api key\n",
    "api_key = getpass.getpass(prompt=\"\\n Please Enter Openai api key: \")\n",
    "\n",
    "os.environ[\"OPENAI_API_KEY\"] = api_key"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b83685c5-1e2e-4353-96c7-e9bdb91d0e23",
   "metadata": {},
   "source": [
    "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>5.2 JSONL file creation for training</b></p>    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>OpenAI accepts training and validation data formatted as JSON Line(JSONL) document. The fine-tuning dataset must be formatted in the conversational format.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "697ae5ee-5892-4d03-a793-258187bf29f6",
   "metadata": {},
   "outputs": [],
   "source": [
    "def write_jsonl(conversations, file_name):\n",
    "    with open(file_name, 'w') as out:\n",
    "        for conversation in conversations:\n",
    "            json_out = json.dumps(conversation) + \"\\n\"\n",
    "            out.write(json_out)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f01447c-b1ac-4cfe-b15d-bebf4c6cd2ee",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The first half of the dataset is used for training, and the remaining will be used for validation. The validation data is optional and is used to ensure that the model does not overfit the training set.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "38d0147a-36af-4fa9-947c-da62d0c6c39f",
   "metadata": {},
   "outputs": [],
   "source": [
    "training_file_name = 'training_data.jsonl'\n",
    "validation_file_name = 'validation_data.jsonl'\n",
    "\n",
    "# Training dataset\n",
    "write_jsonl(dataset[:86], training_file_name)\n",
    "\n",
    "# Validation dataset\n",
    "write_jsonl(dataset[87:172], validation_file_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2cbf92f8",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The above code will generate two files:</p>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'><li> <b>training_data.jsonl</b></li></ul>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'><li> <b>validation_data.jsonl</b></li></ul>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The different model types require various data formats; here, the format complies with the GPT-3.5 Turbo model. The training data and validation data sets consist of input and output examples for how we would like the model to perform.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f2824ff-4b80-47e4-bbcf-2132b847e8fd",
   "metadata": {},
   "source": [
    "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>5.3 Upload files to OpenAI</b></p>    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Next, we must upload the data files to the OpenAI's <b>Files</b> endpoint. The uploaded files are used for fine-tuning.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f9572c7",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><i><b>Note:</b> In case of folowing error:</i></p>\n",
    "<p style = 'font-size:14px;font-family:Arial;color:#00233C'><b>\"AttributeError: module 'openai' has no attribute 'OpenAI'\"</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><i>Make sure to have latest package installed of openai.</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe485d6c-ae3a-44cc-826d-5c796a1c10ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "client = openai.OpenAI(api_key=api_key)\n",
    "\n",
    "with open(training_file_name, \"rb\") as training_fd:\n",
    "    training_response = client.files.create(\n",
    "        file=training_fd, purpose=\"fine-tune\"\n",
    "    )\n",
    "\n",
    "training_file_id = training_response.id\n",
    "\n",
    "with open(validation_file_name, \"rb\") as validation_fd:\n",
    "    validation_response = client.files.create(\n",
    "        file=validation_fd, purpose=\"fine-tune\"\n",
    "    )\n",
    "validation_file_id = validation_response.id\n",
    "\n",
    "print(\"Training file ID:\", training_file_id)\n",
    "print(\"Validation file ID:\", validation_file_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40fa97bf-d23c-47b6-8ac9-244f93296a5d",
   "metadata": {},
   "source": [
    "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>5.4 Initiate Fine-tuning</b></p>    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>With the generated files and suffix(optional) to identify model in place, we are ready to create fine-tuning job and commence fine-tuning process. We can utilize generated file IDs and a model identifier to create fine-tuning job. The <b>id</b> returned in response will help us retrieve updates of the job.</p>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e51efbdf",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><i><b>Note:</b> We might get <i>files not ready</i> error because the processing is done on OpenAI system. In that case, retry after few minutes</i></p></div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4b577da6-e82c-452c-85d9-e6ed343223a5",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = client.fine_tuning.jobs.create(\n",
    "    training_file=training_file_id,\n",
    "    validation_file=validation_file_id,\n",
    "    model=\"gpt-3.5-turbo\",\n",
    "    suffix=\"mental-health\",\n",
    ")\n",
    "\n",
    "job_id = response.id\n",
    "\n",
    "print(\"Job ID:\", response.id)\n",
    "print(\"Status:\", response.status)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd73e62d-5b25-4a2f-8291-40767b465f7b",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><i><b>Note:</b> Run above cell only once or else the fine-tuning process will start again.</i></p></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "594bda9e",
   "metadata": {},
   "source": [
    " <div class=\"alert alert-block alert-warning\">\n",
    " <p style = 'font-size:16px;font-family:Arial;color:#00233C'><i><b>Note: The fine-tuning can take 10-15 minutes depending on the OpenAI server and number of tokens.</b></i></p>\n",
    " </div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67196452-284a-42cf-b679-8484ad7c1b1e",
   "metadata": {},
   "source": [
    "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>5.5 Check job status</b></p>    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Initially the fine-tuning status will read as <i>validating_files</i>, then <i>running</i>, and finally will turn to <i>status: succeeded</i> after competion of job.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "01cf1fc0-5d48-4984-84f0-bc667535c5e8",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = client.fine_tuning.jobs.retrieve(job_id)\n",
    "\n",
    "print(\"Job ID:\", response.id)\n",
    "print(\"Status:\", response.status)\n",
    "print(\"Trained Tokens:\", response.trained_tokens)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d1e8df3-6e92-4f60-a211-defdcd45e2b2",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Once it is completed, we can use the <b>result_files</b> to sample the results from the validation set (if we uploaded one), and use the ID from the <b>fine_tuned_model</b> parameter to invoke our trained model.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3a8d9c1-9b00-4d7c-9632-751ae253810c",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We can track the progress of the fine-tune with the events endpoint.</p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The below cell will show the status of fine-tuning process.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "945c2e05-06b3-4083-a72f-d88b8d8f5205",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Fine-tuning in progress..\")\n",
    "\n",
    "while 1:\n",
    "    sleep(3)\n",
    "    response = client.fine_tuning.jobs.list_events(job_id)\n",
    "    events = response.data\n",
    "    if events[0].data:\n",
    "        break\n",
    "\n",
    "while 1:\n",
    "    sleep(.5)\n",
    "    response = client.fine_tuning.jobs.list_events(job_id)\n",
    "    events = response.data\n",
    "    if events[0].data:\n",
    "        for event in events:\n",
    "            print(event.message, end=\"\\r\")\n",
    "    else:\n",
    "        print(\"\\nFine-tuning completed!\")\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "628d6f84",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><i><b>Note:</b> In case the above process is taking time, go to <a href=\\\"platform.openai.com/finetune/job_ID\\\">platform.openai.com/finetune/job_ID</a>, replace job_ID with a valid one and check the status of fine-tuning for details.</i></p>\n",
    "</div>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>After completion of above cell, we will get a fine-tuned model ID from the job, after that only we can run the below code.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "39c9fd75-d3a3-4d67-9da7-b9b03f859588",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = client.fine_tuning.jobs.retrieve(job_id)\n",
    "fine_tuned_model_id = response.fine_tuned_model\n",
    "\n",
    "if fine_tuned_model_id is None: \n",
    "    raise RuntimeError(\"Fine-tuned model ID not found. The job has likely not been completed yet.\")\n",
    "\n",
    "print(\"Fine-tuned model ID:\", fine_tuned_model_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06768a3b-4093-4ab9-99df-27280e2547dc",
   "metadata": {},
   "source": [
    "<hr style='height:2px;border:none;background-color:#00233C;'>\n",
    "<b style = 'font-size:20px;font-family:Arial;color:#00233c'>6. Testing the fine-tuned model</b>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Congratulations! The GPT-3.5 Turbo model has fine-tuned to the dataset on tackling mental health issues in today's world. To test the model, we can ask a question related to mental health issues.</p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>An example question to test the model: \"Who are you, and how can you help me?\"</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3ebd52ff-9a5b-4737-a01e-09a1142ca3f1",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_messages = []\n",
    "test_messages.append({\"role\": \"system\", \"content\": system_message})\n",
    "user_message = input(prompt=\"Please ask your question \")\n",
    "test_messages.append({\"role\": \"user\", \"content\": user_message})\n",
    "\n",
    "response = client.chat.completions.create(model=fine_tuned_model_id, messages=test_messages, temperature=0, max_tokens=500)\n",
    "print(response.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e51f19a-447d-4ccc-98ae-b7098eb89f83",
   "metadata": {},
   "source": [
    "<div id='section8'></div>\n",
    "<hr style='height:2px;border:none;background-color:#00233C;'>\n",
    "<b style = 'font-size:20px;font-family:Arial;color:#00233c'>7. Cleanup</b>\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>7.1 Work Tables</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Cleanup work tables to prevent errors next time.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e89ace58-2784-4a0f-b84b-6f10b128f5b7",
   "metadata": {},
   "outputs": [],
   "source": [
    "for table in [\"Mental_Health\"]:\n",
    "    try:\n",
    "        db_drop_table(table)\n",
    "\n",
    "    except:\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56b8aa49-a7cc-4c95-b23b-e085ec6dbbb8",
   "metadata": {},
   "source": [
    "<hr style='height:1px;border:none;background-color:#00233C;'>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233c'> <b>7.2 Databases and Tables </b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The following code will clean up tables and databases created above.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31eaee78-b567-4258-8789-0e3a92d23a93",
   "metadata": {},
   "outputs": [],
   "source": [
    "%run -i ../run_procedure.py \"call remove_data('DEMO_LLM_FineTuning');\"        # Takes 5 seconds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b242b35c-37c4-470d-9da4-bf1634a0aba2",
   "metadata": {},
   "outputs": [],
   "source": [
    "remove_context()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d75db563-4be1-4e6d-835d-4c023d8aeb18",
   "metadata": {},
   "source": [
    "<hr style='height:2px;border:none;background-color:#00233C;'>\n",
    "<b style = 'font-size:20px;font-family:Arial;color:#00233c'>Links</b>\n",
    "<ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "    <li>Teradataml Python reference: <a href = 'https://docs.teradata.com/search/all?query=Python+Package+User+Guide&content-lang=en-US'>here</a></li>\n",
    "    <li>OpenAI fine-tuning reference: <a href='https://platform.openai.com/docs/guides/fine-tuning'>here</a></li>\n",
    "</ul>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "849e799a-e70a-4636-b3af-40a37fa7e843",
   "metadata": {},
   "source": [
    "<footer style=\"padding-bottom:35px; background:#f9f9f9; border-bottom:3px solid #00233C\">\n",
    "    <div style=\"float:left;margin-top:14px\">ClearScape Analytics™</div>\n",
    "    <div style=\"float:right;\">\n",
    "        <div style=\"float:left; margin-top:14px\">\n",
    "            Copyright © Teradata Corporation - 2024. All Rights Reserved\n",
    "        </div>\n",
    "    </div>\n",
    "</footer>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}