diff --git a/notebooks/en/_toctree.yml b/notebooks/en/_toctree.yml
index e46efc44..4f84f2a1 100644
--- a/notebooks/en/_toctree.yml
+++ b/notebooks/en/_toctree.yml
@@ -82,6 +82,8 @@
           title: HuatuoGPT-o1 Medical RAG and Reasoning
         - local: fine_tune_chatbot_docs_synthetic
           title: Documentation Chatbot with Meta Synthetic Data Kit
+        - local: optuna_hpo_with_transformers
+          title: Hyperparameter Optimization with Optuna and Transformers
           
         
           
diff --git a/notebooks/en/optuna_hpo_with_transformers.ipynb b/notebooks/en/optuna_hpo_with_transformers.ipynb
new file mode 100644
index 00000000..8817350e
--- /dev/null
+++ b/notebooks/en/optuna_hpo_with_transformers.ipynb
@@ -0,0 +1,4991 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "08092aa8",
+   "metadata": {},
+   "source": [
+    "# Hyperparameter Optimization with Optuna and Transformers\n",
+    "\n",
+    "_Authored by: [Parag Ekbote](https://github.com/ParagEkbote)_\n",
+    "\n",
+    "In this notebook, we are going to use the [optuna](https://github.com/optuna/optuna) library to perform hyperparameter optimization on a light-weight BERT model using a small subset of the IMDB dataset. To learn more about transformers' hyperparameter search, you can check the following documentation [on hyperparemter optimization](https://huggingface.co/docs/transformers/en/hpo_train).\n",
+    "\n",
+    "Optuna allows us to automate the search for the best combination of hyperparameters (like learning rate or weight decay) to gives the best validation performance (e.g., accuracy or F1 score) when fine-tuning a pre-trained model like BERT on a dataset.\n",
+    "\n",
+    "Firstly, we will install the following dependencies for our code:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a309e1a0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install -q datasets evaluate transformers optuna wandb scikit-learn nbformat"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "eff9ccd6",
+   "metadata": {},
+   "source": [
+    "# Prepare the dataset and set the model\n",
+    "\n",
+    "We will load the  IMDB dataset which is a standard dataset for sentiment analysis. We will define 2000 examples for the training split and 1000 examples for validation. Both sets are shuffled with a fixed seed to ensure reproducibility.\n",
+    "\n",
+    "We shall also tokenize the text and map to efficiently preprocesses all the dataset samples. Next, we will load the accuracy metric. We will also initialize the model to be instantiated for binary classification. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2cfb9d5e",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "8ee77ecb1a0f41c2a715840e76cd5727",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "config.json:   0%|          | 0.00/285 [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "fd042613685f42dbbb6ff5c51d17b915",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "vocab.txt: 0.00B [00:00, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "07fb47d86d11433aacc67d4782090b17",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Map:   0%|          | 0/2500 [00:00<?, ? examples/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "d16d34949ee74f31bdc80b1bc75b2e19",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Map:   0%|          | 0/1000 [00:00<?, ? examples/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "from datasets import load_dataset\n",
+    "import evaluate\n",
+    "\n",
+    "from transformers import AutoModelForSequenceClassification\n",
+    "from transformers import AutoTokenizer\n",
+    "from transformers import set_seed\n",
+    "\n",
+    "\n",
+    "\n",
+    "set_seed(42)\n",
+    "\n",
+    "\n",
+    "train_dataset = load_dataset(\"imdb\", split=\"train\").shuffle(seed=42).select(range(2500))\n",
+    "valid_dataset = load_dataset(\"imdb\", split=\"test\").shuffle(seed=42).select(range(1000))\n",
+    "\n",
+    "model_name = \"prajjwal1/bert-tiny\"\n",
+    "tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
+    "\n",
+    "\n",
+    "def tokenize(batch):\n",
+    "    return tokenizer(batch[\"text\"], padding=\"max_length\", truncation=True, max_length=512)\n",
+    "\n",
+    "\n",
+    "tokenized_train = train_dataset.map(tokenize, batched=True).select_columns(\n",
+    "    [\"input_ids\", \"attention_mask\", \"label\"]\n",
+    ")\n",
+    "tokenized_valid = valid_dataset.map(tokenize, batched=True).select_columns(\n",
+    "    [\"input_ids\", \"attention_mask\", \"label\"]\n",
+    ")\n",
+    "\n",
+    "\n",
+    "metric = evaluate.load(\"accuracy\")\n",
+    "\n",
+    "\n",
+    "def model_init():\n",
+    "    return AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "105d9e01",
+   "metadata": {},
+   "source": [
+    "# Define Storage with Optuna\n",
+    "\n",
+    "To store all trials across sessions, we are going to be using `RDBStorage`, which allows all hyperparameter optimization trials to be stored in a  persistent SQLite database. This also allows for the Visualization and analysis of the trials to become more reproducible."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "7652b74b",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "[I 2025-06-25 15:10:18,724] A new study created in RDB with name: transformers_optuna_study\n"
+     ]
+    }
+   ],
+   "source": [
+    "import optuna\n",
+    "from optuna.storages import RDBStorage\n",
+    "\n",
+    "# Define persistent storage\n",
+    "storage = RDBStorage(\"sqlite:///optuna_trials.db\")\n",
+    "\n",
+    "\n",
+    "study = optuna.create_study(\n",
+    "    study_name=\"transformers_optuna_study\",\n",
+    "    direction=\"maximize\",\n",
+    "    storage=storage,\n",
+    "    load_if_exists=True\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5a46fac4",
+   "metadata": {},
+   "source": [
+    "# Initialize the Trainer Class and Setup Observability\n",
+    "\n",
+    "Now, we can define the metric function to calculate evaluation metrics after each eval step. We shall also define the objective function to maximize the accuracy when selecting the best hyperparameters. For observability, we can utilize Weight & Biases to log the hyperparameter trials. It is important to remember to login with your API key to Weight & Biases to track your trial.\n",
+    "\n",
+    "Finally, we will also define the training arguments for the Trainer that will handle the evaluation, checkpointing, logging and hyperparameter search."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "e0ee0bae",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mai_novice2005\u001b[0m to \u001b[32mhttps://api.wandb.ai\u001b[0m. Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [
+       "Tracking run with wandb version 0.20.1"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Run data is saved locally in <code>/teamspace/studios/this_studio/cookbook/notebooks/en/wandb/run-20250625_151029-ivr2ci8c</code>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Syncing run <strong><a href='https://wandb.ai/ai_novice2005/hf-optuna/runs/ivr2ci8c' target=\"_blank\">transformers_optuna_study</a></strong> to <a href='https://wandb.ai/ai_novice2005/hf-optuna' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/developer-guide' target=\"_blank\">docs</a>)<br>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View project at <a href='https://wandb.ai/ai_novice2005/hf-optuna' target=\"_blank\">https://wandb.ai/ai_novice2005/hf-optuna</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View run at <a href='https://wandb.ai/ai_novice2005/hf-optuna/runs/ivr2ci8c' target=\"_blank\">https://wandb.ai/ai_novice2005/hf-optuna/runs/ivr2ci8c</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "2a4b6835e125481597ce8871f368aeeb",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "pytorch_model.bin:   0%|          | 0.00/17.8M [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at prajjwal1/bert-tiny and are newly initialized: ['classifier.bias', 'classifier.weight']\n",
+      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
+     ]
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "a570d3ad87c44539a0eef1321dc34afc",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "model.safetensors:   0%|          | 0.00/17.7M [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "import wandb\n",
+    "from transformers import Trainer, TrainingArguments\n",
+    "\n",
+    "def compute_metrics(eval_pred):\n",
+    "    predictions = eval_pred.predictions.argmax(axis=-1)\n",
+    "    labels = eval_pred.label_ids\n",
+    "    return metric.compute(predictions=predictions, references=labels)\n",
+    "\n",
+    "\n",
+    "def compute_objective(metrics):\n",
+    "    return metrics[\"eval_accuracy\"]\n",
+    "\n",
+    "wandb.init(project=\"hf-optuna\", name=\"transformers_optuna_study\")\n",
+    "\n",
+    "training_args = TrainingArguments(\n",
+    "    output_dir=\"./results\",\n",
+    "        eval_strategy=\"epoch\",\n",
+    "        save_strategy=\"epoch\",\n",
+    "        load_best_model_at_end=True,\n",
+    "        logging_strategy=\"epoch\",\n",
+    "        num_train_epochs=3,\n",
+    "        report_to=\"wandb\",  # Logs to W&B\n",
+    "        logging_dir=\"./logs\",\n",
+    "        run_name=\"transformers_optuna_study\",\n",
+    ")\n",
+    "\n",
+    "\n",
+    "trainer = Trainer(\n",
+    "    model_init=model_init,\n",
+    "    args=training_args,\n",
+    "    train_dataset=tokenized_train,\n",
+    "    eval_dataset=tokenized_valid,\n",
+    "    processing_class=tokenizer,\n",
+    "    compute_metrics=compute_metrics,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b10c26c6",
+   "metadata": {},
+   "source": [
+    "# Define the Search Space and Start the Trials\n",
+    "\n",
+    "We will now define the optuna hyperparameter search space to find the best set of hyperparameters for learning rate, weight decay and batch size. We can now launch the hyperparameter search by passing the following metrics:\n",
+    "\n",
+    "1. direction: We aim to maxime the evaluation metric\n",
+    "2. backend: We will use optuna for searching\n",
+    "3. n_trials: The number of trials optuna will be executed \n",
+    "4. compute_objective: The objective to minimize or maximize from the metrics returned by `evaluate`\n",
+    "5. study_name:  The study name is used to retrieve or continue a specific run.\n",
+    "6. storage: The backend where Optuna will store all trial data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "4ef88312",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "[I 2025-06-25 15:10:41,259] Using an existing study with name 'transformers_optuna_study' instead of creating a new one.\n",
+      "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at prajjwal1/bert-tiny and are newly initialized: ['classifier.bias', 'classifier.weight']\n",
+      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View run <strong style=\"color:#cdcd00\">transformers_optuna_study</strong> at: <a href='https://wandb.ai/ai_novice2005/hf-optuna/runs/ivr2ci8c' target=\"_blank\">https://wandb.ai/ai_novice2005/hf-optuna/runs/ivr2ci8c</a><br> View project at: <a href='https://wandb.ai/ai_novice2005/hf-optuna' target=\"_blank\">https://wandb.ai/ai_novice2005/hf-optuna</a><br>Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Find logs at: <code>./wandb/run-20250625_151029-ivr2ci8c/logs</code>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Tracking run with wandb version 0.20.1"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Run data is saved locally in <code>/teamspace/studios/this_studio/cookbook/notebooks/en/wandb/run-20250625_151042-up8j8xgb</code>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Syncing run <strong><a href='https://wandb.ai/ai_novice2005/huggingface/runs/up8j8xgb' target=\"_blank\">leafy-breeze-5</a></strong> to <a href='https://wandb.ai/ai_novice2005/huggingface' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/developer-guide' target=\"_blank\">docs</a>)<br>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View project at <a href='https://wandb.ai/ai_novice2005/huggingface' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View run at <a href='https://wandb.ai/ai_novice2005/huggingface/runs/up8j8xgb' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface/runs/up8j8xgb</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "    <div>\n",
+       "      \n",
+       "      <progress value='471' max='471' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
+       "      [471/471 00:24, Epoch 3/3]\n",
+       "    </div>\n",
+       "    <table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       " <tr style=\"text-align: left;\">\n",
+       "      <th>Epoch</th>\n",
+       "      <th>Training Loss</th>\n",
+       "      <th>Validation Loss</th>\n",
+       "      <th>Accuracy</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <td>1</td>\n",
+       "      <td>0.649200</td>\n",
+       "      <td>0.605374</td>\n",
+       "      <td>0.682000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>2</td>\n",
+       "      <td>0.529900</td>\n",
+       "      <td>0.528273</td>\n",
+       "      <td>0.751000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>3</td>\n",
+       "      <td>0.440700</td>\n",
+       "      <td>0.509003</td>\n",
+       "      <td>0.764000</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table><p>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "[I 2025-06-25 15:11:09,298] Trial 0 finished with value: 0.764 and parameters: {'learning_rate': 7.23655165533393e-05, 'per_device_train_batch_size': 16, 'weight_decay': 0.013798094328723032}. Best is trial 0 with value: 0.764.\n",
+      "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at prajjwal1/bert-tiny and are newly initialized: ['classifier.bias', 'classifier.weight']\n",
+      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<br>    <style><br>        .wandb-row {<br>            display: flex;<br>            flex-direction: row;<br>            flex-wrap: wrap;<br>            justify-content: flex-start;<br>            width: 100%;<br>        }<br>        .wandb-col {<br>            display: flex;<br>            flex-direction: column;<br>            flex-basis: 100%;<br>            flex: 1;<br>            padding: 10px;<br>        }<br>    </style><br><div class=\"wandb-row\"><div class=\"wandb-col\"><h3>Run history:</h3><br/><table class=\"wandb\"><tr><td>eval/accuracy</td><td>▁▇█</td></tr><tr><td>eval/loss</td><td>█▂▁</td></tr><tr><td>eval/runtime</td><td>█▇▁</td></tr><tr><td>eval/samples_per_second</td><td>▁▂█</td></tr><tr><td>eval/steps_per_second</td><td>▁▂█</td></tr><tr><td>train/epoch</td><td>▁▁▅▅███</td></tr><tr><td>train/global_step</td><td>▁▁▄▄███</td></tr><tr><td>train/grad_norm</td><td>▁█▂</td></tr><tr><td>train/learning_rate</td><td>█▅▁</td></tr><tr><td>train/loss</td><td>█▄▁</td></tr></table><br/></div><div class=\"wandb-col\"><h3>Run summary:</h3><br/><table class=\"wandb\"><tr><td>eval/accuracy</td><td>0.764</td></tr><tr><td>eval/loss</td><td>0.509</td></tr><tr><td>eval/runtime</td><td>1.0937</td></tr><tr><td>eval/samples_per_second</td><td>914.299</td></tr><tr><td>eval/steps_per_second</td><td>114.287</td></tr><tr><td>total_flos</td><td>9528652800000.0</td></tr><tr><td>train/epoch</td><td>3</td></tr><tr><td>train/global_step</td><td>471</td></tr><tr><td>train/grad_norm</td><td>13.57101</td></tr><tr><td>train/learning_rate</td><td>0.0</td></tr><tr><td>train/loss</td><td>0.4407</td></tr><tr><td>train_loss</td><td>0.53993</td></tr><tr><td>train_runtime</td><td>26.9493</td></tr><tr><td>train_samples_per_second</td><td>278.3</td></tr><tr><td>train_steps_per_second</td><td>17.477</td></tr></table><br/></div></div>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View run <strong style=\"color:#cdcd00\">leafy-breeze-5</strong> at: <a href='https://wandb.ai/ai_novice2005/huggingface/runs/up8j8xgb' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface/runs/up8j8xgb</a><br> View project at: <a href='https://wandb.ai/ai_novice2005/huggingface' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface</a><br>Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Find logs at: <code>./wandb/run-20250625_151042-up8j8xgb/logs</code>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Tracking run with wandb version 0.20.1"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Run data is saved locally in <code>/teamspace/studios/this_studio/cookbook/notebooks/en/wandb/run-20250625_151110-1dgqb1s1</code>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Syncing run <strong><a href='https://wandb.ai/ai_novice2005/huggingface/runs/1dgqb1s1' target=\"_blank\">radiant-sound-6</a></strong> to <a href='https://wandb.ai/ai_novice2005/huggingface' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/developer-guide' target=\"_blank\">docs</a>)<br>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View project at <a href='https://wandb.ai/ai_novice2005/huggingface' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View run at <a href='https://wandb.ai/ai_novice2005/huggingface/runs/1dgqb1s1' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface/runs/1dgqb1s1</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "    <div>\n",
+       "      \n",
+       "      <progress value='60' max='60' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
+       "      [60/60 00:18, Epoch 3/3]\n",
+       "    </div>\n",
+       "    <table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       " <tr style=\"text-align: left;\">\n",
+       "      <th>Epoch</th>\n",
+       "      <th>Training Loss</th>\n",
+       "      <th>Validation Loss</th>\n",
+       "      <th>Accuracy</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <td>1</td>\n",
+       "      <td>0.683100</td>\n",
+       "      <td>0.677468</td>\n",
+       "      <td>0.613000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>2</td>\n",
+       "      <td>0.673100</td>\n",
+       "      <td>0.669755</td>\n",
+       "      <td>0.639000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>3</td>\n",
+       "      <td>0.669500</td>\n",
+       "      <td>0.667655</td>\n",
+       "      <td>0.630000</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table><p>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "[I 2025-06-25 15:11:29,907] Trial 1 finished with value: 0.63 and parameters: {'learning_rate': 2.756288216246014e-05, 'per_device_train_batch_size': 128, 'weight_decay': 0.28503663896216014}. Best is trial 0 with value: 0.764.\n",
+      "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at prajjwal1/bert-tiny and are newly initialized: ['classifier.bias', 'classifier.weight']\n",
+      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<br>    <style><br>        .wandb-row {<br>            display: flex;<br>            flex-direction: row;<br>            flex-wrap: wrap;<br>            justify-content: flex-start;<br>            width: 100%;<br>        }<br>        .wandb-col {<br>            display: flex;<br>            flex-direction: column;<br>            flex-basis: 100%;<br>            flex: 1;<br>            padding: 10px;<br>        }<br>    </style><br><div class=\"wandb-row\"><div class=\"wandb-col\"><h3>Run history:</h3><br/><table class=\"wandb\"><tr><td>eval/accuracy</td><td>▁█▆</td></tr><tr><td>eval/loss</td><td>█▂▁</td></tr><tr><td>eval/runtime</td><td>▁█▂</td></tr><tr><td>eval/samples_per_second</td><td>█▁▇</td></tr><tr><td>eval/steps_per_second</td><td>█▁▇</td></tr><tr><td>train/epoch</td><td>▁▁▅▅███</td></tr><tr><td>train/global_step</td><td>▁▁▅▅███</td></tr><tr><td>train/grad_norm</td><td>▅█▁</td></tr><tr><td>train/learning_rate</td><td>█▄▁</td></tr><tr><td>train/loss</td><td>█▃▁</td></tr></table><br/></div><div class=\"wandb-col\"><h3>Run summary:</h3><br/><table class=\"wandb\"><tr><td>eval/accuracy</td><td>0.63</td></tr><tr><td>eval/loss</td><td>0.66765</td></tr><tr><td>eval/runtime</td><td>1.111</td></tr><tr><td>eval/samples_per_second</td><td>900.116</td></tr><tr><td>eval/steps_per_second</td><td>112.515</td></tr><tr><td>total_flos</td><td>9528652800000.0</td></tr><tr><td>train/epoch</td><td>3</td></tr><tr><td>train/global_step</td><td>60</td></tr><tr><td>train/grad_norm</td><td>0.66353</td></tr><tr><td>train/learning_rate</td><td>0.0</td></tr><tr><td>train/loss</td><td>0.6695</td></tr><tr><td>train_loss</td><td>0.67521</td></tr><tr><td>train_runtime</td><td>19.5595</td></tr><tr><td>train_samples_per_second</td><td>383.445</td></tr><tr><td>train_steps_per_second</td><td>3.068</td></tr></table><br/></div></div>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View run <strong style=\"color:#cdcd00\">radiant-sound-6</strong> at: <a href='https://wandb.ai/ai_novice2005/huggingface/runs/1dgqb1s1' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface/runs/1dgqb1s1</a><br> View project at: <a href='https://wandb.ai/ai_novice2005/huggingface' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface</a><br>Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Find logs at: <code>./wandb/run-20250625_151110-1dgqb1s1/logs</code>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Tracking run with wandb version 0.20.1"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Run data is saved locally in <code>/teamspace/studios/this_studio/cookbook/notebooks/en/wandb/run-20250625_151130-jt5mavd3</code>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Syncing run <strong><a href='https://wandb.ai/ai_novice2005/huggingface/runs/jt5mavd3' target=\"_blank\">ancient-dream-7</a></strong> to <a href='https://wandb.ai/ai_novice2005/huggingface' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/developer-guide' target=\"_blank\">docs</a>)<br>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View project at <a href='https://wandb.ai/ai_novice2005/huggingface' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View run at <a href='https://wandb.ai/ai_novice2005/huggingface/runs/jt5mavd3' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface/runs/jt5mavd3</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "    <div>\n",
+       "      \n",
+       "      <progress value='120' max='120' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
+       "      [120/120 00:20, Epoch 3/3]\n",
+       "    </div>\n",
+       "    <table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       " <tr style=\"text-align: left;\">\n",
+       "      <th>Epoch</th>\n",
+       "      <th>Training Loss</th>\n",
+       "      <th>Validation Loss</th>\n",
+       "      <th>Accuracy</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <td>1</td>\n",
+       "      <td>0.690300</td>\n",
+       "      <td>0.688425</td>\n",
+       "      <td>0.553000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>2</td>\n",
+       "      <td>0.689100</td>\n",
+       "      <td>0.687775</td>\n",
+       "      <td>0.562000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>3</td>\n",
+       "      <td>0.689100</td>\n",
+       "      <td>0.687576</td>\n",
+       "      <td>0.570000</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table><p>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "[I 2025-06-25 15:11:52,797] Trial 2 finished with value: 0.57 and parameters: {'learning_rate': 1.2177346043359053e-06, 'per_device_train_batch_size': 64, 'weight_decay': 0.02906341093983704}. Best is trial 0 with value: 0.764.\n",
+      "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at prajjwal1/bert-tiny and are newly initialized: ['classifier.bias', 'classifier.weight']\n",
+      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<br>    <style><br>        .wandb-row {<br>            display: flex;<br>            flex-direction: row;<br>            flex-wrap: wrap;<br>            justify-content: flex-start;<br>            width: 100%;<br>        }<br>        .wandb-col {<br>            display: flex;<br>            flex-direction: column;<br>            flex-basis: 100%;<br>            flex: 1;<br>            padding: 10px;<br>        }<br>    </style><br><div class=\"wandb-row\"><div class=\"wandb-col\"><h3>Run history:</h3><br/><table class=\"wandb\"><tr><td>eval/accuracy</td><td>▁▅█</td></tr><tr><td>eval/loss</td><td>█▃▁</td></tr><tr><td>eval/runtime</td><td>▁██</td></tr><tr><td>eval/samples_per_second</td><td>█▁▁</td></tr><tr><td>eval/steps_per_second</td><td>█▁▁</td></tr><tr><td>train/epoch</td><td>▁▁▅▅███</td></tr><tr><td>train/global_step</td><td>▁▁▅▅███</td></tr><tr><td>train/grad_norm</td><td>▆█▁</td></tr><tr><td>train/learning_rate</td><td>█▅▁</td></tr><tr><td>train/loss</td><td>█▁▁</td></tr></table><br/></div><div class=\"wandb-col\"><h3>Run summary:</h3><br/><table class=\"wandb\"><tr><td>eval/accuracy</td><td>0.57</td></tr><tr><td>eval/loss</td><td>0.68758</td></tr><tr><td>eval/runtime</td><td>1.0959</td></tr><tr><td>eval/samples_per_second</td><td>912.502</td></tr><tr><td>eval/steps_per_second</td><td>114.063</td></tr><tr><td>total_flos</td><td>9528652800000.0</td></tr><tr><td>train/epoch</td><td>3</td></tr><tr><td>train/global_step</td><td>120</td></tr><tr><td>train/grad_norm</td><td>2.34479</td></tr><tr><td>train/learning_rate</td><td>0.0</td></tr><tr><td>train/loss</td><td>0.6891</td></tr><tr><td>train_loss</td><td>0.68947</td></tr><tr><td>train_runtime</td><td>21.6288</td></tr><tr><td>train_samples_per_second</td><td>346.76</td></tr><tr><td>train_steps_per_second</td><td>5.548</td></tr></table><br/></div></div>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View run <strong style=\"color:#cdcd00\">ancient-dream-7</strong> at: <a href='https://wandb.ai/ai_novice2005/huggingface/runs/jt5mavd3' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface/runs/jt5mavd3</a><br> View project at: <a href='https://wandb.ai/ai_novice2005/huggingface' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface</a><br>Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Find logs at: <code>./wandb/run-20250625_151130-jt5mavd3/logs</code>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Tracking run with wandb version 0.20.1"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Run data is saved locally in <code>/teamspace/studios/this_studio/cookbook/notebooks/en/wandb/run-20250625_151153-6eexo8uv</code>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Syncing run <strong><a href='https://wandb.ai/ai_novice2005/huggingface/runs/6eexo8uv' target=\"_blank\">grateful-sound-8</a></strong> to <a href='https://wandb.ai/ai_novice2005/huggingface' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/developer-guide' target=\"_blank\">docs</a>)<br>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View project at <a href='https://wandb.ai/ai_novice2005/huggingface' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View run at <a href='https://wandb.ai/ai_novice2005/huggingface/runs/6eexo8uv' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface/runs/6eexo8uv</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "    <div>\n",
+       "      \n",
+       "      <progress value='120' max='120' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
+       "      [120/120 00:16, Epoch 3/3]\n",
+       "    </div>\n",
+       "    <table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       " <tr style=\"text-align: left;\">\n",
+       "      <th>Epoch</th>\n",
+       "      <th>Training Loss</th>\n",
+       "      <th>Validation Loss</th>\n",
+       "      <th>Accuracy</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <td>1</td>\n",
+       "      <td>0.689400</td>\n",
+       "      <td>0.686730</td>\n",
+       "      <td>0.570000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>2</td>\n",
+       "      <td>0.687000</td>\n",
+       "      <td>0.685327</td>\n",
+       "      <td>0.581000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>3</td>\n",
+       "      <td>0.686700</td>\n",
+       "      <td>0.684904</td>\n",
+       "      <td>0.581000</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table><p>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "[I 2025-06-25 15:12:12,894] Trial 3 finished with value: 0.581 and parameters: {'learning_rate': 2.973185825213819e-06, 'per_device_train_batch_size': 64, 'weight_decay': 0.09102292466460353}. Best is trial 0 with value: 0.764.\n",
+      "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at prajjwal1/bert-tiny and are newly initialized: ['classifier.bias', 'classifier.weight']\n",
+      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<br>    <style><br>        .wandb-row {<br>            display: flex;<br>            flex-direction: row;<br>            flex-wrap: wrap;<br>            justify-content: flex-start;<br>            width: 100%;<br>        }<br>        .wandb-col {<br>            display: flex;<br>            flex-direction: column;<br>            flex-basis: 100%;<br>            flex: 1;<br>            padding: 10px;<br>        }<br>    </style><br><div class=\"wandb-row\"><div class=\"wandb-col\"><h3>Run history:</h3><br/><table class=\"wandb\"><tr><td>eval/accuracy</td><td>▁██</td></tr><tr><td>eval/loss</td><td>█▃▁</td></tr><tr><td>eval/runtime</td><td>▁█▃</td></tr><tr><td>eval/samples_per_second</td><td>█▁▆</td></tr><tr><td>eval/steps_per_second</td><td>█▁▆</td></tr><tr><td>train/epoch</td><td>▁▁▅▅███</td></tr><tr><td>train/global_step</td><td>▁▁▅▅███</td></tr><tr><td>train/grad_norm</td><td>▆█▁</td></tr><tr><td>train/learning_rate</td><td>█▅▁</td></tr><tr><td>train/loss</td><td>█▂▁</td></tr></table><br/></div><div class=\"wandb-col\"><h3>Run summary:</h3><br/><table class=\"wandb\"><tr><td>eval/accuracy</td><td>0.581</td></tr><tr><td>eval/loss</td><td>0.6849</td></tr><tr><td>eval/runtime</td><td>1.0808</td></tr><tr><td>eval/samples_per_second</td><td>925.219</td></tr><tr><td>eval/steps_per_second</td><td>115.652</td></tr><tr><td>total_flos</td><td>9528652800000.0</td></tr><tr><td>train/epoch</td><td>3</td></tr><tr><td>train/global_step</td><td>120</td></tr><tr><td>train/grad_norm</td><td>2.30065</td></tr><tr><td>train/learning_rate</td><td>0.0</td></tr><tr><td>train/loss</td><td>0.6867</td></tr><tr><td>train_loss</td><td>0.68768</td></tr><tr><td>train_runtime</td><td>18.9103</td></tr><tr><td>train_samples_per_second</td><td>396.61</td></tr><tr><td>train_steps_per_second</td><td>6.346</td></tr></table><br/></div></div>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View run <strong style=\"color:#cdcd00\">grateful-sound-8</strong> at: <a href='https://wandb.ai/ai_novice2005/huggingface/runs/6eexo8uv' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface/runs/6eexo8uv</a><br> View project at: <a href='https://wandb.ai/ai_novice2005/huggingface' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface</a><br>Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Find logs at: <code>./wandb/run-20250625_151153-6eexo8uv/logs</code>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Tracking run with wandb version 0.20.1"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Run data is saved locally in <code>/teamspace/studios/this_studio/cookbook/notebooks/en/wandb/run-20250625_151213-5w18j6iv</code>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "Syncing run <strong><a href='https://wandb.ai/ai_novice2005/huggingface/runs/5w18j6iv' target=\"_blank\">hopeful-moon-9</a></strong> to <a href='https://wandb.ai/ai_novice2005/huggingface' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/developer-guide' target=\"_blank\">docs</a>)<br>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View project at <a href='https://wandb.ai/ai_novice2005/huggingface' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       " View run at <a href='https://wandb.ai/ai_novice2005/huggingface/runs/5w18j6iv' target=\"_blank\">https://wandb.ai/ai_novice2005/huggingface/runs/5w18j6iv</a>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "    <div>\n",
+       "      \n",
+       "      <progress value='120' max='120' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
+       "      [120/120 00:17, Epoch 3/3]\n",
+       "    </div>\n",
+       "    <table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       " <tr style=\"text-align: left;\">\n",
+       "      <th>Epoch</th>\n",
+       "      <th>Training Loss</th>\n",
+       "      <th>Validation Loss</th>\n",
+       "      <th>Accuracy</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <td>1</td>\n",
+       "      <td>0.689000</td>\n",
+       "      <td>0.686028</td>\n",
+       "      <td>0.573000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>2</td>\n",
+       "      <td>0.686100</td>\n",
+       "      <td>0.684337</td>\n",
+       "      <td>0.589000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>3</td>\n",
+       "      <td>0.685700</td>\n",
+       "      <td>0.683833</td>\n",
+       "      <td>0.597000</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table><p>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "[I 2025-06-25 15:12:32,824] Trial 4 finished with value: 0.597 and parameters: {'learning_rate': 3.763988365260261e-06, 'per_device_train_batch_size': 64, 'weight_decay': 0.1502192542358606}. Best is trial 0 with value: 0.764.\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "BestRun(run_id='0', objective=0.764, hyperparameters={'learning_rate': 7.23655165533393e-05, 'per_device_train_batch_size': 16, 'weight_decay': 0.013798094328723032}, run_summary=None)\n"
+     ]
+    }
+   ],
+   "source": [
+    "def optuna_hp_space(trial):\n",
+    "    return {\n",
+    "        \"learning_rate\": trial.suggest_float(\"learning_rate\", 1e-6, 1e-4, log=True),\n",
+    "        \"per_device_train_batch_size\": trial.suggest_categorical(\n",
+    "            \"per_device_train_batch_size\", [16, 32, 64, 128]\n",
+    "        ),\n",
+    "        \"weight_decay\": trial.suggest_float(\"weight_decay\", 0.0, 0.3),\n",
+    "    }\n",
+    "\n",
+    "\n",
+    "best_run = trainer.hyperparameter_search(\n",
+    "    direction=\"maximize\",\n",
+    "    backend=\"optuna\",\n",
+    "    hp_space=optuna_hp_space,\n",
+    "    n_trials=5,\n",
+    "    compute_objective=compute_objective,\n",
+    "    study_name=\"transformers_optuna_study\",\n",
+    "    storage=\"sqlite:///optuna_trials.db\",\n",
+    "    load_if_exists=True\n",
+    ")\n",
+    "\n",
+    "print(best_run)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "26a95ef3",
+   "metadata": {},
+   "source": [
+    "# Visualize the results\n",
+    "\n",
+    "After the completion of the trials, we can visualize the results in a simple manner using the `optuna` study object.\n",
+    "We can pass the object and plot visualizations that can help to understand the patterns in the trial outcomes. Here, we are plotting the key hyperparameters and how different hyperparameter combinations relate to performance of the model."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "a8f14007",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.plotly.v1+json": {
+       "config": {
+        "plotlyServerURL": "https://plot.ly"
+       },
+       "data": [
+        {
+         "cliponaxis": false,
+         "hovertemplate": [
+          "learning_rate (FloatDistribution): 0.2388176156849901<extra></extra>",
+          "weight_decay (FloatDistribution): 0.31096699871603595<extra></extra>",
+          "per_device_train_batch_size (CategoricalDistribution): 0.45021538559897395<extra></extra>"
+         ],
+         "name": "Objective Value",
+         "orientation": "h",
+         "text": [
+          "0.24",
+          "0.31",
+          "0.45"
+         ],
+         "textposition": "outside",
+         "type": "bar",
+         "x": [
+          0.2388176156849901,
+          0.31096699871603595,
+          0.45021538559897395
+         ],
+         "y": [
+          "learning_rate",
+          "weight_decay",
+          "per_device_train_batch_size"
+         ]
+        }
+       ],
+       "layout": {
+        "template": {
+         "data": {
+          "bar": [
+           {
+            "error_x": {
+             "color": "#2a3f5f"
+            },
+            "error_y": {
+             "color": "#2a3f5f"
+            },
+            "marker": {
+             "line": {
+              "color": "#E5ECF6",
+              "width": 0.5
+             },
+             "pattern": {
+              "fillmode": "overlay",
+              "size": 10,
+              "solidity": 0.2
+             }
+            },
+            "type": "bar"
+           }
+          ],
+          "barpolar": [
+           {
+            "marker": {
+             "line": {
+              "color": "#E5ECF6",
+              "width": 0.5
+             },
+             "pattern": {
+              "fillmode": "overlay",
+              "size": 10,
+              "solidity": 0.2
+             }
+            },
+            "type": "barpolar"
+           }
+          ],
+          "carpet": [
+           {
+            "aaxis": {
+             "endlinecolor": "#2a3f5f",
+             "gridcolor": "white",
+             "linecolor": "white",
+             "minorgridcolor": "white",
+             "startlinecolor": "#2a3f5f"
+            },
+            "baxis": {
+             "endlinecolor": "#2a3f5f",
+             "gridcolor": "white",
+             "linecolor": "white",
+             "minorgridcolor": "white",
+             "startlinecolor": "#2a3f5f"
+            },
+            "type": "carpet"
+           }
+          ],
+          "choropleth": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "type": "choropleth"
+           }
+          ],
+          "contour": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "colorscale": [
+             [
+              0,
+              "#0d0887"
+             ],
+             [
+              0.1111111111111111,
+              "#46039f"
+             ],
+             [
+              0.2222222222222222,
+              "#7201a8"
+             ],
+             [
+              0.3333333333333333,
+              "#9c179e"
+             ],
+             [
+              0.4444444444444444,
+              "#bd3786"
+             ],
+             [
+              0.5555555555555556,
+              "#d8576b"
+             ],
+             [
+              0.6666666666666666,
+              "#ed7953"
+             ],
+             [
+              0.7777777777777778,
+              "#fb9f3a"
+             ],
+             [
+              0.8888888888888888,
+              "#fdca26"
+             ],
+             [
+              1,
+              "#f0f921"
+             ]
+            ],
+            "type": "contour"
+           }
+          ],
+          "contourcarpet": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "type": "contourcarpet"
+           }
+          ],
+          "heatmap": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "colorscale": [
+             [
+              0,
+              "#0d0887"
+             ],
+             [
+              0.1111111111111111,
+              "#46039f"
+             ],
+             [
+              0.2222222222222222,
+              "#7201a8"
+             ],
+             [
+              0.3333333333333333,
+              "#9c179e"
+             ],
+             [
+              0.4444444444444444,
+              "#bd3786"
+             ],
+             [
+              0.5555555555555556,
+              "#d8576b"
+             ],
+             [
+              0.6666666666666666,
+              "#ed7953"
+             ],
+             [
+              0.7777777777777778,
+              "#fb9f3a"
+             ],
+             [
+              0.8888888888888888,
+              "#fdca26"
+             ],
+             [
+              1,
+              "#f0f921"
+             ]
+            ],
+            "type": "heatmap"
+           }
+          ],
+          "histogram": [
+           {
+            "marker": {
+             "pattern": {
+              "fillmode": "overlay",
+              "size": 10,
+              "solidity": 0.2
+             }
+            },
+            "type": "histogram"
+           }
+          ],
+          "histogram2d": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "colorscale": [
+             [
+              0,
+              "#0d0887"
+             ],
+             [
+              0.1111111111111111,
+              "#46039f"
+             ],
+             [
+              0.2222222222222222,
+              "#7201a8"
+             ],
+             [
+              0.3333333333333333,
+              "#9c179e"
+             ],
+             [
+              0.4444444444444444,
+              "#bd3786"
+             ],
+             [
+              0.5555555555555556,
+              "#d8576b"
+             ],
+             [
+              0.6666666666666666,
+              "#ed7953"
+             ],
+             [
+              0.7777777777777778,
+              "#fb9f3a"
+             ],
+             [
+              0.8888888888888888,
+              "#fdca26"
+             ],
+             [
+              1,
+              "#f0f921"
+             ]
+            ],
+            "type": "histogram2d"
+           }
+          ],
+          "histogram2dcontour": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "colorscale": [
+             [
+              0,
+              "#0d0887"
+             ],
+             [
+              0.1111111111111111,
+              "#46039f"
+             ],
+             [
+              0.2222222222222222,
+              "#7201a8"
+             ],
+             [
+              0.3333333333333333,
+              "#9c179e"
+             ],
+             [
+              0.4444444444444444,
+              "#bd3786"
+             ],
+             [
+              0.5555555555555556,
+              "#d8576b"
+             ],
+             [
+              0.6666666666666666,
+              "#ed7953"
+             ],
+             [
+              0.7777777777777778,
+              "#fb9f3a"
+             ],
+             [
+              0.8888888888888888,
+              "#fdca26"
+             ],
+             [
+              1,
+              "#f0f921"
+             ]
+            ],
+            "type": "histogram2dcontour"
+           }
+          ],
+          "mesh3d": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "type": "mesh3d"
+           }
+          ],
+          "parcoords": [
+           {
+            "line": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "parcoords"
+           }
+          ],
+          "pie": [
+           {
+            "automargin": true,
+            "type": "pie"
+           }
+          ],
+          "scatter": [
+           {
+            "fillpattern": {
+             "fillmode": "overlay",
+             "size": 10,
+             "solidity": 0.2
+            },
+            "type": "scatter"
+           }
+          ],
+          "scatter3d": [
+           {
+            "line": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scatter3d"
+           }
+          ],
+          "scattercarpet": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scattercarpet"
+           }
+          ],
+          "scattergeo": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scattergeo"
+           }
+          ],
+          "scattergl": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scattergl"
+           }
+          ],
+          "scattermap": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scattermap"
+           }
+          ],
+          "scattermapbox": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scattermapbox"
+           }
+          ],
+          "scatterpolar": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scatterpolar"
+           }
+          ],
+          "scatterpolargl": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scatterpolargl"
+           }
+          ],
+          "scatterternary": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scatterternary"
+           }
+          ],
+          "surface": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "colorscale": [
+             [
+              0,
+              "#0d0887"
+             ],
+             [
+              0.1111111111111111,
+              "#46039f"
+             ],
+             [
+              0.2222222222222222,
+              "#7201a8"
+             ],
+             [
+              0.3333333333333333,
+              "#9c179e"
+             ],
+             [
+              0.4444444444444444,
+              "#bd3786"
+             ],
+             [
+              0.5555555555555556,
+              "#d8576b"
+             ],
+             [
+              0.6666666666666666,
+              "#ed7953"
+             ],
+             [
+              0.7777777777777778,
+              "#fb9f3a"
+             ],
+             [
+              0.8888888888888888,
+              "#fdca26"
+             ],
+             [
+              1,
+              "#f0f921"
+             ]
+            ],
+            "type": "surface"
+           }
+          ],
+          "table": [
+           {
+            "cells": {
+             "fill": {
+              "color": "#EBF0F8"
+             },
+             "line": {
+              "color": "white"
+             }
+            },
+            "header": {
+             "fill": {
+              "color": "#C8D4E3"
+             },
+             "line": {
+              "color": "white"
+             }
+            },
+            "type": "table"
+           }
+          ]
+         },
+         "layout": {
+          "annotationdefaults": {
+           "arrowcolor": "#2a3f5f",
+           "arrowhead": 0,
+           "arrowwidth": 1
+          },
+          "autotypenumbers": "strict",
+          "coloraxis": {
+           "colorbar": {
+            "outlinewidth": 0,
+            "ticks": ""
+           }
+          },
+          "colorscale": {
+           "diverging": [
+            [
+             0,
+             "#8e0152"
+            ],
+            [
+             0.1,
+             "#c51b7d"
+            ],
+            [
+             0.2,
+             "#de77ae"
+            ],
+            [
+             0.3,
+             "#f1b6da"
+            ],
+            [
+             0.4,
+             "#fde0ef"
+            ],
+            [
+             0.5,
+             "#f7f7f7"
+            ],
+            [
+             0.6,
+             "#e6f5d0"
+            ],
+            [
+             0.7,
+             "#b8e186"
+            ],
+            [
+             0.8,
+             "#7fbc41"
+            ],
+            [
+             0.9,
+             "#4d9221"
+            ],
+            [
+             1,
+             "#276419"
+            ]
+           ],
+           "sequential": [
+            [
+             0,
+             "#0d0887"
+            ],
+            [
+             0.1111111111111111,
+             "#46039f"
+            ],
+            [
+             0.2222222222222222,
+             "#7201a8"
+            ],
+            [
+             0.3333333333333333,
+             "#9c179e"
+            ],
+            [
+             0.4444444444444444,
+             "#bd3786"
+            ],
+            [
+             0.5555555555555556,
+             "#d8576b"
+            ],
+            [
+             0.6666666666666666,
+             "#ed7953"
+            ],
+            [
+             0.7777777777777778,
+             "#fb9f3a"
+            ],
+            [
+             0.8888888888888888,
+             "#fdca26"
+            ],
+            [
+             1,
+             "#f0f921"
+            ]
+           ],
+           "sequentialminus": [
+            [
+             0,
+             "#0d0887"
+            ],
+            [
+             0.1111111111111111,
+             "#46039f"
+            ],
+            [
+             0.2222222222222222,
+             "#7201a8"
+            ],
+            [
+             0.3333333333333333,
+             "#9c179e"
+            ],
+            [
+             0.4444444444444444,
+             "#bd3786"
+            ],
+            [
+             0.5555555555555556,
+             "#d8576b"
+            ],
+            [
+             0.6666666666666666,
+             "#ed7953"
+            ],
+            [
+             0.7777777777777778,
+             "#fb9f3a"
+            ],
+            [
+             0.8888888888888888,
+             "#fdca26"
+            ],
+            [
+             1,
+             "#f0f921"
+            ]
+           ]
+          },
+          "colorway": [
+           "#636efa",
+           "#EF553B",
+           "#00cc96",
+           "#ab63fa",
+           "#FFA15A",
+           "#19d3f3",
+           "#FF6692",
+           "#B6E880",
+           "#FF97FF",
+           "#FECB52"
+          ],
+          "font": {
+           "color": "#2a3f5f"
+          },
+          "geo": {
+           "bgcolor": "white",
+           "lakecolor": "white",
+           "landcolor": "#E5ECF6",
+           "showlakes": true,
+           "showland": true,
+           "subunitcolor": "white"
+          },
+          "hoverlabel": {
+           "align": "left"
+          },
+          "hovermode": "closest",
+          "mapbox": {
+           "style": "light"
+          },
+          "paper_bgcolor": "white",
+          "plot_bgcolor": "#E5ECF6",
+          "polar": {
+           "angularaxis": {
+            "gridcolor": "white",
+            "linecolor": "white",
+            "ticks": ""
+           },
+           "bgcolor": "#E5ECF6",
+           "radialaxis": {
+            "gridcolor": "white",
+            "linecolor": "white",
+            "ticks": ""
+           }
+          },
+          "scene": {
+           "xaxis": {
+            "backgroundcolor": "#E5ECF6",
+            "gridcolor": "white",
+            "gridwidth": 2,
+            "linecolor": "white",
+            "showbackground": true,
+            "ticks": "",
+            "zerolinecolor": "white"
+           },
+           "yaxis": {
+            "backgroundcolor": "#E5ECF6",
+            "gridcolor": "white",
+            "gridwidth": 2,
+            "linecolor": "white",
+            "showbackground": true,
+            "ticks": "",
+            "zerolinecolor": "white"
+           },
+           "zaxis": {
+            "backgroundcolor": "#E5ECF6",
+            "gridcolor": "white",
+            "gridwidth": 2,
+            "linecolor": "white",
+            "showbackground": true,
+            "ticks": "",
+            "zerolinecolor": "white"
+           }
+          },
+          "shapedefaults": {
+           "line": {
+            "color": "#2a3f5f"
+           }
+          },
+          "ternary": {
+           "aaxis": {
+            "gridcolor": "white",
+            "linecolor": "white",
+            "ticks": ""
+           },
+           "baxis": {
+            "gridcolor": "white",
+            "linecolor": "white",
+            "ticks": ""
+           },
+           "bgcolor": "#E5ECF6",
+           "caxis": {
+            "gridcolor": "white",
+            "linecolor": "white",
+            "ticks": ""
+           }
+          },
+          "title": {
+           "x": 0.05
+          },
+          "xaxis": {
+           "automargin": true,
+           "gridcolor": "white",
+           "linecolor": "white",
+           "ticks": "",
+           "title": {
+            "standoff": 15
+           },
+           "zerolinecolor": "white",
+           "zerolinewidth": 2
+          },
+          "yaxis": {
+           "automargin": true,
+           "gridcolor": "white",
+           "linecolor": "white",
+           "ticks": "",
+           "title": {
+            "standoff": 15
+           },
+           "zerolinecolor": "white",
+           "zerolinewidth": 2
+          }
+         }
+        },
+        "title": {
+         "text": "Hyperparameter Importances"
+        },
+        "xaxis": {
+         "title": {
+          "text": "Hyperparameter Importance"
+         }
+        },
+        "yaxis": {
+         "title": {
+          "text": "Hyperparameter"
+         }
+        }
+       }
+      }
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.plotly.v1+json": {
+       "config": {
+        "plotlyServerURL": "https://plot.ly"
+       },
+       "data": [
+        {
+         "dimensions": [
+          {
+           "label": "Objective Value",
+           "range": [
+            0.57,
+            0.764
+           ],
+           "values": [
+            0.764,
+            0.57,
+            0.581,
+            0.597,
+            0.63
+           ]
+          },
+          {
+           "label": "learning_rate",
+           "range": [
+            -5.914447352451514,
+            -4.140468333501643
+           ],
+           "ticktext": [
+            "1.22e-06",
+            "1e-05",
+            "7.24e-05"
+           ],
+           "tickvals": [
+            -5.914447352451514,
+            -5,
+            -4.140468333501643
+           ],
+           "values": [
+            -4.140468333501643,
+            -5.914447352451514,
+            -5.526777946381027,
+            -5.424351727675991,
+            -4.559675371604521
+           ]
+          },
+          {
+           "label": "per_device_train_...",
+           "range": [
+            0,
+            2
+           ],
+           "ticktext": [
+            "16",
+            "64",
+            "128"
+           ],
+           "tickvals": [
+            0,
+            1,
+            2
+           ],
+           "values": [
+            0,
+            1,
+            1,
+            1,
+            2
+           ]
+          },
+          {
+           "label": "weight_decay",
+           "range": [
+            0.013798094328723032,
+            0.28503663896216014
+           ],
+           "values": [
+            0.013798094328723032,
+            0.02906341093983704,
+            0.09102292466460353,
+            0.1502192542358606,
+            0.28503663896216014
+           ]
+          }
+         ],
+         "labelangle": 30,
+         "labelside": "bottom",
+         "line": {
+          "color": [
+           0.764,
+           0.57,
+           0.581,
+           0.597,
+           0.63
+          ],
+          "colorbar": {
+           "title": {
+            "text": "Objective Value"
+           }
+          },
+          "colorscale": [
+           [
+            0,
+            "rgb(247,251,255)"
+           ],
+           [
+            0.125,
+            "rgb(222,235,247)"
+           ],
+           [
+            0.25,
+            "rgb(198,219,239)"
+           ],
+           [
+            0.375,
+            "rgb(158,202,225)"
+           ],
+           [
+            0.5,
+            "rgb(107,174,214)"
+           ],
+           [
+            0.625,
+            "rgb(66,146,198)"
+           ],
+           [
+            0.75,
+            "rgb(33,113,181)"
+           ],
+           [
+            0.875,
+            "rgb(8,81,156)"
+           ],
+           [
+            1,
+            "rgb(8,48,107)"
+           ]
+          ],
+          "reversescale": false,
+          "showscale": true
+         },
+         "type": "parcoords"
+        }
+       ],
+       "layout": {
+        "template": {
+         "data": {
+          "bar": [
+           {
+            "error_x": {
+             "color": "#2a3f5f"
+            },
+            "error_y": {
+             "color": "#2a3f5f"
+            },
+            "marker": {
+             "line": {
+              "color": "#E5ECF6",
+              "width": 0.5
+             },
+             "pattern": {
+              "fillmode": "overlay",
+              "size": 10,
+              "solidity": 0.2
+             }
+            },
+            "type": "bar"
+           }
+          ],
+          "barpolar": [
+           {
+            "marker": {
+             "line": {
+              "color": "#E5ECF6",
+              "width": 0.5
+             },
+             "pattern": {
+              "fillmode": "overlay",
+              "size": 10,
+              "solidity": 0.2
+             }
+            },
+            "type": "barpolar"
+           }
+          ],
+          "carpet": [
+           {
+            "aaxis": {
+             "endlinecolor": "#2a3f5f",
+             "gridcolor": "white",
+             "linecolor": "white",
+             "minorgridcolor": "white",
+             "startlinecolor": "#2a3f5f"
+            },
+            "baxis": {
+             "endlinecolor": "#2a3f5f",
+             "gridcolor": "white",
+             "linecolor": "white",
+             "minorgridcolor": "white",
+             "startlinecolor": "#2a3f5f"
+            },
+            "type": "carpet"
+           }
+          ],
+          "choropleth": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "type": "choropleth"
+           }
+          ],
+          "contour": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "colorscale": [
+             [
+              0,
+              "#0d0887"
+             ],
+             [
+              0.1111111111111111,
+              "#46039f"
+             ],
+             [
+              0.2222222222222222,
+              "#7201a8"
+             ],
+             [
+              0.3333333333333333,
+              "#9c179e"
+             ],
+             [
+              0.4444444444444444,
+              "#bd3786"
+             ],
+             [
+              0.5555555555555556,
+              "#d8576b"
+             ],
+             [
+              0.6666666666666666,
+              "#ed7953"
+             ],
+             [
+              0.7777777777777778,
+              "#fb9f3a"
+             ],
+             [
+              0.8888888888888888,
+              "#fdca26"
+             ],
+             [
+              1,
+              "#f0f921"
+             ]
+            ],
+            "type": "contour"
+           }
+          ],
+          "contourcarpet": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "type": "contourcarpet"
+           }
+          ],
+          "heatmap": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "colorscale": [
+             [
+              0,
+              "#0d0887"
+             ],
+             [
+              0.1111111111111111,
+              "#46039f"
+             ],
+             [
+              0.2222222222222222,
+              "#7201a8"
+             ],
+             [
+              0.3333333333333333,
+              "#9c179e"
+             ],
+             [
+              0.4444444444444444,
+              "#bd3786"
+             ],
+             [
+              0.5555555555555556,
+              "#d8576b"
+             ],
+             [
+              0.6666666666666666,
+              "#ed7953"
+             ],
+             [
+              0.7777777777777778,
+              "#fb9f3a"
+             ],
+             [
+              0.8888888888888888,
+              "#fdca26"
+             ],
+             [
+              1,
+              "#f0f921"
+             ]
+            ],
+            "type": "heatmap"
+           }
+          ],
+          "histogram": [
+           {
+            "marker": {
+             "pattern": {
+              "fillmode": "overlay",
+              "size": 10,
+              "solidity": 0.2
+             }
+            },
+            "type": "histogram"
+           }
+          ],
+          "histogram2d": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "colorscale": [
+             [
+              0,
+              "#0d0887"
+             ],
+             [
+              0.1111111111111111,
+              "#46039f"
+             ],
+             [
+              0.2222222222222222,
+              "#7201a8"
+             ],
+             [
+              0.3333333333333333,
+              "#9c179e"
+             ],
+             [
+              0.4444444444444444,
+              "#bd3786"
+             ],
+             [
+              0.5555555555555556,
+              "#d8576b"
+             ],
+             [
+              0.6666666666666666,
+              "#ed7953"
+             ],
+             [
+              0.7777777777777778,
+              "#fb9f3a"
+             ],
+             [
+              0.8888888888888888,
+              "#fdca26"
+             ],
+             [
+              1,
+              "#f0f921"
+             ]
+            ],
+            "type": "histogram2d"
+           }
+          ],
+          "histogram2dcontour": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "colorscale": [
+             [
+              0,
+              "#0d0887"
+             ],
+             [
+              0.1111111111111111,
+              "#46039f"
+             ],
+             [
+              0.2222222222222222,
+              "#7201a8"
+             ],
+             [
+              0.3333333333333333,
+              "#9c179e"
+             ],
+             [
+              0.4444444444444444,
+              "#bd3786"
+             ],
+             [
+              0.5555555555555556,
+              "#d8576b"
+             ],
+             [
+              0.6666666666666666,
+              "#ed7953"
+             ],
+             [
+              0.7777777777777778,
+              "#fb9f3a"
+             ],
+             [
+              0.8888888888888888,
+              "#fdca26"
+             ],
+             [
+              1,
+              "#f0f921"
+             ]
+            ],
+            "type": "histogram2dcontour"
+           }
+          ],
+          "mesh3d": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "type": "mesh3d"
+           }
+          ],
+          "parcoords": [
+           {
+            "line": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "parcoords"
+           }
+          ],
+          "pie": [
+           {
+            "automargin": true,
+            "type": "pie"
+           }
+          ],
+          "scatter": [
+           {
+            "fillpattern": {
+             "fillmode": "overlay",
+             "size": 10,
+             "solidity": 0.2
+            },
+            "type": "scatter"
+           }
+          ],
+          "scatter3d": [
+           {
+            "line": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scatter3d"
+           }
+          ],
+          "scattercarpet": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scattercarpet"
+           }
+          ],
+          "scattergeo": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scattergeo"
+           }
+          ],
+          "scattergl": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scattergl"
+           }
+          ],
+          "scattermap": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scattermap"
+           }
+          ],
+          "scattermapbox": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scattermapbox"
+           }
+          ],
+          "scatterpolar": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scatterpolar"
+           }
+          ],
+          "scatterpolargl": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scatterpolargl"
+           }
+          ],
+          "scatterternary": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scatterternary"
+           }
+          ],
+          "surface": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "colorscale": [
+             [
+              0,
+              "#0d0887"
+             ],
+             [
+              0.1111111111111111,
+              "#46039f"
+             ],
+             [
+              0.2222222222222222,
+              "#7201a8"
+             ],
+             [
+              0.3333333333333333,
+              "#9c179e"
+             ],
+             [
+              0.4444444444444444,
+              "#bd3786"
+             ],
+             [
+              0.5555555555555556,
+              "#d8576b"
+             ],
+             [
+              0.6666666666666666,
+              "#ed7953"
+             ],
+             [
+              0.7777777777777778,
+              "#fb9f3a"
+             ],
+             [
+              0.8888888888888888,
+              "#fdca26"
+             ],
+             [
+              1,
+              "#f0f921"
+             ]
+            ],
+            "type": "surface"
+           }
+          ],
+          "table": [
+           {
+            "cells": {
+             "fill": {
+              "color": "#EBF0F8"
+             },
+             "line": {
+              "color": "white"
+             }
+            },
+            "header": {
+             "fill": {
+              "color": "#C8D4E3"
+             },
+             "line": {
+              "color": "white"
+             }
+            },
+            "type": "table"
+           }
+          ]
+         },
+         "layout": {
+          "annotationdefaults": {
+           "arrowcolor": "#2a3f5f",
+           "arrowhead": 0,
+           "arrowwidth": 1
+          },
+          "autotypenumbers": "strict",
+          "coloraxis": {
+           "colorbar": {
+            "outlinewidth": 0,
+            "ticks": ""
+           }
+          },
+          "colorscale": {
+           "diverging": [
+            [
+             0,
+             "#8e0152"
+            ],
+            [
+             0.1,
+             "#c51b7d"
+            ],
+            [
+             0.2,
+             "#de77ae"
+            ],
+            [
+             0.3,
+             "#f1b6da"
+            ],
+            [
+             0.4,
+             "#fde0ef"
+            ],
+            [
+             0.5,
+             "#f7f7f7"
+            ],
+            [
+             0.6,
+             "#e6f5d0"
+            ],
+            [
+             0.7,
+             "#b8e186"
+            ],
+            [
+             0.8,
+             "#7fbc41"
+            ],
+            [
+             0.9,
+             "#4d9221"
+            ],
+            [
+             1,
+             "#276419"
+            ]
+           ],
+           "sequential": [
+            [
+             0,
+             "#0d0887"
+            ],
+            [
+             0.1111111111111111,
+             "#46039f"
+            ],
+            [
+             0.2222222222222222,
+             "#7201a8"
+            ],
+            [
+             0.3333333333333333,
+             "#9c179e"
+            ],
+            [
+             0.4444444444444444,
+             "#bd3786"
+            ],
+            [
+             0.5555555555555556,
+             "#d8576b"
+            ],
+            [
+             0.6666666666666666,
+             "#ed7953"
+            ],
+            [
+             0.7777777777777778,
+             "#fb9f3a"
+            ],
+            [
+             0.8888888888888888,
+             "#fdca26"
+            ],
+            [
+             1,
+             "#f0f921"
+            ]
+           ],
+           "sequentialminus": [
+            [
+             0,
+             "#0d0887"
+            ],
+            [
+             0.1111111111111111,
+             "#46039f"
+            ],
+            [
+             0.2222222222222222,
+             "#7201a8"
+            ],
+            [
+             0.3333333333333333,
+             "#9c179e"
+            ],
+            [
+             0.4444444444444444,
+             "#bd3786"
+            ],
+            [
+             0.5555555555555556,
+             "#d8576b"
+            ],
+            [
+             0.6666666666666666,
+             "#ed7953"
+            ],
+            [
+             0.7777777777777778,
+             "#fb9f3a"
+            ],
+            [
+             0.8888888888888888,
+             "#fdca26"
+            ],
+            [
+             1,
+             "#f0f921"
+            ]
+           ]
+          },
+          "colorway": [
+           "#636efa",
+           "#EF553B",
+           "#00cc96",
+           "#ab63fa",
+           "#FFA15A",
+           "#19d3f3",
+           "#FF6692",
+           "#B6E880",
+           "#FF97FF",
+           "#FECB52"
+          ],
+          "font": {
+           "color": "#2a3f5f"
+          },
+          "geo": {
+           "bgcolor": "white",
+           "lakecolor": "white",
+           "landcolor": "#E5ECF6",
+           "showlakes": true,
+           "showland": true,
+           "subunitcolor": "white"
+          },
+          "hoverlabel": {
+           "align": "left"
+          },
+          "hovermode": "closest",
+          "mapbox": {
+           "style": "light"
+          },
+          "paper_bgcolor": "white",
+          "plot_bgcolor": "#E5ECF6",
+          "polar": {
+           "angularaxis": {
+            "gridcolor": "white",
+            "linecolor": "white",
+            "ticks": ""
+           },
+           "bgcolor": "#E5ECF6",
+           "radialaxis": {
+            "gridcolor": "white",
+            "linecolor": "white",
+            "ticks": ""
+           }
+          },
+          "scene": {
+           "xaxis": {
+            "backgroundcolor": "#E5ECF6",
+            "gridcolor": "white",
+            "gridwidth": 2,
+            "linecolor": "white",
+            "showbackground": true,
+            "ticks": "",
+            "zerolinecolor": "white"
+           },
+           "yaxis": {
+            "backgroundcolor": "#E5ECF6",
+            "gridcolor": "white",
+            "gridwidth": 2,
+            "linecolor": "white",
+            "showbackground": true,
+            "ticks": "",
+            "zerolinecolor": "white"
+           },
+           "zaxis": {
+            "backgroundcolor": "#E5ECF6",
+            "gridcolor": "white",
+            "gridwidth": 2,
+            "linecolor": "white",
+            "showbackground": true,
+            "ticks": "",
+            "zerolinecolor": "white"
+           }
+          },
+          "shapedefaults": {
+           "line": {
+            "color": "#2a3f5f"
+           }
+          },
+          "ternary": {
+           "aaxis": {
+            "gridcolor": "white",
+            "linecolor": "white",
+            "ticks": ""
+           },
+           "baxis": {
+            "gridcolor": "white",
+            "linecolor": "white",
+            "ticks": ""
+           },
+           "bgcolor": "#E5ECF6",
+           "caxis": {
+            "gridcolor": "white",
+            "linecolor": "white",
+            "ticks": ""
+           }
+          },
+          "title": {
+           "x": 0.05
+          },
+          "xaxis": {
+           "automargin": true,
+           "gridcolor": "white",
+           "linecolor": "white",
+           "ticks": "",
+           "title": {
+            "standoff": 15
+           },
+           "zerolinecolor": "white",
+           "zerolinewidth": 2
+          },
+          "yaxis": {
+           "automargin": true,
+           "gridcolor": "white",
+           "linecolor": "white",
+           "ticks": "",
+           "title": {
+            "standoff": 15
+           },
+           "zerolinecolor": "white",
+           "zerolinewidth": 2
+          }
+         }
+        },
+        "title": {
+         "text": "Parallel Coordinate Plot"
+        }
+       }
+      }
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.plotly.v1+json": {
+       "config": {
+        "plotlyServerURL": "https://plot.ly"
+       },
+       "data": [
+        {
+         "type": "scatter",
+         "xaxis": "x",
+         "yaxis": "y"
+        },
+        {
+         "colorbar": {
+          "title": {
+           "text": "Objective Value"
+          }
+         },
+         "colorscale": [
+          [
+           0,
+           "rgb(247,251,255)"
+          ],
+          [
+           0.125,
+           "rgb(222,235,247)"
+          ],
+          [
+           0.25,
+           "rgb(198,219,239)"
+          ],
+          [
+           0.375,
+           "rgb(158,202,225)"
+          ],
+          [
+           0.5,
+           "rgb(107,174,214)"
+          ],
+          [
+           0.625,
+           "rgb(66,146,198)"
+          ],
+          [
+           0.75,
+           "rgb(33,113,181)"
+          ],
+          [
+           0.875,
+           "rgb(8,81,156)"
+          ],
+          [
+           1,
+           "rgb(8,48,107)"
+          ]
+         ],
+         "connectgaps": true,
+         "contours": {
+          "coloring": "heatmap"
+         },
+         "hoverinfo": "none",
+         "line": {
+          "smoothing": 1.3
+         },
+         "reversescale": false,
+         "showscale": true,
+         "type": "contour",
+         "x": [
+          9.927815477793583e-7,
+          0.0000012177346043359053,
+          0.000002973185825213819,
+          0.000003763988365260261,
+          0.00002756288216246014,
+          0.0000723655165533393,
+          0.0000887627231436304
+         ],
+         "xaxis": "x4",
+         "y": [
+          10.399999999999999,
+          16,
+          64,
+          128,
+          133.6
+         ],
+         "yaxis": "y4",
+         "z": {
+          "bdata": "AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/ppvEILBy6D8AAAAAAAD4fwAAAAAAAPh/PQrXo3A94j9kO99PjZfiP7TIdr6fGuM/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8pXI/C9SjkPwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fw==",
+          "dtype": "f8",
+          "shape": "5, 7"
+         }
+        },
+        {
+         "marker": {
+          "color": "black",
+          "line": {
+           "color": "Gray",
+           "width": 2
+          }
+         },
+         "mode": "markers",
+         "name": "Feasible Trial",
+         "showlegend": false,
+         "type": "scatter",
+         "x": [
+          0.0000723655165533393,
+          0.00002756288216246014,
+          0.0000012177346043359053,
+          0.000002973185825213819,
+          0.000003763988365260261
+         ],
+         "xaxis": "x4",
+         "y": [
+          16,
+          128,
+          64,
+          64,
+          64
+         ],
+         "yaxis": "y4"
+        },
+        {
+         "colorbar": {
+          "title": {
+           "text": "Objective Value"
+          }
+         },
+         "colorscale": [
+          [
+           0,
+           "rgb(247,251,255)"
+          ],
+          [
+           0.125,
+           "rgb(222,235,247)"
+          ],
+          [
+           0.25,
+           "rgb(198,219,239)"
+          ],
+          [
+           0.375,
+           "rgb(158,202,225)"
+          ],
+          [
+           0.5,
+           "rgb(107,174,214)"
+          ],
+          [
+           0.625,
+           "rgb(66,146,198)"
+          ],
+          [
+           0.75,
+           "rgb(33,113,181)"
+          ],
+          [
+           0.875,
+           "rgb(8,81,156)"
+          ],
+          [
+           1,
+           "rgb(8,48,107)"
+          ]
+         ],
+         "connectgaps": true,
+         "contours": {
+          "coloring": "heatmap"
+         },
+         "hoverinfo": "none",
+         "line": {
+          "smoothing": 1.3
+         },
+         "reversescale": false,
+         "showscale": false,
+         "type": "contour",
+         "x": [
+          9.927815477793583e-7,
+          0.0000012177346043359053,
+          0.000002973185825213819,
+          0.000003763988365260261,
+          0.00002756288216246014,
+          0.0000723655165533393,
+          0.0000887627231436304
+         ],
+         "xaxis": "x7",
+         "y": [
+          0.00023616709705117506,
+          0.013798094328723032,
+          0.02906341093983704,
+          0.09102292466460353,
+          0.1502192542358606,
+          0.28503663896216014,
+          0.298598566193832
+         ],
+         "yaxis": "y7",
+         "z": {
+          "bdata": "AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/ppvEILBy6D8AAAAAAAD4fwAAAAAAAPh/PQrXo3A94j8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4f2Q730+Nl+I/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H+0yHa+nxrjPwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/KVyPwvUo5D8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8=",
+          "dtype": "f8",
+          "shape": "7, 7"
+         }
+        },
+        {
+         "marker": {
+          "color": "black",
+          "line": {
+           "color": "Gray",
+           "width": 2
+          }
+         },
+         "mode": "markers",
+         "name": "Feasible Trial",
+         "showlegend": false,
+         "type": "scatter",
+         "x": [
+          0.0000723655165533393,
+          0.00002756288216246014,
+          0.0000012177346043359053,
+          0.000002973185825213819,
+          0.000003763988365260261
+         ],
+         "xaxis": "x7",
+         "y": [
+          0.013798094328723032,
+          0.28503663896216014,
+          0.02906341093983704,
+          0.09102292466460353,
+          0.1502192542358606
+         ],
+         "yaxis": "y7"
+        },
+        {
+         "colorbar": {
+          "title": {
+           "text": "Objective Value"
+          }
+         },
+         "colorscale": [
+          [
+           0,
+           "rgb(247,251,255)"
+          ],
+          [
+           0.125,
+           "rgb(222,235,247)"
+          ],
+          [
+           0.25,
+           "rgb(198,219,239)"
+          ],
+          [
+           0.375,
+           "rgb(158,202,225)"
+          ],
+          [
+           0.5,
+           "rgb(107,174,214)"
+          ],
+          [
+           0.625,
+           "rgb(66,146,198)"
+          ],
+          [
+           0.75,
+           "rgb(33,113,181)"
+          ],
+          [
+           0.875,
+           "rgb(8,81,156)"
+          ],
+          [
+           1,
+           "rgb(8,48,107)"
+          ]
+         ],
+         "connectgaps": true,
+         "contours": {
+          "coloring": "heatmap"
+         },
+         "hoverinfo": "none",
+         "line": {
+          "smoothing": 1.3
+         },
+         "reversescale": false,
+         "showscale": false,
+         "type": "contour",
+         "x": [
+          10.399999999999999,
+          16,
+          64,
+          128,
+          133.6
+         ],
+         "xaxis": "x2",
+         "y": [
+          9.927815477793583e-7,
+          0.0000012177346043359053,
+          0.000002973185825213819,
+          0.000003763988365260261,
+          0.00002756288216246014,
+          0.0000723655165533393,
+          0.0000887627231436304
+         ],
+         "yaxis": "y2",
+         "z": {
+          "bdata": "AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H89CtejcD3iPwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/ZDvfT42X4j8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4f7TIdr6fGuM/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fylcj8L1KOQ/AAAAAAAA+H8AAAAAAAD4f6abxCCwcug/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fw==",
+          "dtype": "f8",
+          "shape": "7, 5"
+         }
+        },
+        {
+         "marker": {
+          "color": "black",
+          "line": {
+           "color": "Gray",
+           "width": 2
+          }
+         },
+         "mode": "markers",
+         "name": "Feasible Trial",
+         "showlegend": false,
+         "type": "scatter",
+         "x": [
+          16,
+          128,
+          64,
+          64,
+          64
+         ],
+         "xaxis": "x2",
+         "y": [
+          0.0000723655165533393,
+          0.00002756288216246014,
+          0.0000012177346043359053,
+          0.000002973185825213819,
+          0.000003763988365260261
+         ],
+         "yaxis": "y2"
+        },
+        {
+         "type": "scatter",
+         "xaxis": "x5",
+         "yaxis": "y5"
+        },
+        {
+         "colorbar": {
+          "title": {
+           "text": "Objective Value"
+          }
+         },
+         "colorscale": [
+          [
+           0,
+           "rgb(247,251,255)"
+          ],
+          [
+           0.125,
+           "rgb(222,235,247)"
+          ],
+          [
+           0.25,
+           "rgb(198,219,239)"
+          ],
+          [
+           0.375,
+           "rgb(158,202,225)"
+          ],
+          [
+           0.5,
+           "rgb(107,174,214)"
+          ],
+          [
+           0.625,
+           "rgb(66,146,198)"
+          ],
+          [
+           0.75,
+           "rgb(33,113,181)"
+          ],
+          [
+           0.875,
+           "rgb(8,81,156)"
+          ],
+          [
+           1,
+           "rgb(8,48,107)"
+          ]
+         ],
+         "connectgaps": true,
+         "contours": {
+          "coloring": "heatmap"
+         },
+         "hoverinfo": "none",
+         "line": {
+          "smoothing": 1.3
+         },
+         "reversescale": false,
+         "showscale": false,
+         "type": "contour",
+         "x": [
+          10.399999999999999,
+          16,
+          64,
+          128,
+          133.6
+         ],
+         "xaxis": "x8",
+         "y": [
+          0.00023616709705117506,
+          0.013798094328723032,
+          0.02906341093983704,
+          0.09102292466460353,
+          0.1502192542358606,
+          0.28503663896216014,
+          0.298598566193832
+         ],
+         "yaxis": "y8",
+         "z": {
+          "bdata": "AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/ppvEILBy6D8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/PQrXo3A94j8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4f2Q730+Nl+I/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H+0yHa+nxrjPwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8pXI/C9SjkPwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fw==",
+          "dtype": "f8",
+          "shape": "7, 5"
+         }
+        },
+        {
+         "marker": {
+          "color": "black",
+          "line": {
+           "color": "Gray",
+           "width": 2
+          }
+         },
+         "mode": "markers",
+         "name": "Feasible Trial",
+         "showlegend": false,
+         "type": "scatter",
+         "x": [
+          16,
+          128,
+          64,
+          64,
+          64
+         ],
+         "xaxis": "x8",
+         "y": [
+          0.013798094328723032,
+          0.28503663896216014,
+          0.02906341093983704,
+          0.09102292466460353,
+          0.1502192542358606
+         ],
+         "yaxis": "y8"
+        },
+        {
+         "colorbar": {
+          "title": {
+           "text": "Objective Value"
+          }
+         },
+         "colorscale": [
+          [
+           0,
+           "rgb(247,251,255)"
+          ],
+          [
+           0.125,
+           "rgb(222,235,247)"
+          ],
+          [
+           0.25,
+           "rgb(198,219,239)"
+          ],
+          [
+           0.375,
+           "rgb(158,202,225)"
+          ],
+          [
+           0.5,
+           "rgb(107,174,214)"
+          ],
+          [
+           0.625,
+           "rgb(66,146,198)"
+          ],
+          [
+           0.75,
+           "rgb(33,113,181)"
+          ],
+          [
+           0.875,
+           "rgb(8,81,156)"
+          ],
+          [
+           1,
+           "rgb(8,48,107)"
+          ]
+         ],
+         "connectgaps": true,
+         "contours": {
+          "coloring": "heatmap"
+         },
+         "hoverinfo": "none",
+         "line": {
+          "smoothing": 1.3
+         },
+         "reversescale": false,
+         "showscale": false,
+         "type": "contour",
+         "x": [
+          0.00023616709705117506,
+          0.013798094328723032,
+          0.02906341093983704,
+          0.09102292466460353,
+          0.1502192542358606,
+          0.28503663896216014,
+          0.298598566193832
+         ],
+         "xaxis": "x3",
+         "y": [
+          9.927815477793583e-7,
+          0.0000012177346043359053,
+          0.000002973185825213819,
+          0.000003763988365260261,
+          0.00002756288216246014,
+          0.0000723655165533393,
+          0.0000887627231436304
+         ],
+         "yaxis": "y3",
+         "z": {
+          "bdata": "AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/PQrXo3A94j8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4f2Q730+Nl+I/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H+0yHa+nxrjPwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/KVyPwvUo5D8AAAAAAAD4fwAAAAAAAPh/ppvEILBy6D8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8=",
+          "dtype": "f8",
+          "shape": "7, 7"
+         }
+        },
+        {
+         "marker": {
+          "color": "black",
+          "line": {
+           "color": "Gray",
+           "width": 2
+          }
+         },
+         "mode": "markers",
+         "name": "Feasible Trial",
+         "showlegend": false,
+         "type": "scatter",
+         "x": [
+          0.013798094328723032,
+          0.28503663896216014,
+          0.02906341093983704,
+          0.09102292466460353,
+          0.1502192542358606
+         ],
+         "xaxis": "x3",
+         "y": [
+          0.0000723655165533393,
+          0.00002756288216246014,
+          0.0000012177346043359053,
+          0.000002973185825213819,
+          0.000003763988365260261
+         ],
+         "yaxis": "y3"
+        },
+        {
+         "colorbar": {
+          "title": {
+           "text": "Objective Value"
+          }
+         },
+         "colorscale": [
+          [
+           0,
+           "rgb(247,251,255)"
+          ],
+          [
+           0.125,
+           "rgb(222,235,247)"
+          ],
+          [
+           0.25,
+           "rgb(198,219,239)"
+          ],
+          [
+           0.375,
+           "rgb(158,202,225)"
+          ],
+          [
+           0.5,
+           "rgb(107,174,214)"
+          ],
+          [
+           0.625,
+           "rgb(66,146,198)"
+          ],
+          [
+           0.75,
+           "rgb(33,113,181)"
+          ],
+          [
+           0.875,
+           "rgb(8,81,156)"
+          ],
+          [
+           1,
+           "rgb(8,48,107)"
+          ]
+         ],
+         "connectgaps": true,
+         "contours": {
+          "coloring": "heatmap"
+         },
+         "hoverinfo": "none",
+         "line": {
+          "smoothing": 1.3
+         },
+         "reversescale": false,
+         "showscale": false,
+         "type": "contour",
+         "x": [
+          0.00023616709705117506,
+          0.013798094328723032,
+          0.02906341093983704,
+          0.09102292466460353,
+          0.1502192542358606,
+          0.28503663896216014,
+          0.298598566193832
+         ],
+         "xaxis": "x6",
+         "y": [
+          10.399999999999999,
+          16,
+          64,
+          128,
+          133.6
+         ],
+         "yaxis": "y6",
+         "z": {
+          "bdata": "AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4f6abxCCwcug/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H89CtejcD3iP2Q730+Nl+I/tMh2vp8a4z8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fylcj8L1KOQ/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fwAAAAAAAPh/AAAAAAAA+H8AAAAAAAD4fw==",
+          "dtype": "f8",
+          "shape": "5, 7"
+         }
+        },
+        {
+         "marker": {
+          "color": "black",
+          "line": {
+           "color": "Gray",
+           "width": 2
+          }
+         },
+         "mode": "markers",
+         "name": "Feasible Trial",
+         "showlegend": false,
+         "type": "scatter",
+         "x": [
+          0.013798094328723032,
+          0.28503663896216014,
+          0.02906341093983704,
+          0.09102292466460353,
+          0.1502192542358606
+         ],
+         "xaxis": "x6",
+         "y": [
+          16,
+          128,
+          64,
+          64,
+          64
+         ],
+         "yaxis": "y6"
+        },
+        {
+         "type": "scatter",
+         "xaxis": "x9",
+         "yaxis": "y9"
+        }
+       ],
+       "layout": {
+        "template": {
+         "data": {
+          "bar": [
+           {
+            "error_x": {
+             "color": "#2a3f5f"
+            },
+            "error_y": {
+             "color": "#2a3f5f"
+            },
+            "marker": {
+             "line": {
+              "color": "#E5ECF6",
+              "width": 0.5
+             },
+             "pattern": {
+              "fillmode": "overlay",
+              "size": 10,
+              "solidity": 0.2
+             }
+            },
+            "type": "bar"
+           }
+          ],
+          "barpolar": [
+           {
+            "marker": {
+             "line": {
+              "color": "#E5ECF6",
+              "width": 0.5
+             },
+             "pattern": {
+              "fillmode": "overlay",
+              "size": 10,
+              "solidity": 0.2
+             }
+            },
+            "type": "barpolar"
+           }
+          ],
+          "carpet": [
+           {
+            "aaxis": {
+             "endlinecolor": "#2a3f5f",
+             "gridcolor": "white",
+             "linecolor": "white",
+             "minorgridcolor": "white",
+             "startlinecolor": "#2a3f5f"
+            },
+            "baxis": {
+             "endlinecolor": "#2a3f5f",
+             "gridcolor": "white",
+             "linecolor": "white",
+             "minorgridcolor": "white",
+             "startlinecolor": "#2a3f5f"
+            },
+            "type": "carpet"
+           }
+          ],
+          "choropleth": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "type": "choropleth"
+           }
+          ],
+          "contour": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "colorscale": [
+             [
+              0,
+              "#0d0887"
+             ],
+             [
+              0.1111111111111111,
+              "#46039f"
+             ],
+             [
+              0.2222222222222222,
+              "#7201a8"
+             ],
+             [
+              0.3333333333333333,
+              "#9c179e"
+             ],
+             [
+              0.4444444444444444,
+              "#bd3786"
+             ],
+             [
+              0.5555555555555556,
+              "#d8576b"
+             ],
+             [
+              0.6666666666666666,
+              "#ed7953"
+             ],
+             [
+              0.7777777777777778,
+              "#fb9f3a"
+             ],
+             [
+              0.8888888888888888,
+              "#fdca26"
+             ],
+             [
+              1,
+              "#f0f921"
+             ]
+            ],
+            "type": "contour"
+           }
+          ],
+          "contourcarpet": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "type": "contourcarpet"
+           }
+          ],
+          "heatmap": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "colorscale": [
+             [
+              0,
+              "#0d0887"
+             ],
+             [
+              0.1111111111111111,
+              "#46039f"
+             ],
+             [
+              0.2222222222222222,
+              "#7201a8"
+             ],
+             [
+              0.3333333333333333,
+              "#9c179e"
+             ],
+             [
+              0.4444444444444444,
+              "#bd3786"
+             ],
+             [
+              0.5555555555555556,
+              "#d8576b"
+             ],
+             [
+              0.6666666666666666,
+              "#ed7953"
+             ],
+             [
+              0.7777777777777778,
+              "#fb9f3a"
+             ],
+             [
+              0.8888888888888888,
+              "#fdca26"
+             ],
+             [
+              1,
+              "#f0f921"
+             ]
+            ],
+            "type": "heatmap"
+           }
+          ],
+          "histogram": [
+           {
+            "marker": {
+             "pattern": {
+              "fillmode": "overlay",
+              "size": 10,
+              "solidity": 0.2
+             }
+            },
+            "type": "histogram"
+           }
+          ],
+          "histogram2d": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "colorscale": [
+             [
+              0,
+              "#0d0887"
+             ],
+             [
+              0.1111111111111111,
+              "#46039f"
+             ],
+             [
+              0.2222222222222222,
+              "#7201a8"
+             ],
+             [
+              0.3333333333333333,
+              "#9c179e"
+             ],
+             [
+              0.4444444444444444,
+              "#bd3786"
+             ],
+             [
+              0.5555555555555556,
+              "#d8576b"
+             ],
+             [
+              0.6666666666666666,
+              "#ed7953"
+             ],
+             [
+              0.7777777777777778,
+              "#fb9f3a"
+             ],
+             [
+              0.8888888888888888,
+              "#fdca26"
+             ],
+             [
+              1,
+              "#f0f921"
+             ]
+            ],
+            "type": "histogram2d"
+           }
+          ],
+          "histogram2dcontour": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "colorscale": [
+             [
+              0,
+              "#0d0887"
+             ],
+             [
+              0.1111111111111111,
+              "#46039f"
+             ],
+             [
+              0.2222222222222222,
+              "#7201a8"
+             ],
+             [
+              0.3333333333333333,
+              "#9c179e"
+             ],
+             [
+              0.4444444444444444,
+              "#bd3786"
+             ],
+             [
+              0.5555555555555556,
+              "#d8576b"
+             ],
+             [
+              0.6666666666666666,
+              "#ed7953"
+             ],
+             [
+              0.7777777777777778,
+              "#fb9f3a"
+             ],
+             [
+              0.8888888888888888,
+              "#fdca26"
+             ],
+             [
+              1,
+              "#f0f921"
+             ]
+            ],
+            "type": "histogram2dcontour"
+           }
+          ],
+          "mesh3d": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "type": "mesh3d"
+           }
+          ],
+          "parcoords": [
+           {
+            "line": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "parcoords"
+           }
+          ],
+          "pie": [
+           {
+            "automargin": true,
+            "type": "pie"
+           }
+          ],
+          "scatter": [
+           {
+            "fillpattern": {
+             "fillmode": "overlay",
+             "size": 10,
+             "solidity": 0.2
+            },
+            "type": "scatter"
+           }
+          ],
+          "scatter3d": [
+           {
+            "line": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scatter3d"
+           }
+          ],
+          "scattercarpet": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scattercarpet"
+           }
+          ],
+          "scattergeo": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scattergeo"
+           }
+          ],
+          "scattergl": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scattergl"
+           }
+          ],
+          "scattermap": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scattermap"
+           }
+          ],
+          "scattermapbox": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scattermapbox"
+           }
+          ],
+          "scatterpolar": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scatterpolar"
+           }
+          ],
+          "scatterpolargl": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scatterpolargl"
+           }
+          ],
+          "scatterternary": [
+           {
+            "marker": {
+             "colorbar": {
+              "outlinewidth": 0,
+              "ticks": ""
+             }
+            },
+            "type": "scatterternary"
+           }
+          ],
+          "surface": [
+           {
+            "colorbar": {
+             "outlinewidth": 0,
+             "ticks": ""
+            },
+            "colorscale": [
+             [
+              0,
+              "#0d0887"
+             ],
+             [
+              0.1111111111111111,
+              "#46039f"
+             ],
+             [
+              0.2222222222222222,
+              "#7201a8"
+             ],
+             [
+              0.3333333333333333,
+              "#9c179e"
+             ],
+             [
+              0.4444444444444444,
+              "#bd3786"
+             ],
+             [
+              0.5555555555555556,
+              "#d8576b"
+             ],
+             [
+              0.6666666666666666,
+              "#ed7953"
+             ],
+             [
+              0.7777777777777778,
+              "#fb9f3a"
+             ],
+             [
+              0.8888888888888888,
+              "#fdca26"
+             ],
+             [
+              1,
+              "#f0f921"
+             ]
+            ],
+            "type": "surface"
+           }
+          ],
+          "table": [
+           {
+            "cells": {
+             "fill": {
+              "color": "#EBF0F8"
+             },
+             "line": {
+              "color": "white"
+             }
+            },
+            "header": {
+             "fill": {
+              "color": "#C8D4E3"
+             },
+             "line": {
+              "color": "white"
+             }
+            },
+            "type": "table"
+           }
+          ]
+         },
+         "layout": {
+          "annotationdefaults": {
+           "arrowcolor": "#2a3f5f",
+           "arrowhead": 0,
+           "arrowwidth": 1
+          },
+          "autotypenumbers": "strict",
+          "coloraxis": {
+           "colorbar": {
+            "outlinewidth": 0,
+            "ticks": ""
+           }
+          },
+          "colorscale": {
+           "diverging": [
+            [
+             0,
+             "#8e0152"
+            ],
+            [
+             0.1,
+             "#c51b7d"
+            ],
+            [
+             0.2,
+             "#de77ae"
+            ],
+            [
+             0.3,
+             "#f1b6da"
+            ],
+            [
+             0.4,
+             "#fde0ef"
+            ],
+            [
+             0.5,
+             "#f7f7f7"
+            ],
+            [
+             0.6,
+             "#e6f5d0"
+            ],
+            [
+             0.7,
+             "#b8e186"
+            ],
+            [
+             0.8,
+             "#7fbc41"
+            ],
+            [
+             0.9,
+             "#4d9221"
+            ],
+            [
+             1,
+             "#276419"
+            ]
+           ],
+           "sequential": [
+            [
+             0,
+             "#0d0887"
+            ],
+            [
+             0.1111111111111111,
+             "#46039f"
+            ],
+            [
+             0.2222222222222222,
+             "#7201a8"
+            ],
+            [
+             0.3333333333333333,
+             "#9c179e"
+            ],
+            [
+             0.4444444444444444,
+             "#bd3786"
+            ],
+            [
+             0.5555555555555556,
+             "#d8576b"
+            ],
+            [
+             0.6666666666666666,
+             "#ed7953"
+            ],
+            [
+             0.7777777777777778,
+             "#fb9f3a"
+            ],
+            [
+             0.8888888888888888,
+             "#fdca26"
+            ],
+            [
+             1,
+             "#f0f921"
+            ]
+           ],
+           "sequentialminus": [
+            [
+             0,
+             "#0d0887"
+            ],
+            [
+             0.1111111111111111,
+             "#46039f"
+            ],
+            [
+             0.2222222222222222,
+             "#7201a8"
+            ],
+            [
+             0.3333333333333333,
+             "#9c179e"
+            ],
+            [
+             0.4444444444444444,
+             "#bd3786"
+            ],
+            [
+             0.5555555555555556,
+             "#d8576b"
+            ],
+            [
+             0.6666666666666666,
+             "#ed7953"
+            ],
+            [
+             0.7777777777777778,
+             "#fb9f3a"
+            ],
+            [
+             0.8888888888888888,
+             "#fdca26"
+            ],
+            [
+             1,
+             "#f0f921"
+            ]
+           ]
+          },
+          "colorway": [
+           "#636efa",
+           "#EF553B",
+           "#00cc96",
+           "#ab63fa",
+           "#FFA15A",
+           "#19d3f3",
+           "#FF6692",
+           "#B6E880",
+           "#FF97FF",
+           "#FECB52"
+          ],
+          "font": {
+           "color": "#2a3f5f"
+          },
+          "geo": {
+           "bgcolor": "white",
+           "lakecolor": "white",
+           "landcolor": "#E5ECF6",
+           "showlakes": true,
+           "showland": true,
+           "subunitcolor": "white"
+          },
+          "hoverlabel": {
+           "align": "left"
+          },
+          "hovermode": "closest",
+          "mapbox": {
+           "style": "light"
+          },
+          "paper_bgcolor": "white",
+          "plot_bgcolor": "#E5ECF6",
+          "polar": {
+           "angularaxis": {
+            "gridcolor": "white",
+            "linecolor": "white",
+            "ticks": ""
+           },
+           "bgcolor": "#E5ECF6",
+           "radialaxis": {
+            "gridcolor": "white",
+            "linecolor": "white",
+            "ticks": ""
+           }
+          },
+          "scene": {
+           "xaxis": {
+            "backgroundcolor": "#E5ECF6",
+            "gridcolor": "white",
+            "gridwidth": 2,
+            "linecolor": "white",
+            "showbackground": true,
+            "ticks": "",
+            "zerolinecolor": "white"
+           },
+           "yaxis": {
+            "backgroundcolor": "#E5ECF6",
+            "gridcolor": "white",
+            "gridwidth": 2,
+            "linecolor": "white",
+            "showbackground": true,
+            "ticks": "",
+            "zerolinecolor": "white"
+           },
+           "zaxis": {
+            "backgroundcolor": "#E5ECF6",
+            "gridcolor": "white",
+            "gridwidth": 2,
+            "linecolor": "white",
+            "showbackground": true,
+            "ticks": "",
+            "zerolinecolor": "white"
+           }
+          },
+          "shapedefaults": {
+           "line": {
+            "color": "#2a3f5f"
+           }
+          },
+          "ternary": {
+           "aaxis": {
+            "gridcolor": "white",
+            "linecolor": "white",
+            "ticks": ""
+           },
+           "baxis": {
+            "gridcolor": "white",
+            "linecolor": "white",
+            "ticks": ""
+           },
+           "bgcolor": "#E5ECF6",
+           "caxis": {
+            "gridcolor": "white",
+            "linecolor": "white",
+            "ticks": ""
+           }
+          },
+          "title": {
+           "x": 0.05
+          },
+          "xaxis": {
+           "automargin": true,
+           "gridcolor": "white",
+           "linecolor": "white",
+           "ticks": "",
+           "title": {
+            "standoff": 15
+           },
+           "zerolinecolor": "white",
+           "zerolinewidth": 2
+          },
+          "yaxis": {
+           "automargin": true,
+           "gridcolor": "white",
+           "linecolor": "white",
+           "ticks": "",
+           "title": {
+            "standoff": 15
+           },
+           "zerolinecolor": "white",
+           "zerolinewidth": 2
+          }
+         }
+        },
+        "title": {
+         "text": "Contour Plot"
+        },
+        "xaxis": {
+         "anchor": "y",
+         "domain": [
+          0,
+          0.2888888888888889
+         ],
+         "matches": "x7",
+         "range": [
+          -6.003146303399008,
+          -4.0517693825541485
+         ],
+         "showticklabels": false,
+         "type": "log"
+        },
+        "xaxis2": {
+         "anchor": "y2",
+         "domain": [
+          0.35555555555555557,
+          0.6444444444444445
+         ],
+         "matches": "x8",
+         "range": [
+          10.399999999999999,
+          133.6
+         ],
+         "showticklabels": false
+        },
+        "xaxis3": {
+         "anchor": "y3",
+         "domain": [
+          0.7111111111111111,
+          1
+         ],
+         "matches": "x9",
+         "range": [
+          0.00023616709705117506,
+          0.298598566193832
+         ],
+         "showticklabels": false
+        },
+        "xaxis4": {
+         "anchor": "y4",
+         "domain": [
+          0,
+          0.2888888888888889
+         ],
+         "matches": "x7",
+         "range": [
+          -6.003146303399008,
+          -4.0517693825541485
+         ],
+         "showticklabels": false,
+         "type": "log"
+        },
+        "xaxis5": {
+         "anchor": "y5",
+         "domain": [
+          0.35555555555555557,
+          0.6444444444444445
+         ],
+         "matches": "x8",
+         "range": [
+          10.399999999999999,
+          133.6
+         ],
+         "showticklabels": false
+        },
+        "xaxis6": {
+         "anchor": "y6",
+         "domain": [
+          0.7111111111111111,
+          1
+         ],
+         "matches": "x9",
+         "range": [
+          0.00023616709705117506,
+          0.298598566193832
+         ],
+         "showticklabels": false
+        },
+        "xaxis7": {
+         "anchor": "y7",
+         "domain": [
+          0,
+          0.2888888888888889
+         ],
+         "range": [
+          -6.003146303399008,
+          -4.0517693825541485
+         ],
+         "title": {
+          "text": "learning_rate"
+         },
+         "type": "log"
+        },
+        "xaxis8": {
+         "anchor": "y8",
+         "domain": [
+          0.35555555555555557,
+          0.6444444444444445
+         ],
+         "range": [
+          10.399999999999999,
+          133.6
+         ],
+         "title": {
+          "text": "per_device_train_batch_size"
+         }
+        },
+        "xaxis9": {
+         "anchor": "y9",
+         "domain": [
+          0.7111111111111111,
+          1
+         ],
+         "range": [
+          0.00023616709705117506,
+          0.298598566193832
+         ],
+         "title": {
+          "text": "weight_decay"
+         }
+        },
+        "yaxis": {
+         "anchor": "x",
+         "domain": [
+          0.7333333333333333,
+          1
+         ],
+         "range": [
+          -6.003146303399008,
+          -4.0517693825541485
+         ],
+         "title": {
+          "text": "learning_rate"
+         },
+         "type": "log"
+        },
+        "yaxis2": {
+         "anchor": "x2",
+         "domain": [
+          0.7333333333333333,
+          1
+         ],
+         "matches": "y",
+         "range": [
+          -6.003146303399008,
+          -4.0517693825541485
+         ],
+         "showticklabels": false,
+         "type": "log"
+        },
+        "yaxis3": {
+         "anchor": "x3",
+         "domain": [
+          0.7333333333333333,
+          1
+         ],
+         "matches": "y",
+         "range": [
+          -6.003146303399008,
+          -4.0517693825541485
+         ],
+         "showticklabels": false,
+         "type": "log"
+        },
+        "yaxis4": {
+         "anchor": "x4",
+         "domain": [
+          0.36666666666666664,
+          0.6333333333333333
+         ],
+         "range": [
+          10.399999999999999,
+          133.6
+         ],
+         "title": {
+          "text": "per_device_train_batch_size"
+         }
+        },
+        "yaxis5": {
+         "anchor": "x5",
+         "domain": [
+          0.36666666666666664,
+          0.6333333333333333
+         ],
+         "matches": "y4",
+         "range": [
+          10.399999999999999,
+          133.6
+         ],
+         "showticklabels": false
+        },
+        "yaxis6": {
+         "anchor": "x6",
+         "domain": [
+          0.36666666666666664,
+          0.6333333333333333
+         ],
+         "matches": "y4",
+         "range": [
+          10.399999999999999,
+          133.6
+         ],
+         "showticklabels": false
+        },
+        "yaxis7": {
+         "anchor": "x7",
+         "domain": [
+          0,
+          0.26666666666666666
+         ],
+         "range": [
+          0.00023616709705117506,
+          0.298598566193832
+         ],
+         "title": {
+          "text": "weight_decay"
+         }
+        },
+        "yaxis8": {
+         "anchor": "x8",
+         "domain": [
+          0,
+          0.26666666666666666
+         ],
+         "matches": "y7",
+         "range": [
+          0.00023616709705117506,
+          0.298598566193832
+         ],
+         "showticklabels": false
+        },
+        "yaxis9": {
+         "anchor": "x9",
+         "domain": [
+          0,
+          0.26666666666666666
+         ],
+         "matches": "y7",
+         "range": [
+          0.00023616709705117506,
+          0.298598566193832
+         ],
+         "showticklabels": false
+        }
+       }
+      }
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "import optuna\n",
+    "import optuna.visualization as vis\n",
+    "\n",
+    "storage = optuna.storages.RDBStorage(\"sqlite:///optuna_trials.db\")\n",
+    "\n",
+    "study = optuna.load_study(\n",
+    "    study_name=\"transformers_optuna_study\",\n",
+    "    storage=storage\n",
+    ")\n",
+    "\n",
+    "vis.plot_param_importances(study).show()\n",
+    "\n",
+    "vis.plot_parallel_coordinate(study).show()\n",
+    "\n",
+    "vis.plot_contour(study).show()\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ae8def79",
+   "metadata": {},
+   "source": [
+    "# Perform the Final Training\n",
+    "\n",
+    "We can now train the model by fetching the best parameters we have discovered by performing Hyperparameter Optimization(HPO). We can now pass the optimized training arguments to configure the essential aspects of training."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "6cd2f800",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "8493882fcaf841f7bf64606edcf2b28e",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Map:   0%|          | 0/25000 [00:00<?, ? examples/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "3626375d66dd49b79684b966e0f190dd",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Map:   0%|          | 0/25000 [00:00<?, ? examples/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "f98a9164e26c432da1cba82ab0dc874e",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Map:   0%|          | 0/50000 [00:00<?, ? examples/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "from datasets import load_dataset\n",
+    "from transformers import AutoTokenizer\n",
+    "\n",
+    "# Load IMDb dataset\n",
+    "dataset = load_dataset(\"imdb\")\n",
+    "\n",
+    "# Load tokenizer\n",
+    "tokenizer = AutoTokenizer.from_pretrained(\"bert-base-uncased\")\n",
+    "\n",
+    "# Tokenize the text\n",
+    "def tokenize_function(example):\n",
+    "    return tokenizer(example[\"text\"], padding=\"max_length\", truncation=True)\n",
+    "\n",
+    "# Apply tokenization\n",
+    "tokenized_dataset = dataset.map(tokenize_function, batched=True)\n",
+    "\n",
+    "# Clean up columns\n",
+    "tokenized_dataset = tokenized_dataset.remove_columns([\"text\"])\n",
+    "tokenized_dataset = tokenized_dataset.rename_column(\"label\", \"labels\")\n",
+    "\n",
+    "# Set PyTorch format\n",
+    "tokenized_dataset.set_format(\"torch\", columns=[\"input_ids\", \"attention_mask\", \"labels\"])\n",
+    "\n",
+    "# Subset for quick testing (optional)\n",
+    "train_dataset = tokenized_dataset[\"train\"].shuffle(seed=42).select(range(2000))\n",
+    "valid_dataset = tokenized_dataset[\"test\"].shuffle(seed=42).select(range(500))\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "07849a94",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.bias', 'classifier.weight']\n",
+      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/tmp/ipykernel_5615/1607841219.py:24: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `Trainer.__init__`. Use `processing_class` instead.\n",
+      "  trainer = Trainer(\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "    <div>\n",
+       "      \n",
+       "      <progress value='375' max='375' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
+       "      [375/375 09:21, Epoch 3/3]\n",
+       "    </div>\n",
+       "    <table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       " <tr style=\"text-align: left;\">\n",
+       "      <th>Epoch</th>\n",
+       "      <th>Training Loss</th>\n",
+       "      <th>Validation Loss</th>\n",
+       "      <th>Accuracy</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <td>1</td>\n",
+       "      <td>0.456700</td>\n",
+       "      <td>0.337511</td>\n",
+       "      <td>0.856000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>2</td>\n",
+       "      <td>0.215200</td>\n",
+       "      <td>0.438220</td>\n",
+       "      <td>0.876000</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <td>3</td>\n",
+       "      <td>0.084600</td>\n",
+       "      <td>0.499159</td>\n",
+       "      <td>0.888000</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table><p>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "from transformers import AutoModelForSequenceClassification, TrainingArguments, Trainer\n",
+    "\n",
+    "# Define the model\n",
+    "model = AutoModelForSequenceClassification.from_pretrained(\"bert-base-uncased\", num_labels=2)\n",
+    "\n",
+    "# Load best hyperparameters (already defined earlier as best_hparams)\n",
+    "training_args = TrainingArguments(\n",
+    "    output_dir=\"./final_model\",\n",
+    "    learning_rate=best_hparams[\"learning_rate\"],\n",
+    "    per_device_train_batch_size=best_hparams[\"per_device_train_batch_size\"],\n",
+    "    weight_decay=best_hparams[\"weight_decay\"],\n",
+    "    \n",
+    "    eval_strategy=\"epoch\",\n",
+    "    save_strategy=\"epoch\",\n",
+    "    load_best_model_at_end=True,\n",
+    "    logging_strategy=\"epoch\",\n",
+    "    num_train_epochs=3,\n",
+    "\n",
+    "    report_to=\"wandb\",\n",
+    "    run_name=\"final_run_with_best_hparams\"\n",
+    ")\n",
+    "\n",
+    "# Create Trainer\n",
+    "trainer = Trainer(\n",
+    "    model=model,\n",
+    "    args=training_args,\n",
+    "    train_dataset=train_dataset,\n",
+    "    eval_dataset=valid_dataset,\n",
+    "    tokenizer=tokenizer,  # ✅ use tokenizer here, not processing_class\n",
+    "    compute_metrics=lambda eval_pred: {\n",
+    "        \"accuracy\": (eval_pred.predictions.argmax(-1) == eval_pred.label_ids).mean()\n",
+    "    }\n",
+    ")\n",
+    "\n",
+    "# Train\n",
+    "trainer.train()\n",
+    "\n",
+    "# Save the model\n",
+    "trainer.save_model(\"./final_model\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "de469553",
+   "metadata": {},
+   "source": [
+    "# Uploading to Hugging Face Hub\n",
+    "\n",
+    "As we can see the model was trained in a more efficient manner and showed reproducible and better results due to Hyperparameter optimization.\n",
+    "We can now save the trained model locally and upload it to the Hugging Face Hub. It is important to remember to login to the Hugging Face Hub using the `huggingface-cli` or `notebook_login()`. Now, we can push the model to the Hub."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "e524424b",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "cb7eabc38cd845929c74f7f52ae98aba",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "README.md:   0%|          | 0.00/31.0 [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "35d0f752cae94cea9082ba475c1940d8",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "model.safetensors:   0%|          | 0.00/438M [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/plain": [
+       "CommitInfo(commit_url='https://huggingface.co/AINovice2005/bert-imdb-optuna-hpo/commit/cf4e9bcfd581cc9cd33f7403c5fa2e5074f58e6c', commit_message='Upload tokenizer', commit_description='', oid='cf4e9bcfd581cc9cd33f7403c5fa2e5074f58e6c', pr_url=None, repo_url=RepoUrl('https://huggingface.co/AINovice2005/bert-imdb-optuna-hpo', endpoint='https://huggingface.co', repo_type='model', repo_id='AINovice2005/bert-imdb-optuna-hpo'), pr_revision=None, pr_num=None)"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from transformers import AutoTokenizer, AutoModelForSequenceClassification\n",
+    "\n",
+    "# Load your saved model from the path\n",
+    "model = AutoModelForSequenceClassification.from_pretrained(\"./final_model\")\n",
+    "tokenizer = AutoTokenizer.from_pretrained(\"./final_model\")\n",
+    "\n",
+    "# Push to your repository on the hub\n",
+    "model.push_to_hub(\"AINovice2005/bert-imdb-optuna-hpo\")\n",
+    "tokenizer.push_to_hub(\"AINovice2005/bert-imdb-optuna-hpo\")\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "optuna",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.12.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}