MS3.ipynb

{"cells":[{"cell_type":"markdown","metadata":{"id":"NDfjE0ir7sSc"},"source":["# Text Summarization Project\n","\n","This notebook is dedicated to the task of text summarization using a fine-tuned T5 model. The goal is to effectively summarize large bodies of text into concise, meaningful summaries.\n","\n","## Description\n","\n","In this project, we are exploring different approaches to text summarization:\n","- Summarizing each paragraph individually and then stitching these summaries together.\n","- Summarizing the entire text in one go.\n","\n","We will compare these methods using ROUGE scores to evaluate their effectiveness.\n","\n","## Steps to Run\n","\n","1. **Add Data to the Runtime (IMPORTANT!)**: I'd say the first thing would be to add the data to the runtime on the side column. For us, we shoudl add the kindle_reviews.csv as well as the test.jsonl. Put both of those in the runtime, no need for folder nesting, etc. Should be sufficient as is.\n","\n","2. **Install Required Libraries**: First cell will install libraries. We are using PyTorch for this.\n","\n","3. **Model Fine-Tuning**: The model then creates a class that will prepare the summaries such that they are in a format to be fed in. In other words, we can't just input text, you have to show it that it's a text / summary pair so T5 knows.\n","\n","4. **Text Summarization**: In this section, we apply the fine-tuned model to summarize the test data. The two approaches (paragraph-by-paragraph and full-text summarization) are implemented and compared. Results are saved to an array. The last one is printed out for idk sanity I guess.\n","\n","5. **Evaluate Summaries**: The summaries generated by each method are evaluated using the ROUGE metric just imported easily. The results are displayed and compared to determine the more effective approach. Here is kinda a summary of what they can mean -\n","\n","Recall essentially answers the following question - \"Of all the relevent information present in the initial summary, how much did the generated summary manage to capture\"\n","\n","Precision answers the following question - \"Of all the information presented in the generated summary, how much of it is relevant or actually appears in the reference summary?\"\n","\n","## Notes\n","\n","One thing to be concerned about is the rapid amount of paragraphs... that's why I added the average paragraph to make sure it was spitting out something reasonable.\n","\n","Pretraining & actually predicting take a long time, so just be wary of that when running. I think that it takes around 10 seconds generally speaking to make a prediction and the training output it shown but it took me 4.5 minutes for say 1000 summaries."]},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":9263,"status":"ok","timestamp":1703283254185,"user":{"displayName":"Tanay Chandak","userId":"17324861930604689779"},"user_tz":360},"id":"FFbkjg4Sybgx","outputId":"e815cc73-62f8-4612-f69f-053a442440ef"},"outputs":[{"output_type":"stream","name":"stdout","text":["Requirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (4.35.2)\n","Collecting datasets\n","  Downloading datasets-2.16.0-py3-none-any.whl (507 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m507.1/507.1 kB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting rouge-score\n","  Downloading rouge_score-0.1.2.tar.gz (17 kB)\n","  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n","Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (2.1.0+cu121)\n","Collecting sentencepiece\n","  Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m14.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hCollecting accelerate\n","  Downloading accelerate-0.25.0-py3-none-any.whl (265 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m265.7/265.7 kB\u001b[0m \u001b[31m15.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers) (3.13.1)\n","Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.19.4)\n","Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers) (1.23.5)\n","Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers) (23.2)\n","Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers) (6.0.1)\n","Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers) (2023.6.3)\n","Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers) (2.31.0)\n","Requirement already satisfied: tokenizers<0.19,>=0.14 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.15.0)\n","Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.4.1)\n","Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers) (4.66.1)\n","Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (10.0.1)\n","Collecting pyarrow-hotfix (from datasets)\n","  Downloading pyarrow_hotfix-0.6-py3-none-any.whl (7.9 kB)\n","Collecting dill<0.3.8,>=0.3.0 (from datasets)\n","  Downloading dill-0.3.7-py3-none-any.whl (115 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m115.3/115.3 kB\u001b[0m \u001b[31m17.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets) (1.5.3)\n","Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from datasets) (3.4.1)\n","Collecting multiprocess (from datasets)\n","  Downloading multiprocess-0.70.15-py310-none-any.whl (134 kB)\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m19.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n","\u001b[?25hRequirement already satisfied: fsspec[http]<=2023.10.0,>=2023.1.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (2023.6.0)\n","Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets) (3.9.1)\n","Requirement already satisfied: absl-py in /usr/local/lib/python3.10/dist-packages (from rouge-score) (1.4.0)\n","Requirement already satisfied: nltk in /usr/local/lib/python3.10/dist-packages (from rouge-score) (3.8.1)\n","Requirement already satisfied: six>=1.14.0 in /usr/local/lib/python3.10/dist-packages (from rouge-score) (1.16.0)\n","Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch) (4.5.0)\n","Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch) (1.12)\n","Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch) (3.2.1)\n","Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch) (3.1.2)\n","Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch) (2.1.0)\n","Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate) (5.9.5)\n","Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (23.1.0)\n","Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (6.0.4)\n","Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.9.4)\n","Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.4.1)\n","Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.1)\n","Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (4.0.3)\n","Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (3.3.2)\n","Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (3.6)\n","Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (2.0.7)\n","Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (2023.11.17)\n","Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch) (2.1.3)\n","Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk->rouge-score) (8.1.7)\n","Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk->rouge-score) (1.3.2)\n","Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2.8.2)\n","Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2023.3.post1)\n","Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch) (1.3.0)\n","Building wheels for collected packages: rouge-score\n","  Building wheel for rouge-score (setup.py) ... \u001b[?25l\u001b[?25hdone\n","  Created wheel for rouge-score: filename=rouge_score-0.1.2-py3-none-any.whl size=24933 sha256=d8d504e577623fa06a809178af64cc082a5619345e804f4598ab505d9ac261e3\n","  Stored in directory: /root/.cache/pip/wheels/5f/dd/89/461065a73be61a532ff8599a28e9beef17985c9e9c31e541b4\n","Successfully built rouge-score\n","Installing collected packages: sentencepiece, pyarrow-hotfix, dill, rouge-score, multiprocess, accelerate, datasets\n","Successfully installed accelerate-0.25.0 datasets-2.16.0 dill-0.3.7 multiprocess-0.70.15 pyarrow-hotfix-0.6 rouge-score-0.1.2 sentencepiece-0.1.99\n"]}],"source":["!pip install transformers datasets rouge-score torch sentencepiece accelerate"]},{"cell_type":"code","execution_count":2,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":278},"executionInfo":{"elapsed":10446,"status":"ok","timestamp":1703283264614,"user":{"displayName":"Tanay Chandak","userId":"17324861930604689779"},"user_tz":360},"id":"VYtvrL_nA8en","outputId":"e6713626-abc5-4ee4-e6e0-e05344273fb3"},"outputs":[{"output_type":"stream","name":"stderr","text":["<ipython-input-2-c0e4de9f7a45>:7: FutureWarning: The error_bad_lines argument has been deprecated and will be removed in a future version. Use on_bad_lines in the future.\n","\n","\n","  df = pd.read_csv(\"kindle_reviews.csv\", error_bad_lines=False, nrows=100)\n"]},{"output_type":"execute_result","data":{"text/plain":["   Unnamed: 0        asin helpful  overall  \\\n","0           0  B000F83SZQ  [0, 0]        5   \n","1           1  B000F83SZQ  [2, 2]        4   \n","2           2  B000F83SZQ  [2, 2]        4   \n","3           3  B000F83SZQ  [1, 1]        5   \n","4           4  B000F83SZQ  [0, 1]        4   \n","\n","                                          reviewText   reviewTime  \\\n","0  I enjoy vintage books and movies so I enjoyed ...   05 5, 2014   \n","1  This book is a reissue of an old one; the auth...   01 6, 2014   \n","2  This was a fairly interesting read.  It had ol...   04 4, 2014   \n","3  I'd never read any of the Amy Brewster mysteri...  02 19, 2014   \n","4  If you like period pieces - clothing, lingo, y...  03 19, 2014   \n","\n","       reviewerID                         reviewerName             summary  \\\n","0  A1F6404F1VG29J                           Avidreader  Nice vintage story   \n","1   AN0N05A9LIJEQ                             critters        Different...   \n","2   A795DMNCJILA6                                  dot               Oldie   \n","3  A1FV0SX13TWVXQ  Elaine H. Turley \"Montana Songbird\"  I really liked it.   \n","4  A3SPTOKDG7WBLN                   Father Dowling Fan      Period Mystery   \n","\n","   unixReviewTime  \n","0      1399248000  \n","1      1388966400  \n","2      1396569600  \n","3      1392768000  \n","4      1395187200  "],"text/html":["\n","  <div id=\"df-9db13636-df68-4e7e-a892-8c210434f92b\" class=\"colab-df-container\">\n","    <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Unnamed: 0</th>\n","      <th>asin</th>\n","      <th>helpful</th>\n","      <th>overall</th>\n","      <th>reviewText</th>\n","      <th>reviewTime</th>\n","      <th>reviewerID</th>\n","      <th>reviewerName</th>\n","      <th>summary</th>\n","      <th>unixReviewTime</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0</td>\n","      <td>B000F83SZQ</td>\n","      <td>[0, 0]</td>\n","      <td>5</td>\n","      <td>I enjoy vintage books and movies so I enjoyed ...</td>\n","      <td>05 5, 2014</td>\n","      <td>A1F6404F1VG29J</td>\n","      <td>Avidreader</td>\n","      <td>Nice vintage story</td>\n","      <td>1399248000</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>1</td>\n","      <td>B000F83SZQ</td>\n","      <td>[2, 2]</td>\n","      <td>4</td>\n","      <td>This book is a reissue of an old one; the auth...</td>\n","      <td>01 6, 2014</td>\n","      <td>AN0N05A9LIJEQ</td>\n","      <td>critters</td>\n","      <td>Different...</td>\n","      <td>1388966400</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>2</td>\n","      <td>B000F83SZQ</td>\n","      <td>[2, 2]</td>\n","      <td>4</td>\n","      <td>This was a fairly interesting read.  It had ol...</td>\n","      <td>04 4, 2014</td>\n","      <td>A795DMNCJILA6</td>\n","      <td>dot</td>\n","      <td>Oldie</td>\n","      <td>1396569600</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>3</td>\n","      <td>B000F83SZQ</td>\n","      <td>[1, 1]</td>\n","      <td>5</td>\n","      <td>I'd never read any of the Amy Brewster mysteri...</td>\n","      <td>02 19, 2014</td>\n","      <td>A1FV0SX13TWVXQ</td>\n","      <td>Elaine H. Turley \"Montana Songbird\"</td>\n","      <td>I really liked it.</td>\n","      <td>1392768000</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>4</td>\n","      <td>B000F83SZQ</td>\n","      <td>[0, 1]</td>\n","      <td>4</td>\n","      <td>If you like period pieces - clothing, lingo, y...</td>\n","      <td>03 19, 2014</td>\n","      <td>A3SPTOKDG7WBLN</td>\n","      <td>Father Dowling Fan</td>\n","      <td>Period Mystery</td>\n","      <td>1395187200</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","    <div class=\"colab-df-buttons\">\n","\n","  <div class=\"colab-df-container\">\n","    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9db13636-df68-4e7e-a892-8c210434f92b')\"\n","            title=\"Convert this dataframe to an interactive table.\"\n","            style=\"display:none;\">\n","\n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n","    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n","  </svg>\n","    </button>\n","\n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    .colab-df-buttons div {\n","      margin-bottom: 4px;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","    <script>\n","      const buttonEl =\n","        document.querySelector('#df-9db13636-df68-4e7e-a892-8c210434f92b button.colab-df-convert');\n","      buttonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","      async function convertToInteractive(key) {\n","        const element = document.querySelector('#df-9db13636-df68-4e7e-a892-8c210434f92b');\n","        const dataTable =\n","          await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                    [key], {});\n","        if (!dataTable) return;\n","\n","        const docLinkHtml = 'Like what you see? Visit the ' +\n","          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","          + ' to learn more about interactive tables.';\n","        element.innerHTML = '';\n","        dataTable['output_type'] = 'display_data';\n","        await google.colab.output.renderOutput(dataTable, element);\n","        const docLink = document.createElement('div');\n","        docLink.innerHTML = docLinkHtml;\n","        element.appendChild(docLink);\n","      }\n","    </script>\n","  </div>\n","\n","\n","<div id=\"df-478af60d-9172-435b-922c-f1a0e3d5c98f\">\n","  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-478af60d-9172-435b-922c-f1a0e3d5c98f')\"\n","            title=\"Suggest charts\"\n","            style=\"display:none;\">\n","\n","<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","     width=\"24px\">\n","    <g>\n","        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n","    </g>\n","</svg>\n","  </button>\n","\n","<style>\n","  .colab-df-quickchart {\n","      --bg-color: #E8F0FE;\n","      --fill-color: #1967D2;\n","      --hover-bg-color: #E2EBFA;\n","      --hover-fill-color: #174EA6;\n","      --disabled-fill-color: #AAA;\n","      --disabled-bg-color: #DDD;\n","  }\n","\n","  [theme=dark] .colab-df-quickchart {\n","      --bg-color: #3B4455;\n","      --fill-color: #D2E3FC;\n","      --hover-bg-color: #434B5C;\n","      --hover-fill-color: #FFFFFF;\n","      --disabled-bg-color: #3B4455;\n","      --disabled-fill-color: #666;\n","  }\n","\n","  .colab-df-quickchart {\n","    background-color: var(--bg-color);\n","    border: none;\n","    border-radius: 50%;\n","    cursor: pointer;\n","    display: none;\n","    fill: var(--fill-color);\n","    height: 32px;\n","    padding: 0;\n","    width: 32px;\n","  }\n","\n","  .colab-df-quickchart:hover {\n","    background-color: var(--hover-bg-color);\n","    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n","    fill: var(--button-hover-fill-color);\n","  }\n","\n","  .colab-df-quickchart-complete:disabled,\n","  .colab-df-quickchart-complete:disabled:hover {\n","    background-color: var(--disabled-bg-color);\n","    fill: var(--disabled-fill-color);\n","    box-shadow: none;\n","  }\n","\n","  .colab-df-spinner {\n","    border: 2px solid var(--fill-color);\n","    border-color: transparent;\n","    border-bottom-color: var(--fill-color);\n","    animation:\n","      spin 1s steps(1) infinite;\n","  }\n","\n","  @keyframes spin {\n","    0% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","      border-left-color: var(--fill-color);\n","    }\n","    20% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    30% {\n","      border-color: transparent;\n","      border-left-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","      border-right-color: var(--fill-color);\n","    }\n","    40% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-top-color: var(--fill-color);\n","    }\n","    60% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","    }\n","    80% {\n","      border-color: transparent;\n","      border-right-color: var(--fill-color);\n","      border-bottom-color: var(--fill-color);\n","    }\n","    90% {\n","      border-color: transparent;\n","      border-bottom-color: var(--fill-color);\n","    }\n","  }\n","</style>\n","\n","  <script>\n","    async function quickchart(key) {\n","      const quickchartButtonEl =\n","        document.querySelector('#' + key + ' button');\n","      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n","      quickchartButtonEl.classList.add('colab-df-spinner');\n","      try {\n","        const charts = await google.colab.kernel.invokeFunction(\n","            'suggestCharts', [key], {});\n","      } catch (error) {\n","        console.error('Error during call to suggestCharts:', error);\n","      }\n","      quickchartButtonEl.classList.remove('colab-df-spinner');\n","      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n","    }\n","    (() => {\n","      let quickchartButtonEl =\n","        document.querySelector('#df-478af60d-9172-435b-922c-f1a0e3d5c98f button');\n","      quickchartButtonEl.style.display =\n","        google.colab.kernel.accessAllowed ? 'block' : 'none';\n","    })();\n","  </script>\n","</div>\n","    </div>\n","  </div>\n"]},"metadata":{},"execution_count":2}],"source":["import pandas as pd\n","import csv\n","from transformers import T5Tokenizer, T5ForConditionalGeneration\n","from datasets import load_dataset, load_metric\n","from torch.utils.data import Dataset, DataLoader\n","\n","df = pd.read_csv(\"kindle_reviews.csv\", error_bad_lines=False, nrows=100)\n","df.head()"]},{"cell_type":"code","execution_count":3,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":169,"referenced_widgets":["4f10a90e02104f34831d64e919bf9150","7b96932cabcf4c86a64be0f6d4febfa6","d78377229df541a29a8157ab6af1a8a1","bc4d671277784308985ccc90d9366186","87ca6dea5b4b48d4bba93a8f7b1d12f4","edf07f8adee54862bfc4b19a5c04f600","44d7dc339b3243c4a3dda8dea8a0fd40","4ab2cdb4e083442b9804dce1b89cd8d3","370a6150566f47f88044200816af4719","257d4fd097d147e28e3f43ea7bbbd140","4186c1686d25449cbba9381d29364b9e","6682d2a9de0e4d6f80ff145ac59015b9","aae91871e9034b9f9b4ee91aee305077","2effc81159ea456cbc4ee4b9d307c387","45e95282b4c94a4e8a7afa2f984f338d","9973d5d4717f4a168018a0f40bb510aa","b9a3d0deae0741ebba8346ce9fab3f3e","15832d9b0b6d4f06a0ee273b3bbf2847","ffc0bfb5b0704719826f1060f54f248e","4f00c6cd94224b6a81b9845493c64c23","1a4fe5abfc4e4b9abba89c34c666dbc4","a5f4e12cd59d49cca11ec27c6854f655","ea68ee2bc76b400a818807580fd697cc","dd5be8d490784d14b3ff67a1302e75c7","fda0f2853e844b1cb63e6a957eb33103","4a8812b4b55244bc9f90e2e661b8d69c","829cb4c1baec459eb37eb2f99f7a13b0","c8b8bae0dd7940bdb173e485406c0186","1cb3909ff78844c28a41e5fac1958c34","9736db4d7cc14b71ab556ce44db6fccc","54a9f80cdcbc44ce890d721b012ce9f7","8327a3775ed34603821f8ceaa95c1feb","999f9a86531e4378a7fe723d3932f64a"]},"executionInfo":{"elapsed":1333,"status":"ok","timestamp":1703283265938,"user":{"displayName":"Tanay Chandak","userId":"17324861930604689779"},"user_tz":360},"id":"3u0s-nldA_dL","outputId":"0490b8ee-2617-4d33-d8be-1f32ac8bdac4"},"outputs":[{"output_type":"display_data","data":{"text/plain":["tokenizer_config.json:   0%|          | 0.00/2.32k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"4f10a90e02104f34831d64e919bf9150"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["spiece.model:   0%|          | 0.00/792k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"6682d2a9de0e4d6f80ff145ac59015b9"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["tokenizer.json:   0%|          | 0.00/1.39M [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"ea68ee2bc76b400a818807580fd697cc"}},"metadata":{}},{"output_type":"stream","name":"stderr","text":["You are using the default legacy behaviour of the <class 'transformers.models.t5.tokenization_t5.T5Tokenizer'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thouroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565\n","Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"]}],"source":["class KindleReviewDataset(Dataset):\n","    def __init__(self, tokenizer, data, max_length=512):\n","        self.tokenizer = tokenizer\n","        self.data = data\n","        self.max_length = max_length\n","\n","    def __len__(self):\n","        return len(self.data)\n","\n","    def __getitem__(self, idx):\n","        item = self.data.iloc[idx]\n","        text = item['reviewText']\n","        summary = item['summary']\n","        inputs = self.tokenizer.encode_plus(\n","            text, max_length=self.max_length, truncation=True, padding='max_length', return_tensors='pt')\n","        targets = self.tokenizer.encode_plus(\n","            summary, max_length=self.max_length, truncation=True, padding='max_length', return_tensors='pt')\n","        return {\n","            'input_ids': inputs['input_ids'].flatten(),\n","            'attention_mask': inputs['attention_mask'].flatten(),\n","            'labels': targets['input_ids'].flatten()\n","        }\n","\n","tokenizer = T5Tokenizer.from_pretrained('t5-small')\n","dataset = KindleReviewDataset(tokenizer, df)"]},{"cell_type":"code","execution_count":4,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":516,"referenced_widgets":["be179ef01d2a4ad69f6e83ca8812155f","b92783d96b16436eb23f83720918d6fc","174b37dfeb8c486495684769e2fb0cef","1636f29a67ba491a99f49702b5fe9eef","ecd655ddca4d4d37b453f1411263d37c","71f3c87c86144005a784f833fed2c840","947f606426f9455299a3b18789830096","22203205c7b94257820af76b6f5465d6","649e0563ee7949b6a04371a2ecc60652","cd06ee7bd67b4e0591f2d4c16aab6f17","3f728f23a74b48d28f26a470119295e7","097580da7cf9484da61ec4e0e1356485","0b3243ce3db144bc995a3bb13fd029c4","489706628ee54b19a0631f4679378711","848f66df58f44b54ad11122cf7c74505","2820aea7d53b41b0b657838dad90cbda","7359c7ba2ae6433fa4d4df1673c6c8f9","ca7f54be59454840b7365b346fef9c86","a3f4c789f8b34116a6bd8acbcac8965c","a499177483c3445f8c894347f909cf02","f6107b8552494ef1904f57820601dad3","2a80e9ab5cdd472483279cadafa0ddca","ffe2216377a34eafb5f6c82bbb4dd48c","0f2c9065a25f41a88086683a21142568","101ac791d79f4f1ab108d94f6d34ab7c","fd3b56f0e1b1474d85f1759a04fb0c6b","a61df9e9135d469689f8550827ec3ac0","6ef5d270f0bc4171aee07ea5ecca4fb4","ce0a77c34aec46478c9fbe5ae2c57dbd","c2ca0915ed4c46378b4729fb1657a247","d14609a2155b4b94992b2b4522eaf6f8","2555341f8ab44488aeaeb38f6929533e","1ac420b4733d41df9a758b97b566a7c8"]},"executionInfo":{"elapsed":42740,"status":"ok","timestamp":1703283308677,"user":{"displayName":"Tanay Chandak","userId":"17324861930604689779"},"user_tz":360},"id":"3wAod4w4E1mx","outputId":"11743254-8541-45c1-ff54-e24274603a0c"},"outputs":[{"output_type":"stream","name":"stdout","text":["Index(['Unnamed: 0', 'asin', 'helpful', 'overall', 'reviewText', 'reviewTime',\n","       'reviewerID', 'reviewerName', 'summary', 'unixReviewTime'],\n","      dtype='object')\n","100\n","Using device: cuda\n"]},{"output_type":"display_data","data":{"text/plain":["config.json:   0%|          | 0.00/1.21k [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"be179ef01d2a4ad69f6e83ca8812155f"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["model.safetensors:   0%|          | 0.00/242M [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"097580da7cf9484da61ec4e0e1356485"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["generation_config.json:   0%|          | 0.00/147 [00:00<?, ?B/s]"],"application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"ffe2216377a34eafb5f6c82bbb4dd48c"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":["<IPython.core.display.HTML object>"],"text/html":["\n","    <div>\n","      \n","      <progress value='75' max='75' style='width:300px; height:20px; vertical-align: middle;'></progress>\n","      [75/75 00:26, Epoch 3/3]\n","    </div>\n","    <table border=\"1\" class=\"dataframe\">\n","  <thead>\n"," <tr style=\"text-align: left;\">\n","      <th>Step</th>\n","      <th>Training Loss</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <td>10</td>\n","      <td>24.449000</td>\n","    </tr>\n","    <tr>\n","      <td>20</td>\n","      <td>24.169500</td>\n","    </tr>\n","    <tr>\n","      <td>30</td>\n","      <td>24.284500</td>\n","    </tr>\n","    <tr>\n","      <td>40</td>\n","      <td>22.178500</td>\n","    </tr>\n","    <tr>\n","      <td>50</td>\n","      <td>22.956500</td>\n","    </tr>\n","    <tr>\n","      <td>60</td>\n","      <td>21.009300</td>\n","    </tr>\n","    <tr>\n","      <td>70</td>\n","      <td>20.622200</td>\n","    </tr>\n","  </tbody>\n","</table><p>"]},"metadata":{}},{"output_type":"execute_result","data":{"text/plain":["TrainOutput(global_step=75, training_loss=22.60388264973958, metrics={'train_runtime': 29.1736, 'train_samples_per_second': 10.283, 'train_steps_per_second': 2.571, 'total_flos': 40602540441600.0, 'train_loss': 22.60388264973958, 'epoch': 3.0})"]},"metadata":{},"execution_count":4}],"source":["from transformers import Trainer, TrainingArguments\n","import torch\n","\n","print(df.columns)\n","print(len(df))\n","\n","# important to use google collab torch instead (if we want to pay, >>)\n","device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n","print(f\"Using device: {device}\")\n","\n","model = T5ForConditionalGeneration.from_pretrained('t5-small').to(device)\n","\n","# fine tunes & trains... not sure if we want to mess with this stuff\n","training_args = TrainingArguments(\n","    output_dir='./results',\n","    num_train_epochs=3,\n","    per_device_train_batch_size=4,\n","    warmup_steps=500,\n","    weight_decay=0.01,\n","    logging_dir='./logs',\n","    logging_steps=10,\n",")\n","\n","# trainer\n","trainer = Trainer(\n","    model=model,\n","    args=training_args,\n","    train_dataset=dataset,\n",")\n","\n","trainer.train()\n"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"background_save":true},"id":"jB3RjG-4zXS9"},"outputs":[],"source":["import json\n","\n","# summarizing text function\n","def summarize_text(text, tokenizer, model):\n","    inputs = tokenizer.encode(\"summarize: \" + text, return_tensors=\"pt\", max_length=512, truncation=True)\n","    inputs = inputs.to(model.device)\n","    outputs = model.generate(inputs, max_length=150, min_length=40, length_penalty=2.0, num_beams=4, early_stopping=True)\n","    return tokenizer.decode(outputs[0], skip_special_tokens=True)\n","\n","# laod first x from dataset (im gonna use 100 for now, we can change it later so it doesn't blow up)\n","test_data = []\n","with open('test.jsonl', 'r') as file:\n","    for i, line in enumerate(file):\n","        if i < 300:\n","            test_data.append(json.loads(line))\n","        else:\n","            break\n","\n","# summarize each paragraph vs entire text\n","paragraph_summaries = []\n","full_text_summaries = []\n","for article in test_data:\n","    # kinda does a lot of paragraph breaks just by nature of the way articles are written\n","    paragraphs = article['text'].split('\\n\\n')\n","    stitched_summary = ' '.join([summarize_text(para, tokenizer, model) for para in paragraphs if para.strip()])\n","    full_summary = summarize_text(article['text'], tokenizer, model)\n","    paragraph_summaries.append(stitched_summary)\n","    full_text_summaries.append(full_summary)\n","\n","# just printing an example summary here\n","print(paragraph_summaries[-1])\n","print(full_text_summaries[-1])\n","print(test_data[-1])"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"background_save":true},"id":"wWYkMO5c0hC-"},"outputs":[],"source":["# small code block I wrote to calculate the average number of sentances in the text...\n","\n","total_paragraphs = 0\n","\n","for article in test_data:\n","    paragraphs = article['text'].split('\\n\\n')\n","    non_empty_paragraphs = sum(1 for para in paragraphs if para.strip())\n","    total_paragraphs += non_empty_paragraphs\n","\n","average_paragraphs_per_article = total_paragraphs / len(test_data)\n","print(\"Average paragraphs per article:\", average_paragraphs_per_article)"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"background_save":true},"id":"I4zOitP5zwQ1"},"outputs":[],"source":["# just using library\n","from datasets import load_metric\n","\n","rouge = load_metric(\"rouge\")\n","\n","def calculate_rouge(predictions, references):\n","    return rouge.compute(predictions=predictions, references=references)\n","\n","rouge_scores_paragraph = calculate_rouge(paragraph_summaries, [article['summary'] for article in test_data])\n","rouge_scores_full = calculate_rouge(full_text_summaries, [article['summary'] for article in test_data])\n","\n","print(\"ROUGE Scores for Paragraph Summaries:\", rouge_scores_paragraph)\n","print(rouge_scores_paragraph['rouge1'])\n","print(\"ROUGE Scores for Full Text Summaries:\", rouge_scores_full)\n","print(rouge_scores_full['rouge1'])"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"background_save":true},"id":"MeNBT20nYMeh"},"outputs":[],"source":["# determining which articles have the highest precision on a paragraph / full text level\n","\n","best_paragraph, para_precision, para_index = '', -float('inf'), -1\n","best_full, full_precision, full_index = '', -float('inf'), -1\n","\n","for i, article in enumerate(test_data):\n","  rouge_scores_paragraph = calculate_rouge([paragraph_summaries[i]], [article['summary']])\n","  rouge_scores_full = calculate_rouge([full_text_summaries[i]], [article['summary']])\n","  if rouge_scores_paragraph['rouge1'][1][0] > para_precision:\n","    best_paragraph, para_precision, para_index = article['summary'], rouge_scores_paragraph['rouge1'][1][0], i\n","  if rouge_scores_full['rouge1'][1][0] > full_precision:\n","    best_full, full_precision, full_index = article['summary'], rouge_scores_full['rouge1'][1][0], i\n"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"UEFel0JtlOWF"},"outputs":[],"source":["# determining which articles have the highest recall on a paragraph / full text level\n","\n","best_paragraph, para_recall, para_index = '', -float('inf'), -1\n","best_full, full_recall, full_index = '', -float('inf'), -1\n","perfect_recall = 0 # counting how many articles have a recall of 1.0\n","\n","for i, article in enumerate(test_data):\n","  rouge_scores_paragraph = calculate_rouge([paragraph_summaries[i]], [article['summary']])\n","  rouge_scores_full = calculate_rouge([full_text_summaries[i]], [article['summary']])\n","  if rouge_scores_paragraph['rouge1'][1][1] > para_recall:\n","    best_paragraph, para_recall, para_index = article['summary'], rouge_scores_paragraph['rouge1'][1][1], i\n","  if rouge_scores_full['rouge1'][1][1] > full_recall:\n","    best_full, full_recall, full_index = article['summary'], rouge_scores_full['rouge1'][1][1], i\n","\n","\n","  if rouge_scores_paragraph['rouge1'][1][1] == 1.0 or rouge_scores_full['rouge1'][1][1] == 1.0:\n","    perfect_recall += 1\n","\n"]}],"metadata":{"accelerator":"GPU","colab":{"machine_shape":"hm","provenance":[]},"kernelspec":{"display_name":"Python 3","name":"python3"},"language_info":{"name":"python"},"widgets":{"application/vnd.jupyter.widget-state+json":{"4f10a90e02104f34831d64e919bf9150":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_7b96932cabcf4c86a64be0f6d4febfa6","IPY_MODEL_d78377229df541a29a8157ab6af1a8a1","IPY_MODEL_bc4d671277784308985ccc90d9366186"],"layout":"IPY_MODEL_87ca6dea5b4b48d4bba93a8f7b1d12f4"}},"7b96932cabcf4c86a64be0f6d4febfa6":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_edf07f8adee54862bfc4b19a5c04f600","placeholder":"​","style":"IPY_MODEL_44d7dc339b3243c4a3dda8dea8a0fd40","value":"tokenizer_config.json: 100%"}},"d78377229df541a29a8157ab6af1a8a1":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_4ab2cdb4e083442b9804dce1b89cd8d3","max":2324,"min":0,"orientation":"horizontal","style":"IPY_MODEL_370a6150566f47f88044200816af4719","value":2324}},"bc4d671277784308985ccc90d9366186":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_257d4fd097d147e28e3f43ea7bbbd140","placeholder":"​","style":"IPY_MODEL_4186c1686d25449cbba9381d29364b9e","value":" 2.32k/2.32k [00:00&lt;00:00, 50.8kB/s]"}},"87ca6dea5b4b48d4bba93a8f7b1d12f4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"edf07f8adee54862bfc4b19a5c04f600":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"44d7dc339b3243c4a3dda8dea8a0fd40":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"4ab2cdb4e083442b9804dce1b89cd8d3":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"370a6150566f47f88044200816af4719":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"257d4fd097d147e28e3f43ea7bbbd140":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4186c1686d25449cbba9381d29364b9e":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6682d2a9de0e4d6f80ff145ac59015b9":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_aae91871e9034b9f9b4ee91aee305077","IPY_MODEL_2effc81159ea456cbc4ee4b9d307c387","IPY_MODEL_45e95282b4c94a4e8a7afa2f984f338d"],"layout":"IPY_MODEL_9973d5d4717f4a168018a0f40bb510aa"}},"aae91871e9034b9f9b4ee91aee305077":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_b9a3d0deae0741ebba8346ce9fab3f3e","placeholder":"​","style":"IPY_MODEL_15832d9b0b6d4f06a0ee273b3bbf2847","value":"spiece.model: 100%"}},"2effc81159ea456cbc4ee4b9d307c387":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_ffc0bfb5b0704719826f1060f54f248e","max":791656,"min":0,"orientation":"horizontal","style":"IPY_MODEL_4f00c6cd94224b6a81b9845493c64c23","value":791656}},"45e95282b4c94a4e8a7afa2f984f338d":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_1a4fe5abfc4e4b9abba89c34c666dbc4","placeholder":"​","style":"IPY_MODEL_a5f4e12cd59d49cca11ec27c6854f655","value":" 792k/792k [00:00&lt;00:00, 4.01MB/s]"}},"9973d5d4717f4a168018a0f40bb510aa":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b9a3d0deae0741ebba8346ce9fab3f3e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"15832d9b0b6d4f06a0ee273b3bbf2847":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ffc0bfb5b0704719826f1060f54f248e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4f00c6cd94224b6a81b9845493c64c23":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"1a4fe5abfc4e4b9abba89c34c666dbc4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a5f4e12cd59d49cca11ec27c6854f655":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ea68ee2bc76b400a818807580fd697cc":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_dd5be8d490784d14b3ff67a1302e75c7","IPY_MODEL_fda0f2853e844b1cb63e6a957eb33103","IPY_MODEL_4a8812b4b55244bc9f90e2e661b8d69c"],"layout":"IPY_MODEL_829cb4c1baec459eb37eb2f99f7a13b0"}},"dd5be8d490784d14b3ff67a1302e75c7":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c8b8bae0dd7940bdb173e485406c0186","placeholder":"​","style":"IPY_MODEL_1cb3909ff78844c28a41e5fac1958c34","value":"tokenizer.json: 100%"}},"fda0f2853e844b1cb63e6a957eb33103":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_9736db4d7cc14b71ab556ce44db6fccc","max":1389353,"min":0,"orientation":"horizontal","style":"IPY_MODEL_54a9f80cdcbc44ce890d721b012ce9f7","value":1389353}},"4a8812b4b55244bc9f90e2e661b8d69c":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_8327a3775ed34603821f8ceaa95c1feb","placeholder":"​","style":"IPY_MODEL_999f9a86531e4378a7fe723d3932f64a","value":" 1.39M/1.39M [00:00&lt;00:00, 19.4MB/s]"}},"829cb4c1baec459eb37eb2f99f7a13b0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c8b8bae0dd7940bdb173e485406c0186":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1cb3909ff78844c28a41e5fac1958c34":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"9736db4d7cc14b71ab556ce44db6fccc":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"54a9f80cdcbc44ce890d721b012ce9f7":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"8327a3775ed34603821f8ceaa95c1feb":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"999f9a86531e4378a7fe723d3932f64a":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"be179ef01d2a4ad69f6e83ca8812155f":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b92783d96b16436eb23f83720918d6fc","IPY_MODEL_174b37dfeb8c486495684769e2fb0cef","IPY_MODEL_1636f29a67ba491a99f49702b5fe9eef"],"layout":"IPY_MODEL_ecd655ddca4d4d37b453f1411263d37c"}},"b92783d96b16436eb23f83720918d6fc":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_71f3c87c86144005a784f833fed2c840","placeholder":"​","style":"IPY_MODEL_947f606426f9455299a3b18789830096","value":"config.json: 100%"}},"174b37dfeb8c486495684769e2fb0cef":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_22203205c7b94257820af76b6f5465d6","max":1206,"min":0,"orientation":"horizontal","style":"IPY_MODEL_649e0563ee7949b6a04371a2ecc60652","value":1206}},"1636f29a67ba491a99f49702b5fe9eef":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_cd06ee7bd67b4e0591f2d4c16aab6f17","placeholder":"​","style":"IPY_MODEL_3f728f23a74b48d28f26a470119295e7","value":" 1.21k/1.21k [00:00&lt;00:00, 23.9kB/s]"}},"ecd655ddca4d4d37b453f1411263d37c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"71f3c87c86144005a784f833fed2c840":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"947f606426f9455299a3b18789830096":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"22203205c7b94257820af76b6f5465d6":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"649e0563ee7949b6a04371a2ecc60652":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"cd06ee7bd67b4e0591f2d4c16aab6f17":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3f728f23a74b48d28f26a470119295e7":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"097580da7cf9484da61ec4e0e1356485":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_0b3243ce3db144bc995a3bb13fd029c4","IPY_MODEL_489706628ee54b19a0631f4679378711","IPY_MODEL_848f66df58f44b54ad11122cf7c74505"],"layout":"IPY_MODEL_2820aea7d53b41b0b657838dad90cbda"}},"0b3243ce3db144bc995a3bb13fd029c4":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_7359c7ba2ae6433fa4d4df1673c6c8f9","placeholder":"​","style":"IPY_MODEL_ca7f54be59454840b7365b346fef9c86","value":"model.safetensors: 100%"}},"489706628ee54b19a0631f4679378711":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_a3f4c789f8b34116a6bd8acbcac8965c","max":242043056,"min":0,"orientation":"horizontal","style":"IPY_MODEL_a499177483c3445f8c894347f909cf02","value":242043056}},"848f66df58f44b54ad11122cf7c74505":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_f6107b8552494ef1904f57820601dad3","placeholder":"​","style":"IPY_MODEL_2a80e9ab5cdd472483279cadafa0ddca","value":" 242M/242M [00:02&lt;00:00, 109MB/s]"}},"2820aea7d53b41b0b657838dad90cbda":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7359c7ba2ae6433fa4d4df1673c6c8f9":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ca7f54be59454840b7365b346fef9c86":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"a3f4c789f8b34116a6bd8acbcac8965c":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a499177483c3445f8c894347f909cf02":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"f6107b8552494ef1904f57820601dad3":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2a80e9ab5cdd472483279cadafa0ddca":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ffe2216377a34eafb5f6c82bbb4dd48c":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_0f2c9065a25f41a88086683a21142568","IPY_MODEL_101ac791d79f4f1ab108d94f6d34ab7c","IPY_MODEL_fd3b56f0e1b1474d85f1759a04fb0c6b"],"layout":"IPY_MODEL_a61df9e9135d469689f8550827ec3ac0"}},"0f2c9065a25f41a88086683a21142568":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_6ef5d270f0bc4171aee07ea5ecca4fb4","placeholder":"​","style":"IPY_MODEL_ce0a77c34aec46478c9fbe5ae2c57dbd","value":"generation_config.json: 100%"}},"101ac791d79f4f1ab108d94f6d34ab7c":{"model_module":"@jupyter-widgets/controls","model_name":"FloatProgressModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_c2ca0915ed4c46378b4729fb1657a247","max":147,"min":0,"orientation":"horizontal","style":"IPY_MODEL_d14609a2155b4b94992b2b4522eaf6f8","value":147}},"fd3b56f0e1b1474d85f1759a04fb0c6b":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","model_module_version":"1.5.0","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_2555341f8ab44488aeaeb38f6929533e","placeholder":"​","style":"IPY_MODEL_1ac420b4733d41df9a758b97b566a7c8","value":" 147/147 [00:00&lt;00:00, 4.83kB/s]"}},"a61df9e9135d469689f8550827ec3ac0":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"6ef5d270f0bc4171aee07ea5ecca4fb4":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ce0a77c34aec46478c9fbe5ae2c57dbd":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c2ca0915ed4c46378b4729fb1657a247":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d14609a2155b4b94992b2b4522eaf6f8":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"2555341f8ab44488aeaeb38f6929533e":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","model_module_version":"1.2.0","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1ac420b4733d41df9a758b97b566a7c8":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","model_module_version":"1.5.0","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}}}},"nbformat":4,"nbformat_minor":0}