scientific-document-similarity-search-with-scibert.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "15923ba6",
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "papermill": {
     "duration": 0.037986,
     "end_time": "2022-01-15T06:24:44.218953",
     "exception": false,
     "start_time": "2022-01-15T06:24:44.180967",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Find Similar Documents From Scientific Corpus Using Deep Learning With SciBERT       \n",
    "This kernel is a comprehensive overview of performing semantic similarity of documents with KNN and Cosine Similarity."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f786243",
   "metadata": {
    "papermill": {
     "duration": 0.035371,
     "end_time": "2022-01-15T06:24:44.292640",
     "exception": false,
     "start_time": "2022-01-15T06:24:44.257269",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Introduction  \n",
    "When reading an interesting article, you might want to find similar articles from the a large number of candidate publications. Manual processing is obviously not the strategy to go for. Why not take advantage of the power of Artificial Intelligence to solve such problem? \n",
    "From this article, you will be able to use SciBERT and cosine similarity in order to find articles that are most similar in meaning to your specific query.  \n",
    "\n",
    "# Approach    \n",
    "Here are the different steps performed \n",
    "* Data extraction and cleaning   \n",
    "* Data Processing \n",
    "    * Load the pretrained model  \n",
    "    * Vectorize documents\n",
    "     \n",
    "* Semantic Similarity search \n",
    "    * Cosine Similarity   \n",
    "    * k-NN with Faiss"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0018ab96",
   "metadata": {
    "papermill": {
     "duration": 0.034689,
     "end_time": "2022-01-15T06:24:44.362817",
     "exception": false,
     "start_time": "2022-01-15T06:24:44.328128",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Useful Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "3e8a1dbd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T06:24:44.451983Z",
     "iopub.status.busy": "2022-01-15T06:24:44.450888Z",
     "iopub.status.idle": "2022-01-15T06:25:11.768999Z",
     "shell.execute_reply": "2022-01-15T06:25:11.768232Z",
     "shell.execute_reply.started": "2022-01-15T05:29:52.859368Z"
    },
    "papermill": {
     "duration": 27.370619,
     "end_time": "2022-01-15T06:25:11.769238",
     "exception": false,
     "start_time": "2022-01-15T06:24:44.398619",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting faiss-gpu\r\n",
      "  Downloading faiss_gpu-1.7.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (85.5 MB)\r\n",
      "     |████████████████████████████████| 85.5 MB 134 kB/s             \r\n",
      "\u001b[?25hInstalling collected packages: faiss-gpu\r\n",
      "Successfully installed faiss-gpu-1.7.2\r\n",
      "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "Data Loading and other libraries\n",
    "\"\"\"\n",
    "import warnings\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from tqdm import tqdm\n",
    "\n",
    "\"\"\"\n",
    "Transformer libraries useful to using the pretrained model and data preprocessing\n",
    "\"\"\"\n",
    "import torch\n",
    "from keras.preprocessing.sequence import pad_sequences\n",
    "from transformers import BertTokenizer,  AutoModelForSequenceClassification\n",
    "\n",
    "\"\"\"\n",
    "Similarity search section: cosine similarity search and facebook AI research library\n",
    "\"\"\"\n",
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "!pip install faiss-gpu # please uncomment this line when you're running the notebook for the first time\n",
    "import faiss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7b207d23",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T06:25:11.934500Z",
     "iopub.status.busy": "2022-01-15T06:25:11.933777Z",
     "iopub.status.idle": "2022-01-15T06:25:11.938606Z",
     "shell.execute_reply": "2022-01-15T06:25:11.937972Z",
     "shell.execute_reply.started": "2022-01-15T05:29:52.869183Z"
    },
    "papermill": {
     "duration": 0.088546,
     "end_time": "2022-01-15T06:25:11.938767",
     "exception": false,
     "start_time": "2022-01-15T06:25:11.850221",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4680eefa",
   "metadata": {
    "papermill": {
     "duration": 0.077844,
     "end_time": "2022-01-15T06:25:12.093686",
     "exception": false,
     "start_time": "2022-01-15T06:25:12.015842",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# About the data   \n",
    "- This CORD-19 data set, a resource of over 59,000 scholarly articles, including over 48,000 with full text, about COVID-19, SARS-CoV-2, and related coronaviruses. \n",
    "- It is downloadable from Kaggle. \n",
    "- Further details about the dataset can be found on this [page](https://www.kaggle.com/danielwolffram/discovid-ai-a-search-and-recommendation-engine/data). "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa5c06bd",
   "metadata": {
    "papermill": {
     "duration": 0.078219,
     "end_time": "2022-01-15T06:25:12.250621",
     "exception": false,
     "start_time": "2022-01-15T06:25:12.172402",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "#### Data loading"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "18c408f2",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T06:25:12.411987Z",
     "iopub.status.busy": "2022-01-15T06:25:12.409432Z",
     "iopub.status.idle": "2022-01-15T06:25:47.269807Z",
     "shell.execute_reply": "2022-01-15T06:25:47.270433Z",
     "shell.execute_reply.started": "2022-01-15T05:29:52.877923Z"
    },
    "papermill": {
     "duration": 34.942065,
     "end_time": "2022-01-15T06:25:47.270623",
     "exception": false,
     "start_time": "2022-01-15T06:25:12.328558",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data Shape: (47110, 16)\n"
     ]
    }
   ],
   "source": [
    "data = pd.read_csv(\"../input/cord19createdataframe/cord19_df.csv\") \n",
    "print(\"Data Shape: {}\".format(data.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "9fe1cc7a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T06:25:47.427576Z",
     "iopub.status.busy": "2022-01-15T06:25:47.426521Z",
     "iopub.status.idle": "2022-01-15T06:25:47.512813Z",
     "shell.execute_reply": "2022-01-15T06:25:47.513401Z",
     "shell.execute_reply.started": "2022-01-15T05:30:10.836009Z"
    },
    "papermill": {
     "duration": 0.168791,
     "end_time": "2022-01-15T06:25:47.513581",
     "exception": false,
     "start_time": "2022-01-15T06:25:47.344790",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "paper_id         0.000000\n",
       "body_text        0.000000\n",
       "methods         58.439822\n",
       "results         63.984292\n",
       "source          10.365103\n",
       "title           10.439397\n",
       "doi             12.604543\n",
       "abstract        12.838039\n",
       "publish_time    10.365103\n",
       "authors         11.676926\n",
       "journal         18.199958\n",
       "arxiv_id        98.681809\n",
       "url             10.617703\n",
       "publish_year     0.000000\n",
       "is_covid19       0.000000\n",
       "study_design     0.000000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Percentage of missing column values\n",
    "percent_missing = data.isnull().sum() * 100 / len(data)\n",
    "percent_missing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66095793",
   "metadata": {
    "papermill": {
     "duration": 0.077098,
     "end_time": "2022-01-15T06:25:47.668302",
     "exception": false,
     "start_time": "2022-01-15T06:25:47.591204",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "There are 47110 articles overall, each one having 16 columns. \n",
    "\n",
    "**Note**: We will focuse our analysis on the **abstract** column for simplicity sake, also it is the one with 0% missing data. But you could use other textual columns such as **body_text**; it is up to you. \n",
    "On the other hand, we will use 2000 observation in order to speed the processing. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a7aff582",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T06:25:47.840058Z",
     "iopub.status.busy": "2022-01-15T06:25:47.833027Z",
     "iopub.status.idle": "2022-01-15T06:25:47.963934Z",
     "shell.execute_reply": "2022-01-15T06:25:47.964506Z",
     "shell.execute_reply.started": "2022-01-15T05:30:10.913682Z"
    },
    "papermill": {
     "duration": 0.218601,
     "end_time": "2022-01-15T06:25:47.964691",
     "exception": false,
     "start_time": "2022-01-15T06:25:47.746090",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "paper_id         0.000000\n",
       "body_text        0.000000\n",
       "methods         53.796698\n",
       "results         59.466173\n",
       "source           8.645463\n",
       "title            8.723394\n",
       "doi             11.095417\n",
       "abstract         0.000000\n",
       "publish_time     8.645463\n",
       "authors          9.432078\n",
       "journal         17.473577\n",
       "arxiv_id        98.487653\n",
       "url              8.927963\n",
       "publish_year     0.000000\n",
       "is_covid19       0.000000\n",
       "study_design     0.000000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# remove articles with missing abstract\n",
    "data = data.dropna(subset = ['abstract'])\n",
    "data = data.reset_index(drop = True)\n",
    "percent_missing = data.isnull().sum() * 100 / len(data)\n",
    "percent_missing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "bf1ccada",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T06:25:48.137075Z",
     "iopub.status.busy": "2022-01-15T06:25:48.135967Z",
     "iopub.status.idle": "2022-01-15T06:25:48.141334Z",
     "shell.execute_reply": "2022-01-15T06:25:48.141937Z",
     "shell.execute_reply.started": "2022-01-15T05:30:11.031532Z"
    },
    "papermill": {
     "duration": 0.099329,
     "end_time": "2022-01-15T06:25:48.142121",
     "exception": false,
     "start_time": "2022-01-15T06:25:48.042792",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Article #3252\n",
      " --> Title: Tuning antiviral CD8 T-cell response via proline-altered peptide ligand vaccination\n",
      " --> Abstract: AbstractViral escape from CD8+ cytotoxic T lymphocyte responses correlates with disease progression and represents a significant challenge for vaccination. Here, we demonstrate that CD8+ T cell recognition of the naturally occurring MHC-I-restricted LCMV-associated immune escape variant Y4F is restored following vaccination with a proline-altered peptide ligand (APL). The APL increases MHC/peptide (pMHC) complex stability, rigidifies the peptide and facilitates T cell receptor (TCR) recognition through reduced entropy costs. Structural analyses of pMHC complexes before and after TCR binding, combined with biophysical analyses, revealed that although the TCR binds similarly to all complexes, the p3P modification alters the conformations of a ...\n",
      "\n",
      "\n",
      "Article #36667\n",
      " --> Title: A highly conserved WDYPKCDRA epitope in the RNA directed RNA polymerase of human coronaviruses can be used as epitope-based universal vaccine design\n",
      " --> Abstract: BACKGROUND: Coronaviruses are the diverse group of RNA virus. From 1960, six strains of human coronaviruses have emerged that includes SARS-CoV and the recent infection by deadly MERS-CoV which is now going to cause another outbreak. Prevention of these viruses is urgent and a universal vaccine for all strain could be a promising solution in this circumstance. In this study we aimed to design an epitope based vaccine against all strain of human coronavirus. RESULTS: Multiple sequence alignment (MSA) approach was employed among spike (S), membrane (M), enveloped (E) and nucleocapsid (N) protein and replicase polyprotein 1ab to identify which ...\n",
      "\n",
      "\n",
      "Article #14038\n",
      " --> Title: Development and characterization of a Rift Valley fever virus cell–cell fusion assay using alphavirus replicon vectors\n",
      " --> Abstract: Abstract Rift Valley fever virus (RVFV), a member of the Phlebovirus genus in the Bunyaviridae family, is transmitted by mosquitoes and infects both humans and domestic animals, particularly cattle and sheep. Since primary RVFV strains must be handled in BSL-3+ or BSL-4 facilities, a RVFV cell–cell fusion assay will facilitate the investigation of RVFV glycoprotein function under BSL-2 conditions. As for other members of the Bunyaviridae family, RVFV glycoproteins are targeted to the Golgi, where the virus buds, and are not efficiently delivered to the cell surface. However, overexpression of RVFV glycoproteins using an alphavirus replicon vector resulted in the ...\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Show first N (default value is 100) words of each of the #total_number random articles\n",
    "def show_random_articles(total_number, df, n=100):\n",
    "    \n",
    "    # Get the random number of articles\n",
    "    n_reviews = df.sample(total_number)\n",
    "    \n",
    "    # Print each one of the articles\n",
    "    for val in list(n_reviews.index):\n",
    "        print(\"Article #{}\".format(val))\n",
    "        print(\" --> Title: {}\".format(df.iloc[val][\"title\"]))\n",
    "        print(\" --> Abstract: {} ...\".format(\" \".join(df.iloc[val][\"abstract\"].split()[:n])))\n",
    "        print(\"\\n\")\n",
    "        \n",
    "# Show 3 random headlines\n",
    "show_random_articles(3, data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "651f609a",
   "metadata": {
    "papermill": {
     "duration": 0.079389,
     "end_time": "2022-01-15T06:25:48.300229",
     "exception": false,
     "start_time": "2022-01-15T06:25:48.220840",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Data Processing & Vectorization     \n",
    "The data processing aims to vectorize the articles' body text so that we can perform the similarity analysis. Since we are dealing with scientific document, we will use the SciBERT model and tokenizer to generate an embedding for each of the articles using their text data.  \n",
    "SciBERT is a pretrained language model for Scientific text data. You can find more information about it on the [Semantic Scholar](https://www.semanticscholar.org/paper/SciBERT%3A-A-Pretrained-Language-Model-for-Scientific-Beltagy-Lo/5e98fe2163640da8ab9695b9ee9c433bb30f5353)   \n",
    "Here is how we proceed:  \n",
    "\n",
    "## Load model artifacts   \n",
    "Load the pretrained model & tokenizer. When loading the pretrained model, we need to set the output_hidden_states to True so that we can extract the embeddings.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "49b14b8a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T06:25:48.463138Z",
     "iopub.status.busy": "2022-01-15T06:25:48.462004Z",
     "iopub.status.idle": "2022-01-15T06:26:11.762010Z",
     "shell.execute_reply": "2022-01-15T06:26:11.763432Z",
     "shell.execute_reply.started": "2022-01-15T05:30:11.048060Z"
    },
    "papermill": {
     "duration": 23.385733,
     "end_time": "2022-01-15T06:26:11.763972",
     "exception": false,
     "start_time": "2022-01-15T06:25:48.378239",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "dc6597d6318c4ff7926ff8d6dce72086",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading:   0%|          | 0.00/223k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3b4b1b1bac634c0da3484a7ab9526c99",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading:   0%|          | 0.00/385 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f0460f6005b94f6e9eb9e444563f1cf3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading:   0%|          | 0.00/422M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of the model checkpoint at allenai/scibert_scivocab_uncased were not used when initializing BertForSequenceClassification: ['cls.predictions.transform.dense.weight', 'cls.seq_relationship.bias', 'cls.predictions.decoder.weight', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.decoder.bias', 'cls.predictions.transform.dense.bias', 'cls.seq_relationship.weight']\n",
      "- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
      "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at allenai/scibert_scivocab_uncased and are newly initialized: ['classifier.bias', 'classifier.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    }
   ],
   "source": [
    "# Get the SciBERT pretrained model path from Allen AI repo\n",
    "pretrained_model = 'allenai/scibert_scivocab_uncased'\n",
    "\n",
    "# Get the tokenizer from the previous path\n",
    "sciBERT_tokenizer = BertTokenizer.from_pretrained(pretrained_model, \n",
    "                                          do_lower_case=True)\n",
    "\n",
    "# Get the model\n",
    "model = AutoModelForSequenceClassification.from_pretrained(pretrained_model,\n",
    "                                                          output_attentions=False,\n",
    "                                                          output_hidden_states=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5014609e",
   "metadata": {
    "papermill": {
     "duration": 0.138463,
     "end_time": "2022-01-15T06:26:12.044653",
     "exception": false,
     "start_time": "2022-01-15T06:26:11.906190",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Transform text data to embeddings   \n",
    "This function *convert_single_abstract_to_embedding* is mostly inspired of the BERT Word [Embeddings Tutorial](https://mccormickml.com/2019/05/14/BERT-word-embeddings-tutorial/#3-extracting-embeddings) of Chris McCormick \n",
    "\n",
    "It aims to create an embedding for a given text data using SciBERT pre-trained model. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "01d0cb22",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T06:26:12.300068Z",
     "iopub.status.busy": "2022-01-15T06:26:12.298921Z",
     "iopub.status.idle": "2022-01-15T06:26:12.302406Z",
     "shell.execute_reply": "2022-01-15T06:26:12.301792Z",
     "shell.execute_reply.started": "2022-01-15T05:30:16.767571Z"
    },
    "papermill": {
     "duration": 0.119839,
     "end_time": "2022-01-15T06:26:12.302565",
     "exception": false,
     "start_time": "2022-01-15T06:26:12.182726",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def convert_single_abstract_to_embedding(tokenizer, model, in_text, MAX_LEN = 510):\n",
    "    \n",
    "    input_ids = tokenizer.encode(\n",
    "                        in_text, \n",
    "                        add_special_tokens = True, \n",
    "                        max_length = MAX_LEN,                           \n",
    "                   )    \n",
    "\n",
    "    results = pad_sequences([input_ids], maxlen=MAX_LEN, dtype=\"long\", \n",
    "                              truncating=\"post\", padding=\"post\")\n",
    "    \n",
    "    # Remove the outer list.\n",
    "    input_ids = results[0]\n",
    "\n",
    "    # Create attention masks    \n",
    "    attention_mask = [int(i>0) for i in input_ids]\n",
    "    \n",
    "    # Convert to tensors.\n",
    "    input_ids = torch.tensor(input_ids)\n",
    "    attention_mask = torch.tensor(attention_mask)\n",
    "\n",
    "    # Add an extra dimension for the \"batch\" (even though there is only one \n",
    "    # input in this batch.)\n",
    "    input_ids = input_ids.unsqueeze(0)\n",
    "    attention_mask = attention_mask.unsqueeze(0)\n",
    "    \n",
    "    # Put the model in \"evaluation\" mode, meaning feed-forward operation.\n",
    "    model.eval()\n",
    "\n",
    "    #input_ids = input_ids.to(device)\n",
    "    #attention_mask = attention_mask.to(device)\n",
    "    \n",
    "    # Run the text through BERT, and collect all of the hidden states produced\n",
    "    # from all 12 layers. \n",
    "    with torch.no_grad():        \n",
    "        logits, encoded_layers = model(\n",
    "                                    input_ids = input_ids, \n",
    "                                    token_type_ids = None, \n",
    "                                    attention_mask = attention_mask,\n",
    "                                    return_dict=False)\n",
    "\n",
    "    layer_i = 12 # The last BERT layer before the classifier.\n",
    "    batch_i = 0 # Only one input in the batch.\n",
    "    token_i = 0 # The first token, corresponding to [CLS]\n",
    "        \n",
    "    # Extract the embedding.\n",
    "    embedding = encoded_layers[layer_i][batch_i][token_i]\n",
    "\n",
    "    # Move to the CPU and convert to numpy ndarray.\n",
    "    embedding = embedding.detach().cpu().numpy()\n",
    "\n",
    "    return(embedding)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e178de7",
   "metadata": {
    "papermill": {
     "duration": 0.082731,
     "end_time": "2022-01-15T06:26:12.466308",
     "exception": false,
     "start_time": "2022-01-15T06:26:12.383577",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Test on a single text data  \n",
    "Here we test the function on the \"abstract\" field of the 30th article. You can choose whatever number you want, as long as it exists in the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "b6d8a177",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T06:26:12.644191Z",
     "iopub.status.busy": "2022-01-15T06:26:12.643054Z",
     "iopub.status.idle": "2022-01-15T06:26:14.809335Z",
     "shell.execute_reply": "2022-01-15T06:26:14.810450Z",
     "shell.execute_reply.started": "2022-01-15T05:30:16.778640Z"
    },
    "papermill": {
     "duration": 2.258374,
     "end_time": "2022-01-15T06:26:14.810706",
     "exception": false,
     "start_time": "2022-01-15T06:26:12.552332",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Embedding shape: (768,)\n"
     ]
    }
   ],
   "source": [
    "input_abstract = data.abstract.iloc[30]\n",
    "\n",
    "# Use the model and tokenizer to generate an embedding for the input_abstract\n",
    "abstract_embedding = convert_single_abstract_to_embedding(sciBERT_tokenizer, model, input_abstract)\n",
    "\n",
    "print('Embedding shape: {}'.format(abstract_embedding.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89d96254",
   "metadata": {
    "papermill": {
     "duration": 0.082601,
     "end_time": "2022-01-15T06:26:14.975353",
     "exception": false,
     "start_time": "2022-01-15T06:26:14.892752",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**(768,)** means that the embedding is composed of 768 values. Now that the convert from text to embedding works, we can finally apply it to all the our text data. But, before that, we are going to remove some columns from the data, in order to have less columns in the result of the final query search. Also, I selected only 2000 articles to perform the analysis, so that the overall processing does not become time-consuming."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "415c9aae",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T06:26:15.147735Z",
     "iopub.status.busy": "2022-01-15T06:26:15.146577Z",
     "iopub.status.idle": "2022-01-15T06:26:15.149891Z",
     "shell.execute_reply": "2022-01-15T06:26:15.149331Z",
     "shell.execute_reply.started": "2022-01-15T05:30:18.161125Z"
    },
    "papermill": {
     "duration": 0.092249,
     "end_time": "2022-01-15T06:26:15.150038",
     "exception": false,
     "start_time": "2022-01-15T06:26:15.057789",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def get_min_viable_data(df, sample_size=2000):\n",
    "    \n",
    "    # Select only the columns we need for the analysis\n",
    "    useless_cols = ['methods', 'results', 'source', 'doi',\n",
    "           'body_text', 'publish_time', 'authors', 'journal', 'arxiv_id',\n",
    "           'publish_year', 'is_covid19', 'study_design']\n",
    "\n",
    "    df.drop(useless_cols, axis=1, inplace=True)\n",
    "\n",
    "    \"\"\"\n",
    "    It was taking too much time to run the analysis on the overall dataset, so I decided to take \n",
    "    a subset (2000 observations) of the original dataset in order to speed the processing.\n",
    "    \"\"\"\n",
    "\n",
    "    df = df.sample(sample_size)\n",
    "    \n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ef4dd499",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T06:26:15.322865Z",
     "iopub.status.busy": "2022-01-15T06:26:15.321702Z",
     "iopub.status.idle": "2022-01-15T06:26:15.323953Z",
     "shell.execute_reply": "2022-01-15T06:26:15.324681Z",
     "shell.execute_reply.started": "2022-01-15T05:30:18.175580Z"
    },
    "papermill": {
     "duration": 0.091611,
     "end_time": "2022-01-15T06:26:15.324851",
     "exception": false,
     "start_time": "2022-01-15T06:26:15.233240",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def convert_overall_text_to_embedding(df):\n",
    "    \n",
    "    # The list of all the embeddings\n",
    "    embeddings = []\n",
    "    \n",
    "    # Get overall text data\n",
    "    overall_text_data = data.abstract.values\n",
    "    \n",
    "    # Loop over all the comment and get the embeddings\n",
    "    for abstract in tqdm(overall_text_data):\n",
    "        \n",
    "        # Get the embedding \n",
    "        embedding = convert_single_abstract_to_embedding(sciBERT_tokenizer, model, abstract)\n",
    "        \n",
    "        #add it to the list\n",
    "        embeddings.append(embedding)\n",
    "        \n",
    "    print(\"Conversion Done!\")\n",
    "    \n",
    "    return embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "69b96954",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T06:26:15.496040Z",
     "iopub.status.busy": "2022-01-15T06:26:15.494963Z",
     "iopub.status.idle": "2022-01-15T07:29:52.289239Z",
     "shell.execute_reply": "2022-01-15T07:29:52.288629Z",
     "shell.execute_reply.started": "2022-01-15T05:30:18.192680Z"
    },
    "papermill": {
     "duration": 3816.883004,
     "end_time": "2022-01-15T07:29:52.289401",
     "exception": false,
     "start_time": "2022-01-15T06:26:15.406397",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 2000/2000 [1:03:36<00:00,  1.91s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Conversion Done!\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "# This task can take a lot of time depending on the sample_size value \n",
    "in the \"get_min_viable_data\" function\n",
    "\"\"\"\n",
    "data = get_min_viable_data(data)\n",
    "embeddings = convert_overall_text_to_embedding(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "20a41122",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:29:53.883703Z",
     "iopub.status.busy": "2022-01-15T07:29:53.882598Z",
     "iopub.status.idle": "2022-01-15T07:29:53.886000Z",
     "shell.execute_reply": "2022-01-15T07:29:53.885279Z",
     "shell.execute_reply.started": "2022-01-15T06:07:18.767983Z"
    },
    "papermill": {
     "duration": 0.771002,
     "end_time": "2022-01-15T07:29:53.886147",
     "exception": false,
     "start_time": "2022-01-15T07:29:53.115145",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create a new column that will contain embedding of each body text\n",
    "def create_final_embeddings(df, embeddings):\n",
    "    \n",
    "    df[\"embeddings\"] = embeddings\n",
    "    df[\"embeddings\"] = df[\"embeddings\"].apply(lambda emb: np.array(emb))\n",
    "    df[\"embeddings\"] = df[\"embeddings\"].apply(lambda emb: emb.reshape(1, -1))\n",
    "    \n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "affea732",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:29:55.443836Z",
     "iopub.status.busy": "2022-01-15T07:29:55.442563Z",
     "iopub.status.idle": "2022-01-15T07:29:55.473247Z",
     "shell.execute_reply": "2022-01-15T07:29:55.473840Z",
     "shell.execute_reply.started": "2022-01-15T06:07:18.775986Z"
    },
    "papermill": {
     "duration": 0.815064,
     "end_time": "2022-01-15T07:29:55.474044",
     "exception": false,
     "start_time": "2022-01-15T07:29:54.658980",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>paper_id</th>\n",
       "      <th>title</th>\n",
       "      <th>abstract</th>\n",
       "      <th>url</th>\n",
       "      <th>embeddings</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>11754</th>\n",
       "      <td>9778c2bdf6be32053f9a450194f767b47c87c891</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Graphical Abstract Highlights d The cryo-EM st...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[[-0.3626312, 0.34195867, 0.40458974, -0.65087...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20720</th>\n",
       "      <td>98fc84b407a8632ee5ac42431d12c6acf7dfe3d8</td>\n",
       "      <td>Genetic Loci That Influence Cause of Death in ...</td>\n",
       "      <td>A genome scan was conducted to seek evidence f...</td>\n",
       "      <td>https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7...</td>\n",
       "      <td>[[-0.49350545, -1.1899799, -0.0245932, -0.8623...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15193</th>\n",
       "      <td>fd0fcdc8d711665338facb36ba94489c3b63a0d7</td>\n",
       "      <td>New Pre-pandemic Influenza Vaccines: An Egg-an...</td>\n",
       "      <td>Highly pathogenic avian H5N1 influenza viruses...</td>\n",
       "      <td>http://europepmc.org/articles/pmc2793094?pdf=r...</td>\n",
       "      <td>[[0.3071721, -0.85410595, 0.43242905, 0.003505...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                       paper_id  \\\n",
       "11754  9778c2bdf6be32053f9a450194f767b47c87c891   \n",
       "20720  98fc84b407a8632ee5ac42431d12c6acf7dfe3d8   \n",
       "15193  fd0fcdc8d711665338facb36ba94489c3b63a0d7   \n",
       "\n",
       "                                                   title  \\\n",
       "11754                                                NaN   \n",
       "20720  Genetic Loci That Influence Cause of Death in ...   \n",
       "15193  New Pre-pandemic Influenza Vaccines: An Egg-an...   \n",
       "\n",
       "                                                abstract  \\\n",
       "11754  Graphical Abstract Highlights d The cryo-EM st...   \n",
       "20720  A genome scan was conducted to seek evidence f...   \n",
       "15193  Highly pathogenic avian H5N1 influenza viruses...   \n",
       "\n",
       "                                                     url  \\\n",
       "11754                                                NaN   \n",
       "20720  https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7...   \n",
       "15193  http://europepmc.org/articles/pmc2793094?pdf=r...   \n",
       "\n",
       "                                              embeddings  \n",
       "11754  [[-0.3626312, 0.34195867, 0.40458974, -0.65087...  \n",
       "20720  [[-0.49350545, -1.1899799, -0.0245932, -0.8623...  \n",
       "15193  [[0.3071721, -0.85410595, 0.43242905, 0.003505...  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = create_final_embeddings(data, embeddings)\n",
    "data.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9732c43",
   "metadata": {
    "papermill": {
     "duration": 0.768636,
     "end_time": "2022-01-15T07:29:57.018467",
     "exception": false,
     "start_time": "2022-01-15T07:29:56.249831",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Similarity Search   \n",
    "Each of the body text data has a corresponding embedding. Now, we can perform the similarity analysis between a given ***query*** vector and all the embeddings vectors. The scope of this article is limited to:  \n",
    "- Cosine similarity which ...     \n",
    "- k-Nearest Neighbor (KNN) search "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c386e074",
   "metadata": {
    "papermill": {
     "duration": 0.770171,
     "end_time": "2022-01-15T07:29:58.559383",
     "exception": false,
     "start_time": "2022-01-15T07:29:57.789212",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Utility functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "d1b2b644",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:30:00.430667Z",
     "iopub.status.busy": "2022-01-15T07:30:00.429547Z",
     "iopub.status.idle": "2022-01-15T07:30:00.431795Z",
     "shell.execute_reply": "2022-01-15T07:30:00.432938Z",
     "shell.execute_reply.started": "2022-01-15T06:07:18.821141Z"
    },
    "papermill": {
     "duration": 1.096778,
     "end_time": "2022-01-15T07:30:00.433107",
     "exception": false,
     "start_time": "2022-01-15T07:29:59.336329",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def process_query(query_text):\n",
    "    \"\"\"\n",
    "    # Create a vector for given query and adjust it for cosine similarity search\n",
    "    \"\"\"\n",
    "\n",
    "    query_vect = convert_single_abstract_to_embedding(sciBERT_tokenizer, model, query_text)\n",
    "    query_vect = np.array(query_vect)\n",
    "    query_vect = query_vect.reshape(1, -1)\n",
    "    return query_vect\n",
    "\n",
    "\n",
    "def get_top_N_articles_cosine(query_text, data, top_N=5):\n",
    "    \"\"\"\n",
    "    Retrieve top_N (5 is default value) articles similar to the query\n",
    "    \"\"\"\n",
    "    query_vect = process_query(query_text)\n",
    "    revevant_cols = [\"title\", \"abstract\", \"url\", \"cos_sim\"]\n",
    "    \n",
    "    # Run similarity Search\n",
    "    data[\"cos_sim\"] = data[\"embeddings\"].apply(lambda x: cosine_similarity(query_vect, x))\n",
    "    data[\"cos_sim\"] = data[\"cos_sim\"].apply(lambda x: x[0][0])\n",
    "    \n",
    "    \"\"\"\n",
    "    Sort Cosine Similarity Column in Descending Order \n",
    "    Here we start at 1 to remove similarity with itself because it is always 1\n",
    "    \"\"\"\n",
    "    moost_similar_articles = data.sort_values(by='cos_sim', ascending=False)[1:top_N+1]\n",
    "    \n",
    "    return moost_similar_articles[revevant_cols]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a955cba7",
   "metadata": {
    "papermill": {
     "duration": 0.814271,
     "end_time": "2022-01-15T07:30:02.070142",
     "exception": false,
     "start_time": "2022-01-15T07:30:01.255871",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Similarity Search with Cosine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "91c0dff3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:30:03.635859Z",
     "iopub.status.busy": "2022-01-15T07:30:03.634295Z",
     "iopub.status.idle": "2022-01-15T07:30:06.311902Z",
     "shell.execute_reply": "2022-01-15T07:30:06.311320Z",
     "shell.execute_reply.started": "2022-01-15T06:07:18.830702Z"
    },
    "papermill": {
     "duration": 3.466171,
     "end_time": "2022-01-15T07:30:06.312068",
     "exception": false,
     "start_time": "2022-01-15T07:30:02.845897",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "query_text_test = data.iloc[0].abstract\n",
    "\n",
    "top_articles = get_top_N_articles_cosine(query_text_test, data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "bcb10e7c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:30:07.843342Z",
     "iopub.status.busy": "2022-01-15T07:30:07.842412Z",
     "iopub.status.idle": "2022-01-15T07:30:07.846454Z",
     "shell.execute_reply": "2022-01-15T07:30:07.847192Z",
     "shell.execute_reply.started": "2022-01-15T06:07:20.412576Z"
    },
    "papermill": {
     "duration": 0.777271,
     "end_time": "2022-01-15T07:30:07.847361",
     "exception": false,
     "start_time": "2022-01-15T07:30:07.070090",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>abstract</th>\n",
       "      <th>url</th>\n",
       "      <th>cos_sim</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>493</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Highlights d ENDU-2 nuclease regulates nucleot...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.825622</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17460</th>\n",
       "      <td>Regioselective synthesis of 6-substituted-2-am...</td>\n",
       "      <td>Abstract A series of 2-amino-5-bromo-4(3H)-pyr...</td>\n",
       "      <td>https://doi.org/10.1016/j.ejmech.2013.06.036</td>\n",
       "      <td>0.773827</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8943</th>\n",
       "      <td>Synthesis of 4-aminoquinoline–pyrimidine hybri...</td>\n",
       "      <td>Abstract One of the most viable options to tac...</td>\n",
       "      <td>https://doi.org/10.1016/j.ejmech.2013.05.046</td>\n",
       "      <td>0.761577</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25022</th>\n",
       "      <td>Australia was indeed the “lucky country” in th...</td>\n",
       "      <td>Anton Y Peleg, Wendy J Munckhof\\nAustralia was...</td>\n",
       "      <td>https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7...</td>\n",
       "      <td>0.757679</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30241</th>\n",
       "      <td>Antigen delivery systems for veterinary vaccin...</td>\n",
       "      <td>Abstract The recent advances in molecular gene...</td>\n",
       "      <td>https://doi.org/10.1016/j.vaccine.2008.09.044</td>\n",
       "      <td>0.754393</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                   title  \\\n",
       "493                                                  NaN   \n",
       "17460  Regioselective synthesis of 6-substituted-2-am...   \n",
       "8943   Synthesis of 4-aminoquinoline–pyrimidine hybri...   \n",
       "25022  Australia was indeed the “lucky country” in th...   \n",
       "30241  Antigen delivery systems for veterinary vaccin...   \n",
       "\n",
       "                                                abstract  \\\n",
       "493    Highlights d ENDU-2 nuclease regulates nucleot...   \n",
       "17460  Abstract A series of 2-amino-5-bromo-4(3H)-pyr...   \n",
       "8943   Abstract One of the most viable options to tac...   \n",
       "25022  Anton Y Peleg, Wendy J Munckhof\\nAustralia was...   \n",
       "30241  Abstract The recent advances in molecular gene...   \n",
       "\n",
       "                                                     url   cos_sim  \n",
       "493                                                  NaN  0.825622  \n",
       "17460       https://doi.org/10.1016/j.ejmech.2013.06.036  0.773827  \n",
       "8943        https://doi.org/10.1016/j.ejmech.2013.05.046  0.761577  \n",
       "25022  https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7...  0.757679  \n",
       "30241      https://doi.org/10.1016/j.vaccine.2008.09.044  0.754393  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "top_articles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "356d0946",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:30:09.388046Z",
     "iopub.status.busy": "2022-01-15T07:30:09.387274Z",
     "iopub.status.idle": "2022-01-15T07:30:09.391102Z",
     "shell.execute_reply": "2022-01-15T07:30:09.391650Z",
     "shell.execute_reply.started": "2022-01-15T06:07:20.425161Z"
    },
    "papermill": {
     "duration": 0.778359,
     "end_time": "2022-01-15T07:30:09.391809",
     "exception": false,
     "start_time": "2022-01-15T07:30:08.613450",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Highlights d ENDU-2 nuclease regulates nucleotide metabolism and germ cell proliferation in worms d ENDU-2 expression is induced by nucleotide imbalance and other genotoxic stresses d ENDU-2 inhibits CTP synthase phosphorylation by repressing PKA and HDA-1 in the gut d ENDU-2 function may be conserved in mammalian cells'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "top_articles.iloc[0].abstract"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "29e34bd6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:30:11.239395Z",
     "iopub.status.busy": "2022-01-15T07:30:11.237842Z",
     "iopub.status.idle": "2022-01-15T07:30:11.241828Z",
     "shell.execute_reply": "2022-01-15T07:30:11.242394Z",
     "shell.execute_reply.started": "2022-01-15T06:07:20.436320Z"
    },
    "papermill": {
     "duration": 1.084305,
     "end_time": "2022-01-15T07:30:11.242582",
     "exception": false,
     "start_time": "2022-01-15T07:30:10.158277",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Abstract A series of 2-amino-5-bromo-4(3H)-pyrimidinone derivatives bearing different substituents at the C-6 position were synthesized using a highly regioselective lithiation–substitution protocol, and the effect of structural variation at the C-6 position on their antiviral activity in cell culture was evaluated. Although some of the derivatives were found to be active against various virus strains, they were effective only close to their toxicity threshold.'"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "top_articles.iloc[1].abstract"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "cad8c412",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:30:12.787304Z",
     "iopub.status.busy": "2022-01-15T07:30:12.786384Z",
     "iopub.status.idle": "2022-01-15T07:30:12.790203Z",
     "shell.execute_reply": "2022-01-15T07:30:12.790772Z",
     "shell.execute_reply.started": "2022-01-15T06:07:20.447286Z"
    },
    "papermill": {
     "duration": 0.776279,
     "end_time": "2022-01-15T07:30:12.790965",
     "exception": false,
     "start_time": "2022-01-15T07:30:12.014686",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Abstract One of the most viable options to tackle the growing resistance to the antimalarial drugs such as artemisinin is to resort to synthetic drugs. The multi-target strategy involving the use of hybrid drugs has shown promise. In line with this, new hybrids of quinoline with pyrimidine have been synthesized and evaluated for their antiplasmodial activity against both CQS and CQR strains of Plasmodium falciparum. These depicted activity in nanomolar range and were found to bind to heme as well as AT rich pUC18 DNA.'"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "top_articles.iloc[2].abstract"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "fd8c3f0e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:30:14.322396Z",
     "iopub.status.busy": "2022-01-15T07:30:14.321387Z",
     "iopub.status.idle": "2022-01-15T07:30:14.325267Z",
     "shell.execute_reply": "2022-01-15T07:30:14.325803Z",
     "shell.execute_reply.started": "2022-01-15T06:07:20.458803Z"
    },
    "papermill": {
     "duration": 0.777504,
     "end_time": "2022-01-15T07:30:14.325988",
     "exception": false,
     "start_time": "2022-01-15T07:30:13.548484",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Anton Y Peleg, Wendy J Munckhof\\nAustralia was indeed the \"lucky country\" in the recent worldwide SARS epidemic 229'"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "top_articles.iloc[3].abstract"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b3d5f01",
   "metadata": {
    "papermill": {
     "duration": 0.766799,
     "end_time": "2022-01-15T07:30:15.859072",
     "exception": false,
     "start_time": "2022-01-15T07:30:15.092273",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Similarity Search Using KNN with Faiss   "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "daf5894f",
   "metadata": {
    "papermill": {
     "duration": 0.781402,
     "end_time": "2022-01-15T07:30:17.412324",
     "exception": false,
     "start_time": "2022-01-15T07:30:16.630922",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Faiss is a library developed by [Facebook AI Research](https://research.facebook.com/research-areas/facebook-ai-research-fair/). According to their [wikipage](https://github.com/facebookresearch/faiss/wiki), \n",
    "> Faiss is a library for efficient similarity search and clustering of dense vectors. It contains algorithms that search in sets of vectors of any size, up to ones that possibly do not fit in RAM\n",
    "\n",
    "Here are the steps to build the search engine using the previously built embeddings  \n",
    "- create the flat index: This is used to flat the vectors. The index uses the L2 (Euclidean) distance metrics to mesure the similarity betweeen the query vector and all the vectors (embeddings). \n",
    "- add all the vectors to the index \n",
    "- define the number **K** of similar document we want \n",
    "- run the similarity search  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "7fd3bb96",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:30:19.020570Z",
     "iopub.status.busy": "2022-01-15T07:30:19.018355Z",
     "iopub.status.idle": "2022-01-15T07:30:19.021342Z",
     "shell.execute_reply": "2022-01-15T07:30:19.021889Z",
     "shell.execute_reply.started": "2022-01-15T06:07:20.467889Z"
    },
    "papermill": {
     "duration": 0.782205,
     "end_time": "2022-01-15T07:30:19.022054",
     "exception": false,
     "start_time": "2022-01-15T07:30:18.239849",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "embedding_dimension = len(embeddings[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "0867214e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:30:20.595081Z",
     "iopub.status.busy": "2022-01-15T07:30:20.594376Z",
     "iopub.status.idle": "2022-01-15T07:30:20.604076Z",
     "shell.execute_reply": "2022-01-15T07:30:20.604637Z",
     "shell.execute_reply.started": "2022-01-15T06:07:20.477312Z"
    },
    "papermill": {
     "duration": 0.806593,
     "end_time": "2022-01-15T07:30:20.604854",
     "exception": false,
     "start_time": "2022-01-15T07:30:19.798261",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "indexFlatL2 = faiss.IndexFlatL2(embedding_dimension)\n",
    "\n",
    "# Convert the embeddings list of vectors into a 2D array.\n",
    "vectors = np.stack(embeddings)\n",
    "\n",
    "indexFlatL2.add(vectors)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "46313951",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:30:23.194136Z",
     "iopub.status.busy": "2022-01-15T07:30:23.193212Z",
     "iopub.status.idle": "2022-01-15T07:30:23.197034Z",
     "shell.execute_reply": "2022-01-15T07:30:23.197658Z",
     "shell.execute_reply.started": "2022-01-15T06:07:20.494863Z"
    },
    "papermill": {
     "duration": 0.87878,
     "end_time": "2022-01-15T07:30:23.197823",
     "exception": false,
     "start_time": "2022-01-15T07:30:22.319043",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total Added Number of Vectors: 2000\n"
     ]
    }
   ],
   "source": [
    "print(\"Total Added Number of Vectors: {}\".format(indexFlatL2.ntotal))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fe8130e",
   "metadata": {
    "papermill": {
     "duration": 0.770633,
     "end_time": "2022-01-15T07:30:24.740682",
     "exception": false,
     "start_time": "2022-01-15T07:30:23.970049",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Perform Query    \n",
    "We will use the same query as previously. Change it to another one if you want.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "bd96fcaa",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:30:26.290006Z",
     "iopub.status.busy": "2022-01-15T07:30:26.288938Z",
     "iopub.status.idle": "2022-01-15T07:30:28.037125Z",
     "shell.execute_reply": "2022-01-15T07:30:28.036080Z",
     "shell.execute_reply.started": "2022-01-15T06:07:20.503547Z"
    },
    "papermill": {
     "duration": 2.53105,
     "end_time": "2022-01-15T07:30:28.037337",
     "exception": false,
     "start_time": "2022-01-15T07:30:25.506287",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Get query vector\n",
    "query_text = data.iloc[0].abstract \n",
    "query_vector = process_query(query_text)\n",
    "\n",
    "K = 5\n",
    "\n",
    "# Run the search\n",
    "D, I = indexFlatL2.search(query_vector, K)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "d2774adf",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:30:29.581987Z",
     "iopub.status.busy": "2022-01-15T07:30:29.580958Z",
     "iopub.status.idle": "2022-01-15T07:30:29.584578Z",
     "shell.execute_reply": "2022-01-15T07:30:29.585164Z",
     "shell.execute_reply.started": "2022-01-15T06:07:21.477665Z"
    },
    "papermill": {
     "duration": 0.775361,
     "end_time": "2022-01-15T07:30:29.585402",
     "exception": false,
     "start_time": "2022-01-15T07:30:28.810041",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[   0, 1683, 1021,  348, 1712]])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "I # this contains the index of all the similar articles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "05788a97",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:30:31.128755Z",
     "iopub.status.busy": "2022-01-15T07:30:31.127546Z",
     "iopub.status.idle": "2022-01-15T07:30:31.131953Z",
     "shell.execute_reply": "2022-01-15T07:30:31.132480Z",
     "shell.execute_reply.started": "2022-01-15T06:07:21.484925Z"
    },
    "papermill": {
     "duration": 0.773851,
     "end_time": "2022-01-15T07:30:31.132655",
     "exception": false,
     "start_time": "2022-01-15T07:30:30.358804",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0.     , 181.52379, 230.63765, 244.74084, 245.87946]],\n",
       "      dtype=float32)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "D # this contains the L2 distance values of all the similar articles"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15d6e12d",
   "metadata": {
    "papermill": {
     "duration": 0.953933,
     "end_time": "2022-01-15T07:30:32.907569",
     "exception": false,
     "start_time": "2022-01-15T07:30:31.953636",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**Note**:  \n",
    "I decided to breakdown all the steps on purpose in order to make sure you understand properly. But you can put everything together into a single function.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "6f1e56ca",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-01-15T07:30:34.607593Z",
     "iopub.status.busy": "2022-01-15T07:30:34.606493Z",
     "iopub.status.idle": "2022-01-15T07:30:34.613868Z",
     "shell.execute_reply": "2022-01-15T07:30:34.613333Z",
     "shell.execute_reply.started": "2022-01-15T06:07:21.496254Z"
    },
    "papermill": {
     "duration": 0.788949,
     "end_time": "2022-01-15T07:30:34.614058",
     "exception": false,
     "start_time": "2022-01-15T07:30:33.825109",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "** Article #0 **\n",
      "** --> Abstract : \n",
      "Graphical Abstract Highlights d The cryo-EM structure of full-length human NPC1 was determined at 4.4 Å resolution d Structure-guided biochemical analysis of cholesterol transfer from NPC2 to NPC1 d Low-resolution cryo-EM structure of NPC1 bound to GPcl of Ebola virus was obtained d A trimeric GPcl binds to one NPC1 through the crystal structure-revealed interface**\n",
      "** --> L2 Distance: 0.00**\n",
      "\n",
      "\n",
      "** Article #1683 **\n",
      "** --> Abstract : \n",
      "Highlights d ENDU-2 nuclease regulates nucleotide metabolism and germ cell proliferation in worms d ENDU-2 expression is induced by nucleotide imbalance and other genotoxic stresses d ENDU-2 inhibits CTP synthase phosphorylation by repressing PKA and HDA-1 in the gut d ENDU-2 function may be conserved in mammalian cells**\n",
      "** --> L2 Distance: 181.52**\n",
      "\n",
      "\n",
      "** Article #1021 **\n",
      "** --> Abstract : \n",
      "Abstract A series of 2-amino-5-bromo-4(3H)-pyrimidinone derivatives bearing different substituents at the C-6 position were synthesized using a highly regioselective lithiation–substitution protocol, and the effect of structural variation at the C-6 position on their antiviral activity in cell culture was evaluated. Although some of the derivatives were found to be active against various virus strains, they were effective only close to their toxicity threshold.**\n",
      "** --> L2 Distance: 230.64**\n",
      "\n",
      "\n",
      "** Article #348 **\n",
      "** --> Abstract : \n",
      "Anton Y Peleg, Wendy J Munckhof\n",
      "Australia was indeed the \"lucky country\" in the recent worldwide SARS epidemic 229**\n",
      "** --> L2 Distance: 244.74**\n",
      "\n",
      "\n",
      "** Article #1712 **\n",
      "** --> Abstract : \n",
      "Abstract One of the most viable options to tackle the growing resistance to the antimalarial drugs such as artemisinin is to resort to synthetic drugs. The multi-target strategy involving the use of hybrid drugs has shown promise. In line with this, new hybrids of quinoline with pyrimidine have been synthesized and evaluated for their antiplasmodial activity against both CQS and CQR strains of Plasmodium falciparum. These depicted activity in nanomolar range and were found to bind to heme as well as AT rich pUC18 DNA.**\n",
      "** --> L2 Distance: 245.88**\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for i in range(I.shape[1]):\n",
    "    \n",
    "    article_index = I[0, i]\n",
    "    \n",
    "    abstract = data.iloc[article_index].abstract\n",
    "    print(\"** Article #{} **\".format(article_index))\n",
    "    print(\"** --> Abstract : \\n{}**\".format(abstract))\n",
    "    print(\"** --> L2 Distance: %.2f**\" % D[0, i])\n",
    "    print(\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbebf027",
   "metadata": {
    "papermill": {
     "duration": 0.771986,
     "end_time": "2022-01-15T07:30:36.161750",
     "exception": false,
     "start_time": "2022-01-15T07:30:35.389764",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**Observation**   \n",
    "- The lower the distance is, the most similar the article is to the query.   \n",
    "- The first document has L2 = 0, which means 100% similarity. This is obvious, because the query was compared with itself. \n",
    "- We can simply remove it to the analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71b4ffcf",
   "metadata": {
    "papermill": {
     "duration": 0.772734,
     "end_time": "2022-01-15T07:30:37.709471",
     "exception": false,
     "start_time": "2022-01-15T07:30:36.936737",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Made with ♥️ by Zoumana   \n",
    "Did you like it?  Git it an upvote and [let's connet](https://medium.com/@zoumanakeita)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 3968.222771,
   "end_time": "2022-01-15T07:30:41.940272",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2022-01-15T06:24:33.717501",
   "version": "2.3.3"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "00e8f51bfab94006a174348bfa299c7b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "091678bec26a44e1aa2c0e4faeb117f8": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "09225643d35b4512a0812f87855b76c2": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_cccbfcc8a6a64691ad4a1667504388d2",
       "placeholder": "​",
       "style": "IPY_MODEL_3cf96a463ecf4f659544e847d66979ad",
       "value": " 385/385 [00:00&lt;00:00, 11.0kB/s]"
      }
     },
     "26b1c416648e4e6a827a603be146946b": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_365e20c488724d8ea1c5c2199f78bd76",
       "max": 385.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_091678bec26a44e1aa2c0e4faeb117f8",
       "value": 385.0
      }
     },
     "290bf1ac0031410ba295792e01560623": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_ec461606d3114ca79862034c9806c968",
       "placeholder": "​",
       "style": "IPY_MODEL_d206eeb3d5a547b9b2a2d18b7eabbc52",
       "value": "Downloading: 100%"
      }
     },
     "365e20c488724d8ea1c5c2199f78bd76": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "3766b78fb5e44941a1c5c126538a800d": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "3b4b1b1bac634c0da3484a7ab9526c99": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_e4e933581e9d495c927cd6d4737ad59d",
        "IPY_MODEL_26b1c416648e4e6a827a603be146946b",
        "IPY_MODEL_09225643d35b4512a0812f87855b76c2"
       ],
       "layout": "IPY_MODEL_50a70c29e6bf4c3d81b3bf4a42232ac4"
      }
     },
     "3cf96a463ecf4f659544e847d66979ad": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "4169d7d17ef74e17809015539122e940": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "4aadedc0f7ff4f0c846ec784e712f3da": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_ba8aa29701b8461e916f561274781924",
       "max": 442221694.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_62287e4e175a420a8b496c41509f0a78",
       "value": 442221694.0
      }
     },
     "50a70c29e6bf4c3d81b3bf4a42232ac4": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "62287e4e175a420a8b496c41509f0a78": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "77de0c4ca0f14bd0a533615ef53cbd83": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_c15a120aec2549b491814d4e37b303c7",
       "max": 227845.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_7c586e8c13d84b4f8ffc020630bb3cbe",
       "value": 227845.0
      }
     },
     "7b2916c4f0f44964841f91502ac1e724": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "7c586e8c13d84b4f8ffc020630bb3cbe": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "808fff53918347ed9c3c6cceb6472cfa": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "80af8bc8cf7b4312bc6cf9b1b7f88b6a": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "86eb31333d494361a31d5f2453049752": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "ad058382593c49dd9679f72292f13e05": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "b1458e5cde2746b592c6f0fde398e61e": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_ad058382593c49dd9679f72292f13e05",
       "placeholder": "​",
       "style": "IPY_MODEL_7b2916c4f0f44964841f91502ac1e724",
       "value": "Downloading: 100%"
      }
     },
     "ba8aa29701b8461e916f561274781924": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "c15a120aec2549b491814d4e37b303c7": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "cccbfcc8a6a64691ad4a1667504388d2": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "d206eeb3d5a547b9b2a2d18b7eabbc52": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "d81fc517dec8498b856c73d7d8ec0a22": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_ed4619531f9644a1afebdd47f4f50050",
       "placeholder": "​",
       "style": "IPY_MODEL_00e8f51bfab94006a174348bfa299c7b",
       "value": " 223k/223k [00:00&lt;00:00, 679kB/s]"
      }
     },
     "da72fe1fba434be2a5f9382f7392bd86": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_4169d7d17ef74e17809015539122e940",
       "placeholder": "​",
       "style": "IPY_MODEL_de410b3cdebc427eb9c439f37ca7e626",
       "value": " 422M/422M [00:15&lt;00:00, 32.2MB/s]"
      }
     },
     "dc6597d6318c4ff7926ff8d6dce72086": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_290bf1ac0031410ba295792e01560623",
        "IPY_MODEL_77de0c4ca0f14bd0a533615ef53cbd83",
        "IPY_MODEL_d81fc517dec8498b856c73d7d8ec0a22"
       ],
       "layout": "IPY_MODEL_3766b78fb5e44941a1c5c126538a800d"
      }
     },
     "de410b3cdebc427eb9c439f37ca7e626": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "DescriptionStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "DescriptionStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "StyleView",
       "description_width": ""
      }
     },
     "e4e933581e9d495c927cd6d4737ad59d": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_tooltip": null,
       "layout": "IPY_MODEL_80af8bc8cf7b4312bc6cf9b1b7f88b6a",
       "placeholder": "​",
       "style": "IPY_MODEL_808fff53918347ed9c3c6cceb6472cfa",
       "value": "Downloading: 100%"
      }
     },
     "ec461606d3114ca79862034c9806c968": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "ed4619531f9644a1afebdd47f4f50050": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "f0460f6005b94f6e9eb9e444563f1cf3": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "1.5.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "1.5.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "1.5.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_b1458e5cde2746b592c6f0fde398e61e",
        "IPY_MODEL_4aadedc0f7ff4f0c846ec784e712f3da",
        "IPY_MODEL_da72fe1fba434be2a5f9382f7392bd86"
       ],
       "layout": "IPY_MODEL_86eb31333d494361a31d5f2453049752"
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}