MAML_Algorithm.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UyhH9-WIBqWF"
      },
      "source": [
        "> ###### *UNIVERSITY OF PISA* - *M.Sc. Computer Science (Artificial Intelligence)*\n",
        ">\n",
        "> **Continual Learning 2022/23**\n",
        ">\n",
        "> Irene Pisani \\\n",
        "> Matricola 560104 \\\n",
        "> i.pisani1@studenti.unipi.it \\\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "buacU7VJa-hT"
      },
      "source": [
        "# **An implementation of  Model Agnostic Meta Learning (MAML) for few-shot supervised image classification**\n",
        "\n",
        "\n",
        "---\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WR-SrqwGaUPI"
      },
      "source": [
        "## **Project objectives**\n",
        "\n",
        "---\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lSfQ4ikYccW4"
      },
      "source": [
        "\n",
        "1.   Provide an implementation  of MAML for few shot supervised learning  using Pytorch.\n",
        "\n",
        "2.   Reproduce the  original experiments on the common few-shot images recognition benchmark: the Omniglot dataset.  Follow the original experimental protocol for image classification: fast learning of N-way classification with 1 or 5 shots and N equal to 5 or 20.\n",
        "\n",
        "3.   Performance comparison between the original and the obtained ones.\n",
        "\n",
        "4.   Analyze the impact of the number of inner SGD step during training and evaluation phases.\n",
        "\n",
        "Steps 1. and 2. are addressed in this notebook `MAML_Algorithm.ipynb`, while steps 3. and 4. are addressed in `MAML_ExperimentalResults.ipynb` .\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZNqXblyKzFVB",
        "outputId": "67c4f569-92ca-415b-e536-a4615f58120b",
        "cellView": "form"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ],
      "source": [
        "# @title Import useful tools and libraries\n",
        "\n",
        "import os\n",
        "import os.path\n",
        "\n",
        "import torch\n",
        "\n",
        "from torch.func import vmap\n",
        "from torch.func import grad\n",
        "from torch.func import functional_call\n",
        "\n",
        "from torch import nn\n",
        "import torch.optim as optim\n",
        "import torch.utils.data as data\n",
        "import torch.nn.functional as F\n",
        "from torch.nn.modules import Module\n",
        "\n",
        "import torchvision\n",
        "import torchvision.transforms as transforms\n",
        "\n",
        "from PIL import Image\n",
        "from types import SimpleNamespace\n",
        "from queue import SimpleQueue\n",
        "\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "from google.colab import drive\n",
        "drive.mount('/content/drive', force_remount = True)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ig-Ikc-4z7u9"
      },
      "source": [
        "\n",
        "## **Omniglot Dataset**\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Auza0e2eEKhi"
      },
      "source": [
        "The Omniglot dataset is a common benchmark for few-shot image recognition and it consists of 20 instances of 1623 characters (each istance is drawn by a different person) from 50 different alphabets.\n",
        "\n",
        "**Dataset split**\n",
        "\n",
        "The dataset was directly imported from `torchvision` where it is stored divided into background dataset and evaluation dataset. Since this partition does not reflect the one used by the authors the two datasets were aggregated and resplit following the splitting protocol suggested by the authors:\n",
        "- Training set (TR) is composed of  1200  characters randomly sampled irrespectively to  the alphabet;\n",
        "- Test set (TS) is composed of the remaining 423 characters.\n",
        "\n",
        "In addition, since a Validation set (VL) is still required for performing model checkpointing, 100 characters from the TR set were used as VL set (i.e., the TR set on which the models were trained consists of 1100 characters).\n",
        "\n",
        "**Dataset preproccessing**\n",
        "\n",
        "As described in the original paper, downsampling to 28 × 28 was performed over all the images (image channels = 1) during data preprocessing.\n",
        "\n",
        "Unlike the authors, the Omniglot dataset is not augmented with rotations of multiples of 90 degrees; the impact of this different choice will be discussed along with the experimental results."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "cellView": "form",
        "id": "L1GYACBrzQzx"
      },
      "outputs": [],
      "source": [
        "#@title Load Dataset\n",
        "\n",
        "import torchvision\n",
        "\n",
        "def LoadDatasets(root, img_size):\n",
        "\n",
        "  # --------------------- Load OMNIGLOT BACKGROUND -----------------------------\n",
        "\n",
        "  background_dataset = torchvision.datasets.Omniglot(\n",
        "      root = root,        # Directory for store dataset\n",
        "      download = True,    # Download zip file\n",
        "      background = True,  # Train set\n",
        "\n",
        "      # Apply trasformation on dataset (es. resizing)\n",
        "      transform  = torchvision.transforms.Compose([\n",
        "          lambda x: x.convert('L'),\n",
        "          lambda x: x.resize((img_size, img_size)),\n",
        "          lambda x: np.reshape(x, (img_size, img_size, 1)),\n",
        "          lambda x: np.transpose(x, [2, 0, 1]),\n",
        "          lambda x: x/255.]))\n",
        "\n",
        "  # Transform Backgorund dataset to dictionary\n",
        "  dict_dataset = dict()\n",
        "\n",
        "  # iterate over each (image, target) pairs contained in the dataset\n",
        "  for img, target in background_dataset:\n",
        "\n",
        "    # add target as key of dataset dictionary\n",
        "    if target not in dict_dataset:\n",
        "      dict_dataset[target] = []\n",
        "\n",
        "    # add image as values of the key in dataset dictionary\n",
        "    dict_dataset[target].append(img)\n",
        "\n",
        "  background_dataset = []\n",
        "\n",
        "  # iterate over each (key:target, value :img) pairs of the dictionary\n",
        "  for y, x in dict_dataset.items():\n",
        "\n",
        "    # add image as numpy array in the list dataset\n",
        "    background_dataset.append(np.array(x).astype('float32'))\n",
        "\n",
        "  # --------------------- Load OMNIGLOT EVALUATION -----------------------------\n",
        "\n",
        "  evaluation_dataset = torchvision.datasets.Omniglot(\n",
        "      root = root,        # Directory for store dataset\n",
        "      download = True,    # Download zip file\n",
        "      background = False,  # Test set\n",
        "\n",
        "      # Apply trasformation on dataset - es. resizing -\n",
        "      transform  = torchvision.transforms.Compose([\n",
        "          lambda x: x.convert('L'),\n",
        "          lambda x: x.resize((img_size, img_size)),\n",
        "          lambda x: np.reshape(x, (img_size, img_size, 1)),\n",
        "          lambda x: np.transpose(x, [2, 0, 1]),\n",
        "          lambda x: x/255.]))\n",
        "\n",
        "  # Transform Evaluation dataset to dictionary\n",
        "  dict_dataset = dict()\n",
        "\n",
        "  # iterate over each (image, target) pairs contained in the dataset\n",
        "  for img, target in evaluation_dataset:\n",
        "\n",
        "    # add target as key of dataset dictionary\n",
        "    if target not in dict_dataset:\n",
        "      dict_dataset[target] = []\n",
        "\n",
        "    # add image as values of the key in dataset dictionary\n",
        "    dict_dataset[target].append(img)\n",
        "\n",
        "  evaluation_dataset = []\n",
        "\n",
        "  # iterate over each (key:target, value :img) pairs of the dictionary\n",
        "  for y, x in dict_dataset.items():\n",
        "\n",
        "    # add image as numpy array in the list dataset\n",
        "    evaluation_dataset.append(np.array(x).astype('float32'))\n",
        "\n",
        "  ## --------------------- FULL DATASET and VALIDATION SPLIT -------------------\n",
        "\n",
        "  # Transform to numpy array\n",
        "  background_dataset = np.array(background_dataset).astype('float32')\n",
        "  evaluation_dataset = np.array(evaluation_dataset).astype('float32')\n",
        "\n",
        "  # Concatenation (background + evaluation)\n",
        "  dataset = np.concatenate((background_dataset, evaluation_dataset), axis=0)\n",
        "\n",
        "  # free memory: delete useless versions the dataset\n",
        "  del dict_dataset\n",
        "  del evaluation_dataset\n",
        "  del background_dataset\n",
        "\n",
        "  # Split full dataset in TR, VL and TS set\n",
        "  dataset = {\n",
        "      \"trainval\": dataset[:1200],\n",
        "      \"train\": dataset[:1100],\n",
        "      \"val\": dataset[1100:1200],\n",
        "      \"test\": dataset[1200:]\n",
        "      }\n",
        "\n",
        "  return dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sw0wyk-Usrev"
      },
      "source": [
        "Due to initial  lack of understanding rather than due to complexity of the implementation, designing a meta-batch sampler take considerable time. Other implementations of MAML available on Github were used as reference point and they helped with proper development. In particular, the code concering the metabatch sampler is strictly inspired by  https://github.com/dragen1860/MAML-Pytorch - a popolar MAML implementation whose code developed for sampling a metabatch is highly reused among a wide number of repository.\n",
        "\n",
        "The aim of `MetaBatchSampler` class is to prepare the dataset for being feed to to the base learner under the MAML algortihm settings for n-way k-shot supervised classification.\n",
        "\n",
        "Given the hyper-parameter `n`, `k_support` (k -shot for support set), `k_query` (k-shot for query set), and `meta_batch_size`, each dataset split (TR, VL or TS) is treated has follow:\n",
        "\n",
        "  - `load_metabatch_queque()` preload a queque of 10 metabatch.\n",
        "  - `next()` gets the next the metabatch in the queque for passing it to the learner, refilling the queque if it's empty.\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "cellView": "form",
        "id": "2qnosr25zeh-"
      },
      "outputs": [],
      "source": [
        "#@title Metabach Sampler\n",
        "class MetaBatchSampler:\n",
        "\n",
        "  def __init__(self, options, device = None):\n",
        "\n",
        "    # get configuration values\n",
        "    self.root = options.dataset_root\n",
        "    self.meta_batch_size = options.meta_batch_size\n",
        "    self.n_way = options.n_way\n",
        "    self.k_support = options.k_support\n",
        "    self.k_query = options.k_query\n",
        "    self.img_size = options.img_size\n",
        "\n",
        "    # device\n",
        "    self.device = device\n",
        "\n",
        "    # Load Omniglot dataset\n",
        "    self.datasets = LoadDatasets(root = self.root, img_size = self.img_size)\n",
        "\n",
        "    # dictionary of datasets splits: each one is made of a preloaded queque of metabatches\n",
        "    self.metabatch_queque = {\n",
        "        \"trainval\":  self.load_metabatch_queque(self.datasets[\"trainval\"]),\n",
        "        \"train\": self.load_metabatch_queque(self.datasets[\"train\"]),\n",
        "        \"val\": self.load_metabatch_queque(self.datasets[\"val\"]),\n",
        "        \"test\": self.load_metabatch_queque(self.datasets[\"test\"])\n",
        "        }\n",
        "\n",
        "  # ------------- LOAD A QUEQUE OF METABATCHES ---------------------------------\n",
        "\n",
        "  def load_metabatch_queque(self, dataset):\n",
        "\n",
        "    # size of query set and support set\n",
        "    support_set_size = self.k_support * self.n_way\n",
        "    query_set_size   = self.k_query * self.n_way\n",
        "\n",
        "    # initialize a queque of metabatches with a given num. of preloaded  metabatches\n",
        "    metabatch_queque = SimpleQueue()\n",
        "    preloaded_metabatch = 10\n",
        "\n",
        "    # iterate over num of preloaded metabatch\n",
        "    for _ in range(preloaded_metabatch):\n",
        "\n",
        "      # initialze empty metabatch\n",
        "      metabatch_x_spt, metabatch_y_spt, metabatch_x_qry, metabatch_y_qry = [], [], [], []\n",
        "\n",
        "      # iterate over size of metabatch\n",
        "      for i in range(self.meta_batch_size):\n",
        "\n",
        "        # initialze empty support set and query set\n",
        "        x_support, y_support, x_query, y_query = [], [], [], []\n",
        "\n",
        "        # randomly sample n classes among all the available in the dataset\n",
        "        sampled_class = np.random.choice(dataset.shape[0], self.n_way, False)\n",
        "\n",
        "        # iterate over sampled classes\n",
        "        for j, target in enumerate(sampled_class):\n",
        "\n",
        "          # for each class randomly sample k support + k query images without replacement\n",
        "          sampled_imgs = np.random.choice(dataset.shape[1], self.k_support + self.k_query, False)\n",
        "\n",
        "          # add the first k sampled images to support set and the remaining to query set\n",
        "          x_support.append(dataset[target][sampled_imgs[:self.k_support]])\n",
        "          x_query.append(dataset[target][sampled_imgs[self.k_support:]])\n",
        "\n",
        "          # store corresponding labels\n",
        "          y_support.append([j]*self.k_support)\n",
        "          y_query.append([j]*self.k_query)\n",
        "\n",
        "        # shuffle the support set: [support_size, 1, 28, 28]\n",
        "        perm = np.random.permutation(support_set_size)\n",
        "        x_support = np.array(x_support).reshape(support_set_size, 1, self.img_size, self.img_size)[perm]\n",
        "        y_support = np.array(y_support).reshape(support_set_size)[perm]\n",
        "\n",
        "        # shuffle the query set: [query_size, 1, 28, 28]\n",
        "        perm = np.random.permutation(query_set_size)\n",
        "        x_query = np.array(x_query).reshape(query_set_size, 1, self.img_size, self.img_size)[perm]\n",
        "        y_query = np.array(y_query).reshape(query_set_size)[perm]\n",
        "\n",
        "        # append the created support and query sets to the metabatch\n",
        "        metabatch_x_spt.append(x_support) # metabatch of x support\n",
        "        metabatch_y_spt.append(y_support) # metabatch of y support\n",
        "        metabatch_x_qry.append(x_query) # metabatch of x query\n",
        "        metabatch_y_qry.append(y_query) # metabatch of y query\n",
        "\n",
        "      # meta batch of support set: [meta batch size, support_set_size, 1, 28, 28]\n",
        "      metabatch_x_spt = np.array(metabatch_x_spt).astype('float32').reshape(self.meta_batch_size, support_set_size, 1, self.img_size, self.img_size)\n",
        "      metabatch_y_spt = np.array(metabatch_y_spt).astype(int).reshape(self.meta_batch_size, support_set_size)\n",
        "\n",
        "      # meta batch of support set: [meta batch size, query_set_size, 1, 28, 28]\n",
        "      metabatch_x_qry = np.array(metabatch_x_qry).astype('float32').reshape(self.meta_batch_size, query_set_size, 1, self.img_size, self.img_size)\n",
        "      metabatch_y_qry = np.array(metabatch_y_qry).astype(int).reshape(self.meta_batch_size, query_set_size)\n",
        "\n",
        "      # transform from numpy to pytorch tensor\n",
        "      metabatch_x_spt, metabatch_y_spt, metabatch_x_qry, metabatch_y_qry = [\n",
        "          torch.from_numpy(b).to(self.device) for b in [metabatch_x_spt, metabatch_y_spt, metabatch_x_qry, metabatch_y_qry]\n",
        "      ]\n",
        "\n",
        "      # put the created meta batch to the metabatch queque\n",
        "      metabatch_queque.put([metabatch_x_spt, metabatch_y_spt, metabatch_x_qry, metabatch_y_qry])\n",
        "\n",
        "    return metabatch_queque\n",
        "\n",
        "  # ------------- GET NEXT METABATCH IN THE QUEQUE -----------------------------\n",
        "\n",
        "  def next(self, mode='trainval'):\n",
        "\n",
        "    # update queque if it is empty\n",
        "    if self.metabatch_queque[mode].empty():\n",
        "      self.metabatch_queque[mode] = self.load_metabatch_queque(self.datasets[mode])\n",
        "\n",
        "    # get next metabatch, given the dataset split specified in 'mode' parameter\n",
        "    next_metabatch = self.metabatch_queque[mode].get()\n",
        "\n",
        "    return next_metabatch\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gQ7bnLTI0KUT"
      },
      "source": [
        "## **Model**\n",
        "\n",
        "---\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G-tnkrME_0yE"
      },
      "source": [
        "The model follows the same architecture proposed by the author:\n",
        "4  modules with a 3 × 3 strided convolutions and 64 filters, followed by batch normalization, and ReLU non linearity.The dimensionality of the last hidden layer is 64 and it is fed into a soft-max.\n",
        "\n",
        "Stride = 2 and padding = 1 were used but note that this values may not coincide with the ones used by the author since they do not specify these values inside the paper. In fact, by looking at the original code https://pytorch.org/docs/stable/func.html I was not able to properly understand the stride value and to asses whatever padding were used or not.\n",
        "\n",
        "Batch normalization is used with `track_running_stats = False`. The motivation come from some statements provided by Antreas et al (2019) in *How to train your MAML* and by Bronskill et al (2020) in *TASKNORM: Rethinking Batch Normalization for Meta-Learning*. They both claimed  that in the original MAML implementation, instead of accumulating running statistics, the statistics of the current batch were used for batch normalization also during evaluation phase.\n",
        "\n",
        "`inplace = True` for ReLu computation was initially used to manage some memory issues, but in the end it results being useless. Some preliminary experiments show that this attribute does not affect the performance.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "6vAl89dizkHA"
      },
      "outputs": [],
      "source": [
        "#@title Model architecture\n",
        "\n",
        "class FinnCNN(nn.Module):\n",
        "\n",
        "  def __init__(self, n_way, device):\n",
        "    super().__init__()\n",
        "\n",
        "    self.cnn = nn.Sequential(\n",
        "\n",
        "        # 1° convolutional block\n",
        "        nn.Conv2d(1, 64, 3, stride = 2, padding = 1),\n",
        "        nn.BatchNorm2d(64, affine=True, track_running_stats=False),\n",
        "        nn.ReLU(inplace=True),\n",
        "\n",
        "        # 2° convolutional block\n",
        "        nn.Conv2d(64, 64, 3, stride = 2, padding = 1),\n",
        "        nn.BatchNorm2d(64, affine=True, track_running_stats=False),\n",
        "        nn.ReLU(inplace=True),\n",
        "\n",
        "        # 3° convolutional block\n",
        "        nn.Conv2d(64, 64, 3, stride = 2, padding = 1),\n",
        "        nn.BatchNorm2d(64, affine=True, track_running_stats=False),\n",
        "        nn.ReLU(inplace=True),\n",
        "\n",
        "        # 4° convolutional block\n",
        "        nn.Conv2d(64, 64, 3, stride = 2, padding = 0),\n",
        "        nn.BatchNorm2d(64, affine=True, track_running_stats=False),\n",
        "        nn.ReLU(inplace=True),\n",
        "\n",
        "        # Classifier layer\n",
        "        nn.Flatten(),\n",
        "        nn.Linear(64, n_way)).to(device)\n",
        "\n",
        "  def forward(self, x):\n",
        "    x = self.cnn(x)\n",
        "    return x"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OlQMLrfiBYA4"
      },
      "source": [
        "## **MAML: Model Agnostic Meta Learning**\n",
        "\n",
        "---\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VCFrYIYySgEz"
      },
      "source": [
        "The `MAML()` class performs the core of the algorithm.\n",
        "\n",
        "All the required parameter are given as input together with a preloaded queque of metabatch.\n",
        "It initialize a base learner (by calling `FinnCNN()`) and a gradient-based meta optimizer (Adam).\n",
        "\n",
        "By calling the `MAML.fit_and_evaluate()` method, the model is trained on TR set and evaluated on VL set.\n",
        "\n",
        "**Model checkpointing** is used to find the best model monitoring the validation loss during training - best model has the lowest validation loss.\n",
        "Note that model selection phase with a grid search over the hyperparameter space was not performed due to computational and time constraint. All the hyperparamter involved are the same ones mentioned in the original paper.\n",
        "Anyway, model checkpointing allow to save the best model among the iterations and this technique is required since for 20-way classification significant training oscillations have been observed (they will be further disussed in experimental results section).\n",
        "\n",
        "By calling the `MAML.test()` method, best model's performance are assessed on TS set.  \n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7vpCznhQxlc_"
      },
      "source": [
        "**Implementation details with torch.func**\n",
        "\n",
        "Most of the core steps of MAML were implemented exploiting some torch.func functionalities (note that functorch is currently migrated on torch.func) and following [PyTorch](https://pytorch.org/docs/stable/index.html) and [torch.func](https://pytorch.org/docs/stable/func.html) available documentations.\n",
        "\n",
        "*  `train()` function perform MAML training over a single epoch (during 1 epochs the algorithm is trained with ` total number of class in TR set // meta-batch size` distinct metabatches).\n",
        "\n",
        "  * Run in parallel the inner loop on each task in the metabatch, in order to efficiently get the query losses required for the meta/outer update, using:\n",
        "  ```\n",
        "  # Compute outer losses mappping inner loop function to each task (with corresponding support and query set)\n",
        "  all_inner_loss, all_inner_acc, all_loss, all_acc = vmap(self.inner_loop)(x_support, y_support, x_query, y_query)\n",
        "  ```\n",
        "\n",
        "  * In the inner loop fit the model to a task - get adapted paramter θ' starting from θ - without forgetting the current θ parameter values:  \n",
        "  ```\n",
        "  # get model's output (on support set) by replacing parameters and buffers\n",
        "  logits = functional_call(self.model, (params, buffers), x_support)\n",
        "  ```\n",
        "  allow to compute the output of the models with the specified parameters and  buffers  instead of  the current ones. In this way, the θ parameter that need to be updated in the outer loop were still remembered.\n",
        "  \n",
        "  * In addition, when adapting the model to a task using SGD step - get adapted paramter θ' - it's necessary to explicitly computing the gradient of the support loss via\n",
        "  ```\n",
        "  # compute gradients of inner loss\n",
        "  grad(compute_support_loss)(params, buffers, support_x, support_y)\n",
        "  ```\n",
        "  and use it in SGD step. `backward()` method is not possible here since it would overwrite θ parameter instead of θ' of each task. Also SGD update rules was explicitly computed.\n",
        "\n",
        "* `evaluation()` function is used to asses model performance on TS or VL dataset. Here, the model is evaluted after being adapted to support sets with some SGD steps (note that the number of SGD inner step for training and evaluation could be different) and the performance of the query sets are returned.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "qATJ3uEb_0nW"
      },
      "outputs": [],
      "source": [
        "#@title Model Agnostic Meta Learning\n",
        "\n",
        "class MAML():\n",
        "\n",
        "  # ------------ INITIALIZATION ------------------------------------------------\n",
        "\n",
        "  def __init__(self, dataset, options, device, path):\n",
        "\n",
        "    self.path = path # model checkpoint (for saving or load)\n",
        "\n",
        "    self.meta_lr = options.meta_lr   # beta learning rate in outer loop\n",
        "    self.inner_lr = options.inner_lr # alfa learning rate in inner loop\n",
        "    self.tr_inner_steps = options.inner_step # optimization step in inner loop during training\n",
        "    self.all_ts_inner_steps = options.ts_inner_step # all considered optimization step in inner loop during evaluation\n",
        "    self.ts_inner_steps = 3 # set 3 for 5-way and set 5 for 20-way : deafault optimization step in inner loop during evaluation\n",
        "    self.iterations = options.iterations  # epochs\n",
        "\n",
        "    # Omniglot datasets (TR, VL, TS) already divided in metabatch\n",
        "    self.dataset = dataset\n",
        "\n",
        "    # Base Learner: Convolutional NN described by Finn et. al\n",
        "    self.model = FinnCNN(options.n_way, device)\n",
        "\n",
        "    # Meta optimizer: Adam\n",
        "    self.meta_optimizer = optim.Adam(self.model.parameters(), lr = self.meta_lr)\n",
        "\n",
        "    # Num. of metabatch for each dataset split\n",
        "    self.num_tr_metabatch = self.dataset.datasets['train'].shape[0] // self.dataset.meta_batch_size\n",
        "    self.num_trvl_metabatch = self.dataset.datasets['trainval'].shape[0] // self.dataset.meta_batch_size\n",
        "    self.num_vl_metabatch = self.dataset.datasets['val'].shape[0] // self.dataset.meta_batch_size\n",
        "    self.num_ts_metabatch = self.dataset.datasets['test'].shape[0] // self.dataset.meta_batch_size\n",
        "\n",
        "    # initialize optimal loss for model checkpointing\n",
        "    self.optimal_loss = 100000\n",
        "\n",
        "    # Store performance history\n",
        "    self.tr_history = {\n",
        "        \"tr_loss\" : [],\n",
        "        \"tr_accuracy\": [],\n",
        "        \"tr_inner_loss\": [],\n",
        "        \"tr_inner_accuracy\" : []\n",
        "        }\n",
        "    self.vl_history = {\n",
        "        \"vl_loss\" : [],\n",
        "        \"vl_accuracy\": [],\n",
        "        \"vl_inner_loss\": [],\n",
        "        \"vl_inner_accuracy\" : [],\n",
        "\n",
        "        }\n",
        "    self.ts_history = {\n",
        "        \"ts_inner_steps\":[],\n",
        "        \"ts_loss\" : [],\n",
        "        \"ts_accuracy\" : [],\n",
        "        \"ts_inner_loss\" : [],\n",
        "        \"ts_inner_accuracy\" : [],\n",
        "    }\n",
        "\n",
        "  def get_inner_loss(self, params, buffers, x, y):\n",
        "\n",
        "    # Get model's output on support set by replacing parameters and buffers\n",
        "    logits = functional_call(self.model, (params, buffers), x)\n",
        "\n",
        "    # compute cross entropy loss function on support set\n",
        "    loss = F.cross_entropy(logits, y)\n",
        "\n",
        "    return loss\n",
        "\n",
        "\n",
        "  # ------------------------ TRAIN - INNER LOOP on SINGLE TASK -----------------\n",
        "\n",
        "  def inner_loop(self, x_support, y_support, x_query, y_query):\n",
        "\n",
        "    # store model parameters and buffers\n",
        "    params = dict(self.model.named_parameters())\n",
        "    buffers = dict(self.model.named_buffers())\n",
        "\n",
        "    # Size of query set\n",
        "    query_size = x_query.size(dim = 0)\n",
        "    support_size = x_support.size(dim = 0)\n",
        "\n",
        "    # iterate over inner steps\n",
        "    for step in range(self.tr_inner_steps):\n",
        "\n",
        "      if step + 1 == self.tr_inner_steps:\n",
        "        # get model's output (on support set) by replacing parameters and buffers\n",
        "        logits = functional_call(self.model, (params, buffers), x_support)\n",
        "\n",
        "        # compute cross entropy loss function on support set\n",
        "        support_loss = F.cross_entropy(logits, y_support)\n",
        "\n",
        "        # compute accuracy on support set\n",
        "        support_acc = (logits.argmax(dim=1) == y_support).sum() / support_size\n",
        "\n",
        "      # compute gradients of inner loss\n",
        "      grads = grad(self.get_inner_loss)(params, buffers, x_support, y_support)\n",
        "\n",
        "      # compute adapted parameters with gradient descent:\n",
        "      # (!) params = params - alfa * gridient of the loss\n",
        "      params = {k: params[k] - g * self.inner_lr for k, g, in grads.items()}\n",
        "\n",
        "    # get model's output (on query set) by replacing parameters and buffers\n",
        "    logits = functional_call(self.model, (params, buffers), x_query)\n",
        "\n",
        "    # compute cross entropy loss function on query set\n",
        "    query_loss = F.cross_entropy(logits, y_query)\n",
        "\n",
        "    # compute accuracy on query set\n",
        "    query_acc = (logits.argmax(dim=1) == y_query).sum() / query_size\n",
        "\n",
        "\n",
        "    return support_loss, support_acc, query_loss, query_acc\n",
        "\n",
        "\n",
        "  # ------------------------ TRAIN - OUTER LOOP  -------------------------------\n",
        "\n",
        "  def train(self, retrain = True):\n",
        "\n",
        "    # set model in training state\n",
        "    self.model.train()\n",
        "\n",
        "    # initialize metrics and loss for the whole epoch\n",
        "    meta_loss, meta_acc = 0, 0\n",
        "    inner_loss, inner_acc = 0, 0\n",
        "\n",
        "    # set train dataset (TR or TR+VL) and num metabatches\n",
        "    if retrain:\n",
        "      mode_train = 'trainval'\n",
        "      num_metabatch = self.num_trvl_metabatch\n",
        "    else:\n",
        "      mode_train = 'train'\n",
        "      num_metabatch = self.num_tr_metabatch\n",
        "\n",
        "    # iterate over total number of metabach per iterations\n",
        "    for i in range(num_metabatch):\n",
        "\n",
        "      # Sample a single metabatch (inner + query) from train set\n",
        "      x_support, y_support, x_query, y_query = self.dataset.next(mode = mode_train)\n",
        "\n",
        "      # total number of tasks in the sampled metabatch\n",
        "      num_tasks = x_support.size(dim = 0)\n",
        "\n",
        "      # set gradients to zero before updating\n",
        "      self.meta_optimizer.zero_grad()\n",
        "\n",
        "      # (!) for each task in the current metabach train one model (inner loop on single task)\n",
        "\n",
        "      # Compute outer losses mappping inner loop function to each task (with corresponding support and query set)\n",
        "      all_inner_loss, all_inner_acc, all_loss, all_acc = vmap(self.inner_loop)(x_support, y_support, x_query, y_query)\n",
        "\n",
        "      # add chunk_size = 1 size to perform vmap in a for-fashioned loop (to avoid cuda out of memory issues)\n",
        "      #all_inner_loss, all_inner_acc, all_loss, all_acc = vmap(self.inner_loop, chunk_size = 1)(x_support, y_support, x_query, y_query)\n",
        "\n",
        "      # Compute gradients of the meta (query) losses\n",
        "      all_loss.sum().backward()\n",
        "\n",
        "      # Meta update: update model parameters\n",
        "      self.meta_optimizer.step()\n",
        "\n",
        "      all_inner_loss = all_inner_loss.detach().sum() / num_tasks\n",
        "      all_inner_acc = 100. * all_inner_acc.sum() / num_tasks\n",
        "\n",
        "      # Compute mean meta-loss and mean-meta accuracy over tasks\n",
        "      all_loss = all_loss.detach().sum() / num_tasks\n",
        "      all_acc = 100. * all_acc.sum() / num_tasks\n",
        "\n",
        "      # update loss and accuracy after each metabach\n",
        "      meta_loss += all_loss.item()\n",
        "      meta_acc += all_acc.item()\n",
        "      inner_loss += all_inner_loss.item()\n",
        "      inner_acc += all_inner_acc.item()\n",
        "\n",
        "    # Get meta-loss and meta-accuracy of the overall iteration\n",
        "    inner_loss = inner_loss / num_metabatch\n",
        "    inner_acc = inner_acc / num_metabatch\n",
        "    meta_loss = meta_loss / num_metabatch\n",
        "    meta_acc = meta_acc / num_metabatch\n",
        "\n",
        "    return inner_loss, inner_acc, meta_loss, meta_acc\n",
        "\n",
        "\n",
        "  # ------------------------ EVALUATION on VL/TS -------------------------------\n",
        "\n",
        "  def evaluation(self, mode_eval):\n",
        "\n",
        "    # set model in evaluation status\n",
        "    self.model.eval()\n",
        "\n",
        "    # initialize array for storing perforamance\n",
        "    qry_losses = []\n",
        "    qry_accs = []\n",
        "    spt_losses = []\n",
        "    spt_accs = []\n",
        "\n",
        "    # choose dataset for evaluation\n",
        "    if mode_eval == \"val\":\n",
        "        num_metabatch = self.num_vl_metabatch\n",
        "    else:\n",
        "        num_metabatch = self.num_ts_metabatch\n",
        "\n",
        "    # iterate over the number of metabatch\n",
        "    for i in range(num_metabatch):\n",
        "\n",
        "      # sample a single metabatch (inner + query) from train set\n",
        "      x_support, y_support, x_query, y_query = self.dataset.next(mode = mode_eval)\n",
        "\n",
        "      # total number of tasks in the sampled metabatch\n",
        "      task_num = x_support.size(dim = 0)\n",
        "\n",
        "      # iterate over the number of tasks\n",
        "      for j in range(task_num):\n",
        "\n",
        "        # Inner loop on single task\n",
        "\n",
        "        # get original model parameter and buffer\n",
        "        new_params = dict(self.model.named_parameters())\n",
        "        buffers = dict(self.model.named_buffers())\n",
        "\n",
        "        # Adapt model to the task by iterating over inner steps\n",
        "        for _ in range(self.ts_inner_steps):\n",
        "\n",
        "          # get model's output on support set replacing paramters and buffers\n",
        "          spt_logits = functional_call(self.model, (new_params, buffers), x_support[j])\n",
        "\n",
        "          # compute inner loss function on support set\n",
        "          spt_loss = F.cross_entropy(spt_logits, y_support[j])\n",
        "\n",
        "          # compute gradients of inner loss function\n",
        "          grads = torch.autograd.grad(spt_loss, new_params.values())\n",
        "\n",
        "          # update model parameter using stocastic gradient descent\n",
        "          new_params = {k: new_params[k] - g * self.inner_lr for k, g, in zip(new_params, grads)}\n",
        "\n",
        "        # compute logits on query set using adapted parameter\n",
        "        qry_logits = functional_call(self.model, (new_params, buffers), x_query[j]).detach()\n",
        "\n",
        "        # compute loss on query set\n",
        "        qry_loss = F.cross_entropy(qry_logits, y_query[j], reduction='none')\n",
        "\n",
        "        # add support loss and accuracy to history\n",
        "        spt_losses.append(spt_loss.detach())\n",
        "        spt_accs.append((spt_logits.argmax(dim=1) == y_support[j]).detach())\n",
        "\n",
        "        # add query loss and accuracy to history\n",
        "        qry_losses.append(qry_loss.detach())\n",
        "        qry_accs.append((qry_logits.argmax(dim=1) == y_query[j]).detach())\n",
        "\n",
        "    # get mean inner performance over metabatches\n",
        "    spt_losses = torch.mean(torch.stack(spt_losses)).item()\n",
        "    spt_accs = 100. * torch.mean(torch.stack(spt_accs).float()).item()\n",
        "\n",
        "    # get mean meta performance over metabatches\n",
        "    qry_losses = torch.cat(qry_losses).mean().item()\n",
        "    qry_accs = 100. * torch.cat(qry_accs).float().mean().item()\n",
        "\n",
        "    return  spt_losses, spt_accs, qry_losses, qry_accs\n",
        "\n",
        "\n",
        "  # ------------------------ FIT MAML to DATASET -------------------------------\n",
        "\n",
        "  def fit_and_evaluate(self):\n",
        "\n",
        "    # ---> RUN TRAINING on TRAIN SET and VALIDATE on VALIDATION SET\n",
        "    #self.model.train()\n",
        "    for iteration in range(self.iterations):\n",
        "\n",
        "      # train on Train Set (no final retraining on TR+VL) and evaluate on Validation set\n",
        "      tr_inner_loss, tr_inner_acc, tr_loss, tr_acc = self.train(retrain = False)\n",
        "      vl_inner_loss, vl_inner_acc, vl_loss, vl_acc = self.evaluation(mode_eval = \"val\")\n",
        "\n",
        "      # add performance to history\n",
        "      self.tr_history[\"tr_accuracy\"].append(tr_acc)\n",
        "      self.vl_history[\"vl_accuracy\"].append(vl_acc)\n",
        "      self.tr_history[\"tr_loss\"].append(tr_loss)\n",
        "      self.vl_history[\"vl_loss\"].append(vl_loss)\n",
        "      self.tr_history[\"tr_inner_loss\"].append(tr_inner_loss)\n",
        "      self.tr_history[\"tr_inner_accuracy\"].append(tr_inner_acc)\n",
        "      self.vl_history[\"vl_inner_loss\"].append(vl_inner_loss)\n",
        "      self.vl_history[\"vl_inner_accuracy\"].append(vl_inner_acc)\n",
        "\n",
        "      '''\n",
        "      (!) Typically Hold-Out Validation technique requires a final retraining on full TR+VL set\n",
        "      Here final retraining has been not executed due to time constraints\n",
        "      (Training phase is too time-consuming)\n",
        "      VL set is still useful model checkponinting\n",
        "      '''\n",
        "\n",
        "      # show performance to monitor training\n",
        "      if iteration % 10 == 0 or iteration == 99:\n",
        "        print(\n",
        "            f'[Epoch {iteration}] | ',\n",
        "            f'[TR] Meta Loss: {tr_loss:.2f} - Meta Acc: {tr_acc:.2f} -',\n",
        "            f'Inner Loss: {tr_inner_loss:.2f} -  Inner Acc: {tr_inner_acc:.2f} |',\n",
        "            f'[VL] Meta Loss: {vl_loss:.2f} -  Meta Acc: {vl_acc:.2f} -',\n",
        "            f'Inner Loss: {vl_inner_loss:.2f} - Inner Acc: {vl_inner_acc:.2f} |'\n",
        "            )\n",
        "\n",
        "      # model checkpointing: save model if the val loss is the lowest so far\n",
        "      if vl_loss <= self.optimal_loss:\n",
        "\n",
        "        # update optimal loss\n",
        "        self.optimal_loss = vl_loss\n",
        "\n",
        "        # save model\n",
        "        torch.save({\n",
        "                'epoch': iteration,\n",
        "                'model_state_dict': self.model.state_dict(),\n",
        "                'optimizer_state_dict': self.meta_optimizer.state_dict(),\n",
        "                'loss': vl_loss,\n",
        "                }, self.path)\n",
        "\n",
        "\n",
        "    return self.tr_history, self.vl_history\n",
        "\n",
        "  # ------------------------ MAKE INFERENCE on TEST SET ------------------------\n",
        "\n",
        "  def test(self, more_ts_inner_step = True):\n",
        "\n",
        "    # load best model from checkpoint\n",
        "    checkpoint = torch.load(self.path)\n",
        "    self.model.load_state_dict(checkpoint['model_state_dict'])\n",
        "    self.meta_optimizer.load_state_dict(checkpoint['optimizer_state_dict'])\n",
        "\n",
        "    if more_ts_inner_step:\n",
        "      # Make inference with different number of inner step\n",
        "\n",
        "      for innerstep in self.all_ts_inner_steps:\n",
        "        self.ts_inner_steps = innerstep\n",
        "\n",
        "        for _ in range(10): # Make inference 10 times\n",
        "\n",
        "          # make inference\n",
        "          ts_inner_loss, ts_inner_acc, ts_loss, ts_acc = self.evaluation(mode_eval =\"test\")\n",
        "\n",
        "          # save test performance\n",
        "          self.ts_history[\"ts_inner_steps\"].append(self.ts_inner_steps)\n",
        "          self.ts_history[\"ts_loss\"].append(ts_loss)\n",
        "          self.ts_history[\"ts_accuracy\"].append(ts_acc)\n",
        "          self.ts_history[\"ts_inner_loss\"].append(ts_inner_loss)\n",
        "          self.ts_history[\"ts_inner_accuracy\"].append(ts_inner_acc)\n",
        "\n",
        "      return self.ts_history\n",
        "\n",
        "    else: # make inference just for deafult ts_inner_step\n",
        "\n",
        "      # make inference\n",
        "      ts_inner_loss, ts_inner_acc, ts_loss, ts_acc = self.evaluation(mode_eval =\"test\")\n",
        "\n",
        "      # show test performance\n",
        "      print(\n",
        "          f'inner-step: {self.ts_inner_steps}|',\n",
        "          f'Meta-Test Loss: {ts_loss:.2f} | Meta-Test accuracy: {ts_acc:.2f} | ',\n",
        "          f'Inner-Test Loss: {ts_inner_loss:.2f} | Inner-Test Acc: {ts_inner_acc:.2f} | '\n",
        "            )\n",
        "\n",
        "      return\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "srPxUNsCRUU4"
      },
      "source": [
        "## **Experiments**\n",
        "\n",
        "---\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "| Hyper-Parameters or config. | 5 way | 20 way |\n",
        "|-----|--------|---|\n",
        "| k shot for support| 5 or 1 | 5 or 1 |\n",
        "| k shot for query |  20 - k shot for support | 20 - k shot for support|\n",
        "| Inner and Outer Loss | Cross Entropy | Cross Entropy |\n",
        "| Outer optimizer | Adam | Adam |\n",
        "| Outer learning rate | 1e-3 | 1e-3 |\n",
        "| Inner optimizer | SGD | SGD |\n",
        "| Inner learning rate | 0.4 | 0.1 |\n",
        "| Meta batch size | 32 | 16 |\n",
        "| Deafult inner step (trainining) | 1 | 5 |\n",
        "| Deafult inner step (evaluation) | 3 | 5 |\n",
        "| Max epochs | 100 | 100 |\n",
        "| image input size | 28 | 28 |\n",
        "| augmentation with rotations | no | no |\n",
        "| image input channel | 1 | 1 |\n",
        "| Convolutional Model | Finn et. al (2017) | Finn et. al (2017) |\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "cmmkjx23BaUN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "All the model were trained over 100 epochs according to the experiments carried out in *How to train your MAML*, since training the models over 60000 iterations - like in the original paper by Finn (2017) - would have been too expensive."
      ],
      "metadata": {
        "id": "vHucFS55IxC1"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "JIwufjZgzNIX"
      },
      "outputs": [],
      "source": [
        "#@title Configuration and hyperparameters set-up\n",
        "options = SimpleNamespace(\n",
        "\n",
        "    # path to data\n",
        "    dataset_root='.' + os.sep + 'dataset',\n",
        "\n",
        "    # number of classes for n way classification\n",
        "    n_way = 5, # 5 or 20\n",
        "\n",
        "    # k shot for support set\n",
        "    k_support = 5, # 5 or 1\n",
        "\n",
        "    # k shot for query set\n",
        "    k_query = 15,\n",
        "\n",
        "    # Number of tasks per meta batch\n",
        "    meta_batch_size = 32, # 16 for 20-way and 32 for 5-way\n",
        "\n",
        "    # Meta learning rate (alfa)\n",
        "    meta_lr = 1e-3,\n",
        "\n",
        "    # Inner learning rate (beta)\n",
        "    inner_lr = 0.4, #0.4 for 5-way and 0.1 for 20-way\n",
        "\n",
        "    # Image size for resizing\n",
        "    img_size = 28,\n",
        "\n",
        "    # Iterations\n",
        "    iterations = 100,\n",
        "\n",
        "    # Inner steps  [tested values for 5-way: 1 3 5]\n",
        "    inner_step = 1, # deafult: 1 for 5-way and 5 for 20-way\n",
        "\n",
        "    # Eval inner steps [tested values: 1 3 5 8]\n",
        "    ts_inner_step = [1, 3, 5, 10], # deafult 3 for 5-way, 5 for 20-way\n",
        "\n",
        "    # seed\n",
        "    seed = 1,\n",
        "\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "CrgUh_Dr_5Bv"
      },
      "outputs": [],
      "source": [
        "# Specify seed and device\n",
        "\n",
        "torch.manual_seed(options.seed)\n",
        "\n",
        "if torch.cuda.is_available():\n",
        "    torch.cuda.manual_seed_all(options.seed)\n",
        "np.random.seed(options.seed)\n",
        "\n",
        "if torch.cuda.is_available():\n",
        "  device = 'cuda'\n",
        "else:\n",
        "  device= 'cpu'\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dtSzWsmIGso3",
        "outputId": "7738ab3e-f306-4369-bb66-35399e052a40"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Files already downloaded and verified\n",
            "Files already downloaded and verified\n"
          ]
        }
      ],
      "source": [
        "# Initialize the metabatch sampler for loading and preparing Omniglot dataset\n",
        "data_sampler = MetaBatchSampler(options, device)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "83q1O01DFfsO",
        "outputId": "9ea61801-a1a6-4f02-fa7d-3e2a62877cfc"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[Epoch 0] |  [TR] Meta Loss: 0.69 - Meta Acc: 81.27 - Inner Loss: 1.65 -  Inner Acc: 19.78 | [VL] Meta Loss: 0.42 -  Meta Acc: 86.92 - Inner Loss: 0.13 - Inner Acc: 99.75 |\n",
            "[Epoch 10] |  [TR] Meta Loss: 0.07 - Meta Acc: 98.01 - Inner Loss: 1.67 -  Inner Acc: 20.42 | [VL] Meta Loss: 0.14 -  Meta Acc: 95.22 - Inner Loss: 0.01 - Inner Acc: 100.00 |\n",
            "[Epoch 20] |  [TR] Meta Loss: 0.04 - Meta Acc: 98.66 - Inner Loss: 1.68 -  Inner Acc: 20.24 | [VL] Meta Loss: 0.12 -  Meta Acc: 95.58 - Inner Loss: 0.01 - Inner Acc: 100.00 |\n",
            "[Epoch 30] |  [TR] Meta Loss: 0.03 - Meta Acc: 98.95 - Inner Loss: 1.70 -  Inner Acc: 20.37 | [VL] Meta Loss: 0.14 -  Meta Acc: 95.42 - Inner Loss: 0.00 - Inner Acc: 99.96 |\n",
            "[Epoch 40] |  [TR] Meta Loss: 0.03 - Meta Acc: 99.22 - Inner Loss: 1.70 -  Inner Acc: 20.20 | [VL] Meta Loss: 0.13 -  Meta Acc: 95.57 - Inner Loss: 0.00 - Inner Acc: 100.00 |\n",
            "[Epoch 50] |  [TR] Meta Loss: 0.03 - Meta Acc: 99.15 - Inner Loss: 1.73 -  Inner Acc: 19.63 | [VL] Meta Loss: 0.10 -  Meta Acc: 96.46 - Inner Loss: 0.00 - Inner Acc: 100.00 |\n",
            "[Epoch 60] |  [TR] Meta Loss: 0.02 - Meta Acc: 99.33 - Inner Loss: 1.72 -  Inner Acc: 20.13 | [VL] Meta Loss: 0.11 -  Meta Acc: 96.79 - Inner Loss: 0.00 - Inner Acc: 99.92 |\n",
            "[Epoch 70] |  [TR] Meta Loss: 0.02 - Meta Acc: 99.51 - Inner Loss: 1.71 -  Inner Acc: 19.54 | [VL] Meta Loss: 0.09 -  Meta Acc: 97.03 - Inner Loss: 0.00 - Inner Acc: 100.00 |\n",
            "[Epoch 80] |  [TR] Meta Loss: 0.02 - Meta Acc: 99.48 - Inner Loss: 1.73 -  Inner Acc: 19.96 | [VL] Meta Loss: 0.11 -  Meta Acc: 96.33 - Inner Loss: 0.00 - Inner Acc: 100.00 |\n",
            "[Epoch 90] |  [TR] Meta Loss: 0.01 - Meta Acc: 99.56 - Inner Loss: 1.75 -  Inner Acc: 19.98 | [VL] Meta Loss: 0.12 -  Meta Acc: 96.69 - Inner Loss: 0.01 - Inner Acc: 99.71 |\n",
            "[Epoch 99] |  [TR] Meta Loss: 0.01 - Meta Acc: 99.52 - Inner Loss: 1.74 -  Inner Acc: 19.94 | [VL] Meta Loss: 0.09 -  Meta Acc: 96.97 - Inner Loss: 0.00 - Inner Acc: 100.00 |\n",
            "inner-step: 3| Meta-Test Loss: 0.05 | Meta-Test accuracy: 98.23 |  Inner-Test Loss: 0.00 | Inner-Test Acc: 99.99 | \n"
          ]
        }
      ],
      "source": [
        "# path to load or store model\n",
        "checkpoint_path  = str(f'/content/drive/MyDrive/MAML/{options.n_way}way/{options.k_support}shot/Model/NEW{options.inner_step}innerstep.pt')\n",
        "\n",
        "# path for storing performance\n",
        "history_train = str(f'/content/drive/MyDrive/MAML/{options.n_way}way/{options.k_support}shot/TR_{options.inner_step}innerstep.csv')\n",
        "history_val = str(f'/content/drive/MyDrive/MAML/{options.n_way}way/{options.k_support}shot/VL_{options.inner_step}innerstep.csv')\n",
        "history_test  = str(f'/content/drive/MyDrive/MAML/{options.n_way}way/{options.k_support}shot/TS_{options.inner_step}-{options.ts_inner_step}innerstep.csv')\n",
        "\n",
        "# initialize MAML\n",
        "maml = MAML(data_sampler, options, device, checkpoint_path)\n",
        "\n",
        "# train on TR and evaluate on VL\n",
        "tr_history, vl_history = maml.fit_and_evaluate()\n",
        "\n",
        "# test on TS with deafult number of SGD inner step for evaluation\n",
        "maml.test(more_ts_inner_step = False)\n",
        "\n",
        "# test on TS with different number of SGD inner step for evaluation\n",
        "ts_history = maml.test()\n",
        "\n",
        "\n",
        "# Save TR performance in csv files for further analysis\n",
        "df = pd.DataFrame(tr_history)\n",
        "df.to_csv(history_train)\n",
        "\n",
        "# Save VL performance in csv files for further analysis\n",
        "df = pd.DataFrame(vl_history)\n",
        "df.to_csv(history_val)\n",
        "\n",
        "# Save TS performance in csv files for further analysis\n",
        "df = pd.DataFrame(ts_history)\n",
        "df.to_csv(history_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gQkiJddplveP"
      },
      "source": [
        "Experimental results are further explored and discussed in `MAML_ExperimalResults.ipynb` notebook.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ao8acx8YjQ36"
      },
      "source": [
        "## **References**\n",
        "\n",
        "\n",
        "---\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bWO5wqcDjXKl"
      },
      "source": [
        "\n",
        "\n",
        "*  [Chelsea Finn, Pieter Abbeel, and Sergey Levine. *Model-agnostic meta-learning for fast adaptation of deep networks.* (2017).](https://arxiv.org/pdf/1703.03400.pdf)\n",
        "*   [Antreas Antoniou, Amos Storkey, Harrison Edwards. *How to train your MAML.* (2019).](https://arxiv.org/pdf/1810.09502.pdf)\n",
        "*   [Bronskill, Gordon, Requeima,  Nowozin, Turner. *TaskNorm: Rethinking Batch Normalization for Meta-Learning*. (2020).](https://arxiv.org/pdf/2003.03284.pdf)\n",
        "*[Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra,\n",
        "Timothy Lillicrap. *Meta-Learning with Memory-Augmented Neural Networks.* (2016).](https://proceedings.mlr.press/v48/santoro16.pdf)\n",
        "*   https://github.com/cbfinn/maml\n",
        "*   https://github.com/dragen1860/MAML-Pytorch\n",
        "*   https://pytorch.org/docs/stable/func.html\n",
        "\n",
        "\n",
        "\n"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}