Super_Fine_Tuned_GPT2_Piano.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Super Fine-Tuned GPT2 Piano",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "machine_shape": "hm",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/asigalov61/Amazing-GPT2-Piano/blob/master/Super_Fine_Tuned_GPT2_Piano.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H7LoMj4GA4n_",
        "colab_type": "text"
      },
      "source": [
        "# Super Fine-Tuned GPT2 Piano\n",
        "\n",
        "##  Finetune SOTA GPT-2 Text-Generating Model to Generate Chamber Music\n",
        "\n",
        "All thanks and credit for this colab go out to [Max Woolf](http://minimaxir.com)\n",
        "\n",
        "*Last updated: November 10th, 2019*\n",
        "\n",
        "Retrain an advanced text generating neural network on any text dataset **for free on a GPU using Collaboratory** using `gpt-2-simple`!\n",
        "\n",
        "For more about `gpt-2-simple`, you can visit [this GitHub repository](https://github.com/minimaxir/gpt-2-simple). You can also read my [blog post](https://minimaxir.com/2019/09/howto-gpt2/) for more information how to use this notebook!\n",
        "\n",
        "\n",
        "To get started:\n",
        "\n",
        "1. Copy this notebook to your Google Drive to keep it and save your changes. (File -> Save a Copy in Drive)\n",
        "2. Make sure you're running the notebook in Google Chrome.\n",
        "3. Run the cells below:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3F0XhRHyJWnF",
        "colab_type": "code",
        "colab": {},
        "cellView": "form"
      },
      "source": [
        "#@title Install all dependencies and packages\n",
        "%tensorflow_version 1.x\n",
        "!pip install -q gpt-2-simple\n",
        "\n",
        "\n",
        "\n",
        "!pip install pyknon\n",
        "!pip install pretty_midi\n",
        "!pip install pypianoroll\n",
        "!pip install mir_eval\n",
        "!apt install fluidsynth #Pip does not work for some reason. Only apt works\n",
        "!pip install midi2audio\n",
        "!cp /usr/share/sounds/sf2/FluidR3_GM.sf2 /content/font.sf2\n",
        "\n",
        "\n",
        "import gpt_2_simple as gpt2\n",
        "from datetime import datetime\n",
        "from google.colab import files\n",
        "\n",
        "!nvidia-smi"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0wXB05bPDYxS",
        "colab_type": "text"
      },
      "source": [
        "## Downloading GPT-2\n",
        "\n",
        "If you're retraining a model on new text, you need to download the GPT-2 model first. \n",
        "\n",
        "There are three released sizes of GPT-2:\n",
        "\n",
        "* `124M` (default): the \"small\" model, 500MB on disk.\n",
        "* `355M`: the \"medium\" model, 1.5GB on disk.\n",
        "* `774M`: the \"large\" model, cannot currently be finetuned with Colaboratory but can be used to generate text from the pretrained model (see later in Notebook)\n",
        "* `1558M`: the \"extra large\", true model. Will not work if a K80 GPU is attached to the notebook. (like `774M`, it cannot be finetuned).\n",
        "\n",
        "Larger models have more knowledge, but take longer to finetune and longer to generate text. You can specify which base model to use by changing `model_name` in the cells below.\n",
        "\n",
        "The next cell downloads it from Google Cloud Storage and saves it in the Colaboratory VM at `/models/<model_name>`.\n",
        "\n",
        "This model isn't permanently saved in the Colaboratory VM; you'll have to redownload it if you want to retrain it at a later time."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "P8wSlgXoDPCR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "gpt2.download_gpt2(model_name=\"774M\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N8KXuKWzQSsN",
        "colab_type": "text"
      },
      "source": [
        "## Mounting Google Drive\n",
        "\n",
        "The best way to get input text to-be-trained into the Colaboratory VM, and to get the trained model *out* of Colaboratory, is to route it through Google Drive *first*.\n",
        "\n",
        "Running this cell (which will only work in Colaboratory) will mount your personal Google Drive in the VM, which later cells can use to get data in/out. (it will ask for an auth code; that auth is not saved anywhere)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "puq4iC6vUAHc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "gpt2.mount_gdrive()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VVijzX0LSXDm",
        "colab_type": "code",
        "colab": {},
        "cellView": "form"
      },
      "source": [
        "#@title (OPTIONAL) Download ready-to-use Piano and Chamber Notewise DataSets\n",
        "%cd /content/\n",
        "!wget 'https://github.com/asigalov61/SuperPiano/raw/master/Super%20Chamber%20Piano%20Violin%20Notewise%20DataSet.zip'\n",
        "!unzip '/content/Super Chamber Piano Violin Notewise DataSet.zip'\n",
        "!rm '/content/Super Chamber Piano Violin Notewise DataSet.zip'\n",
        "\n",
        "!wget 'https://github.com/asigalov61/SuperPiano/raw/master/Super%20Chamber%20Piano%20Only%20Notewise%20DataSet.zip'\n",
        "!unzip '/content/Super Chamber Piano Only Notewise DataSet.zip'\n",
        "!rm '/content/Super Chamber Piano Only Notewise DataSet.zip'"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6OFnPCLADfll",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "file_name = \"/content/notewise_chamber.txt\""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HeeSKtNWUedE",
        "colab_type": "text"
      },
      "source": [
        "If your text file is larger than 10MB, it is recommended to upload that file to Google Drive first, then copy that file from Google Drive to the Colaboratory VM."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-Z6okFD8VKtS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "gpt2.copy_file_from_gdrive(file_name)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LdpZQXknFNY3",
        "colab_type": "text"
      },
      "source": [
        "## Finetune GPT-2\n",
        "\n",
        "The next cell will start the actual finetuning of GPT-2. It creates a persistent TensorFlow session which stores the training config, then runs the training for the specified number of `steps`. (to have the finetuning run indefinitely, set `steps = -1`)\n",
        "\n",
        "The model checkpoints will be saved in `/checkpoint/run1` by default. The checkpoints are saved every 500 steps (can be changed) and when the cell is stopped.\n",
        "\n",
        "The training might time out after 4ish hours; make sure you end training and save the results so you don't lose them!\n",
        "\n",
        "**IMPORTANT NOTE:** If you want to rerun this cell, **restart the VM first** (Runtime -> Restart Runtime). You will need to rerun imports but not recopy files.\n",
        "\n",
        "Other optional-but-helpful parameters for `gpt2.finetune`:\n",
        "\n",
        "\n",
        "*  **`restore_from`**: Set to `fresh` to start training from the base GPT-2, or set to `latest` to restart training from an existing checkpoint.\n",
        "* **`sample_every`**: Number of steps to print example output\n",
        "* **`print_every`**: Number of steps to print training progress.\n",
        "* **`learning_rate`**:  Learning rate for the training. (default `1e-4`, can lower to `1e-5` if you have <1MB input data)\n",
        "*  **`run_name`**: subfolder within `checkpoint` to save the model. This is useful if you want to work with multiple models (will also need to specify  `run_name` when loading the model)\n",
        "* **`overwrite`**: Set to `True` if you want to continue finetuning an existing model (w/ `restore_from='latest'`) without creating duplicate copies. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aeXshJM-Cuaf",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 106
        },
        "outputId": "0685b05b-a59d-4ad3-9b74-84e2a12c1d2f"
      },
      "source": [
        "sess = gpt2.start_tf_sess()\n",
        "\n",
        "gpt2.finetune(sess,\n",
        "              dataset=file_name,\n",
        "              model_name='774M',\n",
        "              steps=1000,\n",
        "              restore_from='fresh',\n",
        "              run_name='run1',\n",
        "              print_every=10,\n",
        "              sample_every=200,\n",
        "              save_every=500\n",
        "              )"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[360 | 410.12] loss=0.64 avg=0.77\n",
            "[370 | 420.20] loss=0.73 avg=0.77\n",
            "[380 | 430.29] loss=0.62 avg=0.76\n",
            "[390 | 440.38] loss=0.71 avg=0.76\n",
            "[400 | 450.47] loss=0.81 avg=0.76\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IXSuTNERaw6K",
        "colab_type": "text"
      },
      "source": [
        "After the model is trained, you can copy the checkpoint folder to your own Google Drive.\n",
        "\n",
        "If you want to download it to your personal computer, it's strongly recommended you copy it there first, then download from Google Drive. The checkpoint folder is copied as a `.rar` compressed file; you can download it and uncompress it locally."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VHdTL8NDbAh3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "gpt2.copy_checkpoint_to_gdrive(run_name='run1')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qQJgV_b4bmzd",
        "colab_type": "text"
      },
      "source": [
        "You're done! Feel free to go to the **Generate Text From The Trained Model** section to generate text based on your retrained model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pel-uBULXO2L",
        "colab_type": "text"
      },
      "source": [
        "## Load a Trained Model Checkpoint\n",
        "\n",
        "Running the next cell will copy the `.rar` checkpoint file from your Google Drive into the Colaboratory VM."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DCcx5u7sbPTD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "gpt2.copy_checkpoint_from_gdrive(run_name='run1')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RTa6zf3e_9gV",
        "colab_type": "text"
      },
      "source": [
        "The next cell will allow you to load the retrained model checkpoint + metadata necessary to generate text.\n",
        "\n",
        "**IMPORTANT NOTE:** If you want to rerun this cell, **restart the VM first** (Runtime -> Restart Runtime). You will need to rerun imports but not recopy files."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-fxL77nvAMAX",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "sess = gpt2.start_tf_sess()\n",
        "gpt2.load_gpt2(sess, run_name='run1')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ClJwpF_ACONp",
        "colab_type": "text"
      },
      "source": [
        "## Generate Text From The Trained Model\n",
        "\n",
        "After you've trained the model or loaded a retrained model from checkpoint, you can now generate text. `generate` generates a single text from the loaded model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4RNY6RBI9LmL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "gpt2.generate(sess, run_name='run1')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oF4-PqF0Fl7R",
        "colab_type": "text"
      },
      "source": [
        "If you're creating an API based on your model and need to pass the generated text elsewhere, you can do `text = gpt2.generate(sess, return_as_list=True)[0]`\n",
        "\n",
        "You can also pass in a `prefix` to the generate function to force the text to start with a given character sequence and generate text from there (good if you add an indicator when the text starts).\n",
        "\n",
        "You can also generate multiple texts at a time by specifing `nsamples`. Unique to GPT-2, you can pass a `batch_size` to generate multiple samples in parallel, giving a massive speedup (in Colaboratory, set a maximum of 20 for `batch_size`).\n",
        "\n",
        "Other optional-but-helpful parameters for `gpt2.generate` and friends:\n",
        "\n",
        "*  **`length`**: Number of tokens to generate (default 1023, the maximum)\n",
        "* **`temperature`**: The higher the temperature, the crazier the text (default 0.7, recommended to keep between 0.7 and 1.0)\n",
        "* **`top_k`**: Limits the generated guesses to the top *k* guesses (default 0 which disables the behavior; if the generated output is super crazy, you may want to set `top_k=40`)\n",
        "* **`top_p`**: Nucleus sampling: limits the generated guesses to a cumulative probability. (gets good results on a dataset with `top_p=0.9`)\n",
        "* **`truncate`**: Truncates the input text until a given sequence, excluding that sequence (e.g. if `truncate='<|endoftext|>'`, the returned text will include everything before the first `<|endoftext|>`). It may be useful to combine this with a smaller `length` if the input texts are short.\n",
        "*  **`include_prefix`**: If using `truncate` and `include_prefix=False`, the specified `prefix` will not be included in the returned text."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_Xryo2f_Irvp",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "gen_file = 'output.txt'\n",
        "\n",
        "gpt2.generate_to_file(sess,\n",
        "                      destination_path=gen_file,\n",
        "                      length=1023,\n",
        "                      temperature=0.6,\n",
        "                      prefix=\"p45 wait6 p54 wait3 p26 wait5 endp45 wait1 p33 wait5 p42 wait13 p47 wait5 endp54\",\n",
        "                      nsamples=1,\n",
        "                      batch_size=1,\n",
        "                      )"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uV-VYBxhIUmN",
        "colab_type": "code",
        "colab": {},
        "cellView": "form"
      },
      "source": [
        "#@title Plot, Graph, and Listen to the Output :)\n",
        "graphs_length_inches = 18 #@param {type:\"slider\", min:0, max:20, step:1}\n",
        "notes_graph_height = 6 #@param {type:\"slider\", min:0, max:20, step:1}\n",
        "highest_displayed_pitch = 92 #@param {type:\"slider\", min:1, max:128, step:1}\n",
        "lowest_displayed_pitch = 24 #@param {type:\"slider\", min:1, max:128, step:1}\n",
        "\n",
        "\n",
        "#@title Generate and Download resulting MIDI file\n",
        "time_coefficient = 1 #@param {type:\"integer\"}\n",
        "\n",
        "import numpy as np\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "\n",
        "from torch import optim\n",
        "import torch.nn.functional as F\n",
        "\n",
        "import keras\n",
        "from keras.utils import to_categorical\n",
        "\n",
        "import time\n",
        "\n",
        "import pretty_midi\n",
        "from midi2audio import FluidSynth\n",
        "from google.colab import output\n",
        "from IPython.display import display, Javascript, HTML, Audio\n",
        "\n",
        "dtype = torch.float\n",
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
        "\n",
        "\n",
        "sample_freq_variable = 12 #@param {type:\"number\"}\n",
        "note_range_variable = 62 #@param {type:\"number\"}\n",
        "note_offset_variable = 33 #@param {type:\"number\"}\n",
        "number_of_instruments = 2 #@param {type:\"number\"}\n",
        "chamber_option = True #@param {type:\"boolean\"}\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "import os\n",
        "import dill as pickle\n",
        "from pathlib import Path\n",
        "import random\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "from math import floor\n",
        "from pyknon.genmidi import Midi\n",
        "from pyknon.music import NoteSeq, Note\n",
        "import music21\n",
        "import random\n",
        "import os, argparse\n",
        "\n",
        "# default settings: sample_freq=12, note_range=62\n",
        "\n",
        "def decoder(filename):\n",
        "    \n",
        "    filedir = '/content/'\n",
        "\n",
        "    notetxt = filedir + filename\n",
        "\n",
        "    with open(notetxt, 'r') as file:\n",
        "        notestring=file.read()\n",
        "\n",
        "    score_note = notestring.split(\" \")\n",
        "\n",
        "    # define some parameters (from encoding script)\n",
        "    sample_freq=sample_freq_variable\n",
        "    note_range=note_range_variable\n",
        "    note_offset=note_offset_variable\n",
        "    chamber=chamber_option\n",
        "    numInstruments=number_of_instruments\n",
        "\n",
        "    # define variables and lists needed for chord decoding\n",
        "    speed=time_coefficient/sample_freq\n",
        "    piano_notes=[]\n",
        "    violin_notes=[]\n",
        "    time_offset=0\n",
        "\n",
        "    # start decoding here\n",
        "    score = score_note\n",
        "\n",
        "    i=0\n",
        "\n",
        "    # for outlier cases, not seen in sonat-1.txt\n",
        "    # not exactly sure what scores would have \"p_octave_\" or \"eoc\" (end of chord?)\n",
        "    # it seems to insert new notes to the score whenever these conditions are met\n",
        "    while i<len(score):\n",
        "        if score[i][:9]==\"p_octave_\":\n",
        "            add_wait=\"\"\n",
        "            if score[i][-3:]==\"eoc\":\n",
        "                add_wait=\"eoc\"\n",
        "                score[i]=score[i][:-3]\n",
        "            this_note=score[i][9:]\n",
        "            score[i]=\"p\"+this_note\n",
        "            score.insert(i+1, \"p\"+str(int(this_note)+12)+add_wait)\n",
        "            i+=1\n",
        "        i+=1\n",
        "\n",
        "\n",
        "    # loop through every event in the score\n",
        "    for i in range(len(score)):\n",
        "\n",
        "        # if the event is a blank, space, \"eos\" or unknown, skip and go to next event\n",
        "        if score[i] in [\"\", \" \", \"<eos>\", \"<unk>\"]:\n",
        "            continue\n",
        "\n",
        "        # if the event starts with 'end' indicating an end of note\n",
        "        elif score[i][:3]==\"end\":\n",
        "\n",
        "            # if the event additionally ends with eoc, increare the time offset by 1\n",
        "            if score[i][-3:]==\"eoc\":\n",
        "                time_offset+=1\n",
        "            continue\n",
        "\n",
        "        # if the event is wait, increase the timestamp by the number after the \"wait\"\n",
        "        elif score[i][:4]==\"wait\":\n",
        "            time_offset+=int(score[i][4:])\n",
        "            continue\n",
        "\n",
        "        # in this block, we are looking for notes   \n",
        "        else:\n",
        "            # Look ahead to see if an end<noteid> was generated\n",
        "            # soon after.  \n",
        "            duration=1\n",
        "            has_end=False\n",
        "            note_string_len = len(score[i])\n",
        "            for j in range(1,200):\n",
        "                if i+j==len(score):\n",
        "                    break\n",
        "                if score[i+j][:4]==\"wait\":\n",
        "                    duration+=int(score[i+j][4:])\n",
        "                if score[i+j][:3+note_string_len]==\"end\"+score[i] or score[i+j][:note_string_len]==score[i]:\n",
        "                    has_end=True\n",
        "                    break\n",
        "                if score[i+j][-3:]==\"eoc\":\n",
        "                    duration+=1\n",
        "\n",
        "            if not has_end:\n",
        "                duration=12\n",
        "\n",
        "            add_wait = 0\n",
        "            if score[i][-3:]==\"eoc\":\n",
        "                score[i]=score[i][:-3]\n",
        "                add_wait = 1\n",
        "\n",
        "            try: \n",
        "                new_note=music21.note.Note(int(score[i][1:])+note_offset)    \n",
        "                new_note.duration = music21.duration.Duration(duration*speed)\n",
        "                new_note.offset=time_offset*speed\n",
        "                if score[i][0]==\"v\":\n",
        "                    violin_notes.append(new_note)\n",
        "                else:\n",
        "                    piano_notes.append(new_note)                \n",
        "            except:\n",
        "                print(\"Unknown note: \" + score[i])\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "            time_offset+=add_wait\n",
        "\n",
        "    # list of all notes for each instrument should be ready at this stage\n",
        "\n",
        "    # creating music21 instrument objects      \n",
        "    \n",
        "    piano=music21.instrument.fromString(\"Piano\")\n",
        "    violin=music21.instrument.fromString(\"Violin\")\n",
        "\n",
        "    # insert instrument object to start (0 index) of notes list\n",
        "    \n",
        "    piano_notes.insert(0, piano)\n",
        "    violin_notes.insert(0, violin)\n",
        "    # create music21 stream object for individual instruments\n",
        "    \n",
        "    piano_stream=music21.stream.Stream(piano_notes)\n",
        "    violin_stream=music21.stream.Stream(violin_notes)\n",
        "    # merge both stream objects into a single stream of 2 instruments\n",
        "    note_stream = music21.stream.Stream([piano_stream, violin_stream])\n",
        "\n",
        "    \n",
        "    note_stream.write('midi', fp=\"/content/\"+filename[:-4]+\".mid\")\n",
        "    print(\"Done! Decoded midi file saved to 'content/'\")\n",
        "\n",
        "    \n",
        "decoder('output.txt')\n",
        "from google.colab import files\n",
        "files.download('/content/output.mid')\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "import librosa\n",
        "import numpy as np\n",
        "import pretty_midi\n",
        "import pypianoroll\n",
        "from pypianoroll import Multitrack, Track\n",
        "import matplotlib\n",
        "import matplotlib.pyplot as plt\n",
        "#matplotlib.use('SVG')\n",
        "# For plotting\n",
        "import mir_eval.display\n",
        "import librosa.display\n",
        "%matplotlib inline\n",
        "\n",
        "\n",
        "midi_data = pretty_midi.PrettyMIDI('/content/output.mid')\n",
        "\n",
        "def plot_piano_roll(pm, start_pitch, end_pitch, fs=100):\n",
        "    # Use librosa's specshow function for displaying the piano roll\n",
        "    librosa.display.specshow(pm.get_piano_roll(fs)[start_pitch:end_pitch],\n",
        "                             hop_length=1, sr=fs, x_axis='time', y_axis='cqt_note',\n",
        "                             fmin=pretty_midi.note_number_to_hz(start_pitch))\n",
        "\n",
        "\n",
        "\n",
        "roll = np.zeros([int(graphs_length_inches), 128])\n",
        "# Plot the output\n",
        "\n",
        "track = Multitrack('/content/output.mid', name='track')\n",
        "plt.figure(figsize=[graphs_length_inches, notes_graph_height])\n",
        "fig, ax = track.plot()\n",
        "fig.set_size_inches(graphs_length_inches, notes_graph_height)\n",
        "plt.figure(figsize=[graphs_length_inches, notes_graph_height])\n",
        "ax2 = plot_piano_roll(midi_data, int(lowest_displayed_pitch), int(highest_displayed_pitch))\n",
        "plt.show(block=False)\n",
        "\n",
        "\n",
        "FluidSynth(\"/content/font.sf2\", 16000).midi_to_audio('/content/output.mid', '/content/output.wav')\n",
        "Audio('/content/output.wav', rate=16000)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wmTXWNUygS5E",
        "colab_type": "text"
      },
      "source": [
        "# LICENSE\n",
        "\n",
        "MIT License\n",
        "\n",
        "Copyright (c) 2019 Max Woolf\n",
        "\n",
        "Permission is hereby granted, free of charge, to any person obtaining a copy\n",
        "of this software and associated documentation files (the \"Software\"), to deal\n",
        "in the Software without restriction, including without limitation the rights\n",
        "to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n",
        "copies of the Software, and to permit persons to whom the Software is\n",
        "furnished to do so, subject to the following conditions:\n",
        "\n",
        "The above copyright notice and this permission notice shall be included in all\n",
        "copies or substantial portions of the Software.\n",
        "\n",
        "THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n",
        "IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n",
        "FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n",
        "AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n",
        "LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n",
        "OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n",
        "SOFTWARE."
      ]
    }
  ]
}