Network_Bending_Static_Images.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Network Bending-Static Images.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyNk3Y6+386qk0426om8HOsr",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/dvschultz/ml-art-colabs/blob/master/Network_Bending_Static_Images.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mRGVsNV4APak",
        "colab_type": "text"
      },
      "source": [
        "# Network Bending\n",
        "## Manipulate StyleGAN2 models through rotation, translation, etc.\n",
        "\n",
        "[Paper](https://arxiv.org/abs/2005.12420) | [Video](https://youtu.be/IlSMQ2RRTh8) | [GitHub](https://github.com/terrybroad/network-bending)\n",
        "\n",
        "Thanks to [Sid Black](https://twitter.com/realmeatyhuman) for a lot of the code used here to get this up and running on Colab."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9sTjMUOhAdVH",
        "colab_type": "text"
      },
      "source": [
        "## Install Library\n",
        "\n",
        "This code uses the PyTorch version of StyleGAN2. Because of that the install process for this may take a couple minutes."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Pq4gcOTXwzSV",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Install libraries\n",
        "!git clone https://github.com/dvschultz/network-bending\n",
        "!pip install Ninja kmeans-pytorch\n",
        "!apt-get install -y vim make gdb\n",
        "!wget https://download.pytorch.org/libtorch/nightly/cu101/libtorch-shared-with-deps-latest.zip\n",
        "!unzip /content/libtorch-shared-with-deps-latest.zip -d ~/\n",
        "%cd network-bending\n",
        "\n",
        "#install custom transformations\n",
        "!chmod +x /content/network-bending/build_custom_transforms.sh\n",
        "!/content/network-bending/build_custom_transforms.sh /root/libtorch/"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dlKlP0NoB8Zp",
        "colab_type": "text"
      },
      "source": [
        "## Download .pt file\n",
        "\n",
        "As mentioned above, this library uses the PyTorch version of StyleGAN2. If you have a .pkl file, you’ll need to convert it to a .pt file. I have a notebook to do that here."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-ijf6j9Iw8_u",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        },
        "outputId": "05634f93-f4f9-453f-b052-c5e590754b9f"
      },
      "source": [
        "!gdown --id 1huJHdsDlj6x50j_uI1wvsIY8zW6O4lVb"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=1huJHdsDlj6x50j_uI1wvsIY8zW6O4lVb\n",
            "To: /content/network-bending/FreaGAN-10k.pt\n",
            "133MB [00:00, 135MB/s]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kzJ33m8GCdW5",
        "colab_type": "text"
      },
      "source": [
        "## Generate Image Samples\n",
        "\n",
        "First, edit `example_transform_config.yaml` to include various transforms. At the bottom of this file is a sample for each transform possible. I also recommend reviewing the [repo](https://github.com/terrybroad/network-bending) for additional samples.\n",
        "\n",
        "Then run the next command. While there are a nummber of arguments you cna pass into the script here are a few required or helpful ones.\n",
        "\n",
        "*   `--ckpt` pass your `.pt` model file in\n",
        "*   `--config` pass your `example_transform_config.yaml` to this\n",
        "*   `--pics` how many sample images to produce\n",
        "*   `--truncation` defaults to `0.5`, so change if you want something else\n",
        "*   `--size` if you’re not using a 1024 square model you’ll need to include this with the dimension (note: network bending doesn’t work with non-square models)\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YK8tkjZ5OqlW",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "e20e4d68-43fe-4d48-8358-47d6f8292d4a"
      },
      "source": [
        "!python generate.py --ckpt /content/network-bending/FreaGAN-10k.pt --pics 1 --config /content/network-bending/configs/example_transform_config.yaml"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "  0% 0/1 [00:00<?, ?it/s]torch.Size([1, 512])\n",
            "100% 1/1 [00:00<00:00,  1.44it/s]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Wo-dVZt8Dp-O",
        "colab_type": "text"
      },
      "source": [
        "Rather than try to look at them in Colab (which I find tedious) let’s just zip up the file and download it."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0bazmXRa0zhw",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 221
        },
        "outputId": "03c56baa-7dde-495b-ab9d-c975ac2cb1bb"
      },
      "source": [
        "!zip -r samples-freagan-10-strips.zip ./sample/\n",
        "#once completed go to the side panel, right-click on the zip name and download"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "  adding: sample/ (stored 0%)\n",
            "  adding: sample/000004.png (deflated 0%)\n",
            "  adding: sample/000002.png (deflated 0%)\n",
            "  adding: sample/000006.png (deflated 0%)\n",
            "  adding: sample/config.yaml (deflated 13%)\n",
            "  adding: sample/000000.png (deflated 0%)\n",
            "  adding: sample/000008.png (deflated 0%)\n",
            "  adding: sample/000003.png (deflated 0%)\n",
            "  adding: sample/000007.png (deflated 0%)\n",
            "  adding: sample/000009.png (deflated 0%)\n",
            "  adding: sample/000005.png (deflated 0%)\n",
            "  adding: sample/000001.png (deflated 0%)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mQkbRaOuD85t",
        "colab_type": "text"
      },
      "source": [
        "##Generating Sample Strips\n",
        "\n",
        "That was pretty easy, but if you want to see what each transform does to an image it’s gonna be pretty frustrating and tedious. Thankfully there’s a script that will help generate a transform at every layer of the network, and output those images to a single image strip. You may actually want to run this before generating samples, but its helpful to understand the single image generation process first.\n",
        "\n",
        "This time, edit the `/configs/sample_strip_config.yaml` file, then pass it in to the `--config` argument. Most of the same options as above (pics, truncation, etc) are available here as well."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "o7LtrjsMyL-5",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "05bc5700-fe49-4e71-a35e-921d4a69c4c3"
      },
      "source": [
        "!python generate_sample_strips.py --ckpt /content/network-bending/FreaGAN-10k.pt --pics 1 --config /content/network-bending/configs/sample_strip_config.yaml"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "100% 1/1 [00:11<00:00, 11.32s/it]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Rf6sZbOfeSVS",
        "colab_type": "text"
      },
      "source": [
        "## Helper functions"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dYPiAAU6j_i1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# empty sample folder\n",
        "rm -rf /content/network-bending/sample/*.png"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}