t81_558_class_07_1_img_generative.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZggjUZ5oPvzH"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/jeffheaton/app_deep_learning/blob/main/t81_558_class_07_1_img_generative.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YDTXd8-Lmp8Q"
   },
   "source": [
    "# T81-558: Applications of Deep Neural Networks\n",
    "\n",
    "**Module 7: Image Generative Models**\n",
    "\n",
    "- Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
    "- For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ncNrAEpzmp8S"
   },
   "source": [
    "# Module 7 Material\n",
    "\n",
    "- **Part 7.1 Introduction to Generative AI** [[Video]](https://www.youtube.com/watch?v=2FbkbSnS8sg&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_07_1_img_generative.ipynb)\n",
    "- Part 7.2 Generating Faces with StyleGAN3 [[Video]](https://www.youtube.com/watch?v=VcI2o1yEQa0&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_07_2_gan_intro.ipynb)\n",
    "- Part 7.3 GANS to Enhance Old Photographs Deoldify [[Video]](https://www.youtube.com/watch?v=y7HvjfKsZ50&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_07_3_deoldify.ipynb)\n",
    "- Part 7.4 Text to Images with StableDiffusion [[Video]](https://www.youtube.com/watch?v=gLj6-gJ-lR4&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_07_4_stable_diff.ipynb)\n",
    "- Part 7.5 Finetuning with Dreambooth [[Video]](https://www.youtube.com/watch?v=G_FYFSzkB5Y&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_07_5_dream_booth.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HKQqQljyPvzK"
   },
   "source": [
    "# Google CoLab Instructions\n",
    "\n",
    "The following code ensures that Google CoLab is running the correct version of TensorFlow.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fU9UhAxTmp8S",
    "outputId": "2ac151e9-fbbe-4325-e166-380063f57018"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: using Google CoLab\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    import google.colab\n",
    "    COLAB = True\n",
    "    print(\"Note: using Google CoLab\")\n",
    "except:\n",
    "    print(\"Note: not using Google CoLab\")\n",
    "    COLAB = False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "i-qb-mcqmp8U"
   },
   "source": [
    "# Part 7.1: Introduction to Generative AI\n",
    "\n",
    "Generative AI is a fascinating and rapidly evolving subfield of artificial intelligence that focuses on creating new content, from art, music, and stories to more technical applications such as generating code or predicting complex systems. This is in contrast to other types of AI that are more concerned with analyzing and interpreting existing data, such as classifying images or predicting stock prices. While traditional AI systems are built to respond with the most accurate or optimal answer from a set of possibilities, generative AI systems are designed to create new possibilities, generating new data that is similar in some way to the data they were trained on.\n",
    "\n",
    "For example, a generative AI system might be trained on a large dataset of classical music and then be able to create new compositions in the style of Bach or Beethoven. Another application could be training the model on a vast amount of text data to create a chatbot that can generate human-like responses to questions or prompts, as we've seen with OpenAI's GPT models. Other examples include generating realistic images, creating virtual environments, or even designing molecules for new drugs.\n",
    "\n",
    "This remarkable capability of generative AI opens up a world of possibilities and applications that are only beginning to be explored. In this section, we will delve deeper into the workings of generative AI, examining the technologies that power it, such as Generative Adversarial Networks (GANs) and Large Language Models (LLMs), and explore some of the most exciting applications and future directions of this innovative field.\n",
    "\n",
    "## The Beginnings of Generative AI\n",
    "\n",
    "The history of generative AI can be traced back to the early days of artificial intelligence itself. In the 1950s and 60s, researchers were already experimenting with algorithms that could generate new content, such as the ELIZA chatbot developed at MIT in the 1966, which could generate responses to user input in a manner that mimicked a psychotherapist. However, the true potential of generative AI remained largely untapped until the advent of more powerful computing hardware and larger datasets in the 21st century. While ELIZA seemed advanced for the time; its conversations lacked any significant depth:\n",
    "\n",
    "```\n",
    "> Hello, I am Eliza. \n",
    "* I am afraid I will never learn Python.\n",
    "> Do you believe it is normal to be afraid you will never learn Python? \n",
    "* Yes\n",
    "> I see.\n",
    "```\n",
    "\n",
    "In 2006, Geoffrey Hinton and his colleagues at the University of Toronto developed a type of neural network called a Deep Belief Network (DBN), which could be trained to generate new data that was similar to the data it was trained on. This marked a key milestone in the development of generative AI, as it demonstrated that neural networks could be used to not only classify and analyze data, but also to create new data. Initially, this DBN was trained to recognize MNIST digits, such as seen in Figure 7.MNIST.\n",
    "\n",
    "**Figure 7.MNIST: Sample of the MNIST Digits**\n",
    "\n",
    "![MNIST Digits](https://data.heatonresearch.com/images/wustl/class/class_8_mnist.png)\n",
    "\n",
    "The next major breakthrough came in 2014, with the introduction of Generative Adversarial Networks (GANs) by Ian Goodfellow and his colleagues. GANs consist of two neural networks, a generator and a discriminator, which are trained simultaneously through a kind of game. The generator tries to create data that is indistinguishable from real data, while the discriminator tries to tell if the data is real or fake. This adversarial training process results in the generator becoming increasingly adept at creating realistic data, and has been used to generate everything from realistic images of faces that do not exist to new video game levels. \n",
    "\n",
    "Figure 7.GAN-MNIST shows a GAN that generates new digits based on MNIST digits that it was trained on. \n",
    "\n",
    "**Figure 7.GAN-MNIST: Generated Digits**\n",
    "\n",
    "![MNIST Digits](https://data.heatonresearch.com/images/wustl/class/gan_digits.jpg)\n",
    "\n",
    "The same early technique was applied to create entirely new faces, as demonstrated by Figure 7.\n",
    "\n",
    "**Figure 7.GAN-FACE: Generated Faces**\n",
    "\n",
    "![Early GAN Faces](https://s3.amazonaws.com/data.heatonresearch.com/images/wustl/class/gan_face.jpg)\n",
    "\n",
    "NVIDIA took GAN generated images to an entirely new level with StyleGAN, which could generate photo-realistic faces as seen in figure 7.GAN-STYLE.\n",
    "\n",
    "**Figure 7.GAN-STYLE: Generated Faces**\n",
    "\n",
    "![StyleGAN](https://data.heatonresearch.com/images/wustl/class/stylegan2-hires.jpg)\n",
    "\n",
    "Stable Diffusion is a recent development in the field of generative AI that has garnered significant attention. Traditional methods like GANs and VAEs have their drawbacks, such as mode collapse in GANs. Stable Diffusion, on the other hand, uses a different approach by adopting a diffusion-based probabilistic model. This approach involves transforming the data in a way that spreads or 'diffuses' it out, and then running the process in reverse to generate new data. The model is trained by predicting the next diffusion step in the reverse process from a given state, which allows it to learn a detailed and high-quality representation of the data. This results in the generation of more realistic and detailed images compared to other methods. Moreover, Stable Diffusion models have been found to be more stable and easier to train compared to GANs, making them a promising alternative for generating high-quality content in various applications.\n",
    "\n",
    "With stable diffusion you can use prompts to render the image, giving you great control. Figure 7.GAN-STYLE shows the same woman in multiple poses and settings.\n",
    "\n",
    "**Figure 7.GAN-STYLE: Same Person in Stable Diffusion**\n",
    "\n",
    "![Stable Diffusion](https://data.heatonresearch.com/images/wustl/class/stable-diff.jpg)\n",
    "\n",
    "\n",
    "\n",
    "In recent years, the development of large language models (LLMs) such as OpenAI's GPT (Generative Pre-trained Transformer) series, has brought generative AI to the forefront of public attention. These models are trained on vast amounts of text data and can generate human-like text on a wide variety of topics. This has led to a plethora of applications, from chatbots and virtual assistants to creative writing and content generation.\n",
    "\n",
    "The development of generative AI is still ongoing, and there are many challenges to be addressed and exciting avenues to explore. However, the progress that has been made in recent years is nothing short of astounding, and has opened up a world of possibilities that were once the stuff of science fiction.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0eBtaFbimp-M"
   },
   "source": [
    "# Module 7 Assignment\n",
    "\n",
    "You can find the seventh assignment here: [assignment 7](https://github.com/jeffheaton/app_deep_learning/blob/main/assignments/assignment_yourname_t81_558_class7.ipynb)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "colab": {
   "collapsed_sections": [],
   "name": "t81_558_class_06_1_python_images.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3.11 (torch)",
   "language": "python",
   "name": "pytorch"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}