![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
"
- ]
- },
- {
- "cell_type": "markdown",
- "source": [
- "# Introduction to TensorFlow, Deep Learning and Transfer Learning (work in progress)\n",
- "\n",
- "* **Project:** Dog Vision 🐶👁 - Using computer vision to classify dog photos into different breeds.\n",
- "* **Goals:** Learn TensorFlow, deep learning and transfer learning.\n",
- "* **Domain:** Computer vision.\n",
- "* **Data:** Images of dogs from [Stanford Dogs Dataset](http://vision.stanford.edu/aditya86/ImageNetDogs/) (120 dog breeds, 20,000+ images).\n",
- "* **Problem type:** Multi-class classification (120 different classes).\n",
- "\n",
- "Welcome, welcome!\n",
- "\n",
- "The focus of this notebook is to give a quick overview of deep learning with TensorFlow.\n",
- "\n",
- "How?\n",
- "\n",
- "We're going to go through the machine learning workflow steps and build a computer vision project to classify photos of dogs into their respective dog breed.\n",
- "\n",
- "TK - image of workflow - e.g. dog photo -> model -> dog breed"
- ],
- "metadata": {
- "id": "0FJFQ_1h0G3s"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "# Quick timestamp\n",
- "import datetime\n",
- "print(f\"Last updated: {datetime.datetime.now()}\")"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "n8IXkgE22aCZ",
- "outputId": "8cf6b243-4e6c-4ba3-d3eb-61b348a97ced"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "Last updated: 2023-11-09 01:52:47.025946\n"
- ]
- }
- ]
- },
- {
- "cell_type": "markdown",
- "source": [
- "## TK - What we're going to cover\n",
- "\n",
- "In this project, we're going to be introduced to the power of deep learning and more specifically, transfer learning using TensorFlow.\n",
- "\n",
- "We'll go through each of these in the context of the [6 step machine learning framework](https://dev.mrdbourke.com/zero-to-mastery-ml/a-6-step-framework-for-approaching-machine-learning-projects/):\n",
- "\n",
- "1. **Problem defintion** - Use computer vision to classify photos of dogs into different dog breeds.\n",
- "2. **Data** - 20,000+ images of dogs from 120 different dog breeds from the [Stanford Dogs dataset](http://vision.stanford.edu/aditya86/ImageNetDogs/).\n",
- "3. **Evaluation** - We'd like to beat the original paper's results (22% mean accuracy across all classes, **tip:** A good way to practice your skills is to find some results online and try to beat them).\n",
- "4. **Features** - Because we're using deep learning, our model will learn the features on its own.\n",
- "5. **Modelling** - We're going to use a pretrained convolutional neural network (CNN) and transfer learning.\n",
- "6. **Experiments** - We'll try different amounts of data with the same model to see the effects on our results.\n",
- "\n",
- "> **Note:** It's okay not to know these exact steps ahead of time. When starting a new project, it's often the case you'll figure it out as you go. These steps are only filled out because I've had practice working on several machine learning projects. You'll pick up these ideas overtime.\n",
- "\n",
- "\n",
- "## TK - Table of contents\n",
- "\n",
- "* Problem type (e.g. multi-class classification)\n",
- "* Domain (e.g. computer vision)\n",
- "* Data type (e.g. unstructured vs structured)"
- ],
- "metadata": {
- "id": "nYCWwASwpPNl"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "# Problem Defintion\n",
- "\n"
- ],
- "metadata": {
- "id": "_D3rcso-4ExT"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "\n",
- "\n",
- "# Data\n"
- ],
- "metadata": {
- "id": "Cet87lHn4O0c"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "\n",
- "# Evaluation\n"
- ],
- "metadata": {
- "id": "Hc3KOtBk4PxR"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "\n",
- "# Features\n"
- ],
- "metadata": {
- "id": "I7NUZ4O54Qq6"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "\n",
- "# Modelling\n"
- ],
- "metadata": {
- "id": "CFSO_k7G4Rsq"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "\n",
- "# Experiments"
- ],
- "metadata": {
- "id": "4MnFw_cO4SfS"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "## TK - Where can can you get help?\n",
- "\n",
- "All of the materials for this course [live on GitHub](https://github.com/mrdbourke/zero-to-mastery-ml/tree/master).\n",
- "\n",
- "If you run into trouble, you can ask a question on the course [GitHub Discussions page](https://github.com/mrdbourke/zero-to-mastery-ml/discussions) there too."
- ],
- "metadata": {
- "id": "BLK0aCjmo8j6"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "## Quick definitions\n",
- "\n",
- "Let's start by breaking down some of the most important topics we're going to go through.\n",
- "\n",
- "### TK - What is TensorFlow?\n",
- "\n",
- "[TensorFlow](https://www.tensorflow.org/) is an open source machine learning and deep learning framework originally developed by Google.\n",
- "\n",
- "### TK - Why use TensorFlow?\n",
- "\n",
- "TensorFlow allows you to manipulate data and write deep learning algorithms using Python code.\n",
- "\n",
- "It also has several built-in capabilities to leverage accelerated computing hardware (e.g. GPUs, Graphics Processing Units and TPUs, Tensor Processing Units).\n",
- "\n",
- "Many of world's largest companies [power their machine learning workloads with TensorFlow](https://www.tensorflow.org/about/case-studies).\n",
- "\n",
- "### TK - What is deep learning?\n",
- "\n",
- "[Deep learning](https://en.wikipedia.org/wiki/Deep_learning) is a form of machine learning where data passes through a series of progressive layers which all contribute to learning an overall representation of that data.\n",
- "\n",
- "The series of progressive layers combines to form what's referred to as a [**neural network**](https://en.wikipedia.org/wiki/Artificial_neural_network).\n",
- "\n",
- "For example, a photo may be turned into numbers and those numbers are then manipulated mathematically through each progressive layer to learn patterns in the photo.\n",
- "\n",
- "The \"deep\" in deep learning comes from the number of layers used in the neural network.\n",
- "\n",
- "So when someone says deep learning or (artificial neural networks), they're typically referring to same thing.\n",
- "\n",
- "### TK - What can deep learning be used for?\n",
- "\n",
- "Deep learning is such a powerful technique that new use cases are being discovered everyday.\n",
- "\n",
- "Most of the modern forms of artifical intelligence (AI) applications you see, are powered by deep learning.\n",
- "\n",
- "[ChatGPT](https://chat.openai.com) uses deep learning to process text and return a response.\n",
- "\n",
- "Tesla's [self-driving cars use deep learning](https://www.tesla.com/AI) to power their computer vision systems.\n",
- "\n",
- "Apple's Photos app uses deep learning to [recognize faces in images](https://machinelearning.apple.com/research/recognizing-people-photos) and create Photo Memories.\n",
- "\n",
- "[Nutrify](https://nutrify.app) (an app my brother and I build) uses deep learning to recognize food in images.\n",
- "\n",
- "TK - image of examples\n",
- "\n",
- "### TK - What is transfer learning?\n",
- "\n",
- "[Transfer learning](https://en.wikipedia.org/wiki/Transfer_learning) is one of the most powerful and useful techniques in modern AI and machine learning.\n",
- "\n",
- "It involves taking what one model (or neural network) has learned in a similar domain and applying to your own.\n",
- "\n",
- "In our case, we're going to use transfer learning to take the patterns a neural network has learned from the 1 million+ images and over 1000 classes in [ImageNet](https://www.image-net.org/) (a gold standard computer vision benchmark) and apply them to our own problem of recognizing dog breeds.\n",
- "\n",
- "The biggest benefit of transfer learning is that it often allows you to get outstanding results with less data and time.\n",
- "\n",
- "TK - Transfer learning workflow - Large data -> Large model -> Patterns -> Custom data -> Custom model"
- ],
- "metadata": {
- "id": "h05v4NwY7EHj"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "## TK - Getting setup\n",
- "\n",
- "This section of the course is taught with Google Colab, an online Jupyter Notebook that provides free access to GPUs (Graphics Processing Units, we'll hear more on these later).\n",
- "\n",
- "For a quick rundown on how to use Google Colab, see their [introductory guide](https://colab.research.google.com/notebooks/basic_features_overview.ipynb) (it's quite similar to a Jupyter Notebook with a few different options).\n",
- "\n",
- "Google Colab also comes with many data science and machine learning libraries, including TensorFlow, pre-installed.\n",
- "\n",
- "### Getting a GPU\n",
- "\n",
- "Before running any code, we'll make sure our Google Colab instance is connected to a GPU.\n",
- "\n",
- "You can do this via going to Runtime -> Change runtime type -> GPU (this may restart your existing runtime).\n",
- "\n",
- "Why use a GPU?\n",
- "\n",
- "Since neural networks perform a large amount of calculations behind the scenes (the main one being matrix multiplication), you need a computer chip that perform these calculations quickly, otherwise you'll be waiting all day for a model to train.\n",
- "\n",
- "And in short, GPUs are much faster at performing matrix multiplications than CPUs.\n",
- "\n",
- "Why this is the case is behind the scope of this project (you can search \"why are GPUs faster than CPUs for machine learning?\" for more).\n",
- "\n",
- "The main thing to remember is: generally, in deep learning, GPUs = faster than CPUs.\n",
- "\n",
- "> **Note:** A good experiment would be to run the neural networks we're going to build later on with and without a GPU and see the difference in their training times.\n",
- "\n",
- "Ok, enough talking, let's start by importing TensorFlow!\n",
- "\n",
- "We'll do so using the common abbreviation `tf`."
- ],
- "metadata": {
- "id": "Mr8xFrmA0qeG"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "# TK - TODO: Check compatibility with Keras 3.0 by installing tf-nightly, see: https://x.com/fchollet/status/1719448117064659352?s=20\n",
- "import tensorflow as tf\n",
- "tf.__version__"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/",
- "height": 35
- },
- "id": "Ao399CMu9888",
- "outputId": "877345b2-ae50-426d-916c-77d28743dc8c"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "execute_result",
- "data": {
- "text/plain": [
- "'2.14.0'"
- ],
- "application/vnd.google.colaboratory.intrinsic+json": {
- "type": "string"
- }
- },
- "metadata": {},
- "execution_count": 2
- }
- ]
- },
- {
- "cell_type": "markdown",
- "source": [
- "Nice!\n",
- "\n",
- "> **Note:** If you want to run TensorFlow locally, you can follow the [TensorFlow installation guide](https://www.tensorflow.org/install).\n",
- "\n",
- "Now let's check to see if TensorFlow has access to a GPU (this isn't 100% required to complete this project but will speed things up dramatically).\n",
- "\n",
- "We can do so with the method [`tf.config.list_physical_devices()`](https://www.tensorflow.org/api_docs/python/tf/config/list_physical_devices)."
- ],
- "metadata": {
- "id": "l4tz_cCL2our"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "# Do we have access to a GPU?\n",
- "device_list = tf.config.list_physical_devices()\n",
- "if \"GPU\" in [device.device_type for device in device_list]:\n",
- " print(f\"[INFO] TensorFlow has GPU available to use. Woohoo!! Computing will be sped up!\")\n",
- " print(f\"[INFO] Accessible devices:\\n{device_list}\")\n",
- "else:\n",
- " print(f\"[INFO] TensorFlow does not have GPU available to use. Models may take a while to train.\")\n",
- " print(f\"[INFO] Accessible devices:\\n{device_list}\")"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "7uNWd2rs2fZW",
- "outputId": "49d51bb7-3339-40d9-d508-763e9446e469"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "[INFO] TensorFlow has GPU available to use. Woohoo!! Computing will be sped up!\n",
- "[INFO] Accessible devices:\n",
- "[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'), PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]\n"
- ]
- }
- ]
- },
- {
- "cell_type": "markdown",
- "source": [
- "# TK - Getting Data\n",
- "\n",
- "There are several options and locations to get data for a deep learning project.\n",
- "\n",
- "| Resource | Description |\n",
- "|----------|-------------|\n",
- "| [Kaggle Datasets](https://www.kaggle.com/datasets) | A collection of datasets across a wide range of topics. |\n",
- "| [TensorFlow Datasets](https://www.tensorflow.org/datasets) | A collection of ready-to-use machine learning datasets ready for use under the `tf.data.Datasets` API. You can see a list of [all available datasets](https://www.tensorflow.org/datasets/catalog/overview#all_datasets) in the TensorFlow documentation. |\n",
- "| [Hugging Face Datasets](https://huggingface.co/datasets) | A continually growing resource of datasets broken into several different kinds of topics. |\n",
- "| [Google Dataset Search](https://datasetsearch.research.google.com/) | A search engine by Google specifically focused on searching online datasets. |\n",
- "| Original sources | Datasets which are made available by researchers or companies with the release of a product or research paper (sources for these will vary, they could be a link on a website or a link to an application form). |\n",
- "| Custom datasets | These are datasets comprised of your own custom source of data. You may build these from scratch on your own or have access to them from an existing product or service. For example, your entire photos library could be your own custom dataset or your entire notes and documents folder or your company's custom order history. |\n",
- "\n",
- "In our case, the dataset we're going to use is called the Stanford Dogs dataset (or ImageNet dogs, as the images are dogs separated from ImageNet).\n",
- "\n",
- "Because the Stanford Dogs dataset has been around for a while (since 2011, which as of writing this in 2023 is like a lifetime in deep learning), it's available from several resources:\n",
- "\n",
- "* The [original project website](http://vision.stanford.edu/aditya86/ImageNetDogs/) via link download\n",
- "* Inside [TensorFlow datasets under `stanford_dogs`](https://www.tensorflow.org/datasets/catalog/stanford_dogs)\n",
- "* On [Kaggle as a downloadable dataset](https://www.kaggle.com/datasets/jessicali9530/stanford-dogs-dataset)\n",
- "\n",
- "The point here is that when you're starting out with practicing deep learning projects, there's no shortage of datasets available.\n",
- "\n",
- "However, when you start wanting to work on your own projects or within a company environment, you'll likely start to work on custom datasets (datasets you build yourself or aren't available publicly online).\n",
- "\n",
- "The main difference between existing datasets and custom datasets is that existing datasets often come preformatted and ready to use.\n",
- "\n",
- "Where as custom datasets often require some preprocessing before they're ready to use within a machine learning project.\n",
- "\n",
- "To practice formatting a dataset for a machine learning problem, we're going to download the Stanford Dogs dataset from the original website.\n",
- "\n",
- "Before we do so, the following code is an example of how we'd get the Stanford Dogs dataset from TensorFlow Datasets."
- ],
- "metadata": {
- "id": "PDvfRmUqx0C5"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "# Download the dataset into train and test split using TensorFlow Datasets\n",
- "# import tensorflow_datasets as tfds\n",
- "# ds_train, ds_test = tfds.load('stanford_dogs', split=['train', 'test'])"
- ],
- "metadata": {
- "id": "b_md5OPJ98nb"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "### TK - Download data directly from Stanford Dogs website\n",
- "\n",
- "Our overall project goal is to build a computer vision model which performs better than the original Stanford Dogs paper (average of 22% accuracy per class across 120 classes).\n",
- "\n",
- "To do so, we need some data.\n",
- "\n",
- "Let's download the original Stanford Dogs dataset from the project website.\n",
- "\n",
- "The data comes in three main files:\n",
- "\n",
- "1. [Images](http://vision.stanford.edu/aditya86/ImageNetDogs/images.tar) (757MB) - `images.tar`\n",
- "3. [Annotations](http://vision.stanford.edu/aditya86/ImageNetDogs/annotation.tar) (21MB) - `annotation.tar`\n",
- "3. [Lists](http://vision.stanford.edu/aditya86/ImageNetDogs/lists.tar), with train/test splits (0.5MB) - `lists.tar`\n",
- "\n",
- "Our goal is to get a file structure like this:\n",
- "\n",
- "```\n",
- "dog_vision_data/\n",
- " images.tar\n",
- " annotation.tar\n",
- " lists.tar\n",
- "```\n",
- "\n",
- "> **Note:** If you're using Google Colab for this project, remember that any data uploaded to the Google Colab session gets deleted if the session disconnects. So to save us redownloading the data every time, we're going to download it once and save it to Google Drive.\n",
- ">\n",
- "> **Resource:** For a good guide on getting data in and out of Google Colab, see the [Google Colab `io.ipynb` tutorial](https://colab.research.google.com/notebooks/io.ipynb#scrollTo=RWSJpsyKqHjH).\n",
- "\n",
- "To make sure we don't have to keep redownloading the data every time we leave and come back to Google Colab, we're going to:\n",
- "\n",
- "1. Download the data if it doesn't already exist on Google Drive.\n",
- "2. Copy it to Google Drive (because Google Colab connects nicely with Google Drive) if it isn't already there.\n",
- "3. If the data already exists on Google Drive (we've been through steps 1 & 2), we'll import it instead.\n",
- "\n",
- "There are two main options to connect Google Colab instances to Google Drive:\n",
- "\n",
- "1. Click \"Mount Drive\" in \"Files\" menu on the left.\n",
- "2. Mount programmatically with `from google.colab import drive` -> `drive.mount('/content/drive')`.\n",
- "\n",
- "More specifically, we're going to follow the following steps:\n",
- "\n",
- "1. Mount Google Drive.\n",
- "2. Setup constants such as our base directory to save files to, the target files we'd like to download and target URL we'd like to download from.\n",
- "3. Setup our target local path to save to.\n",
- "4. Check if the target files all exist in Google Drive and if they do, copy them locally.\n",
- "5. If the target files don't exist in Google Drive, download them from the target URL with the [`!wget`](https://www.gnu.org/software/wget/manual/wget.html) command.\n",
- "6. Create a file on Google Drive to store the download files.\n",
- "7. Copy the downloaded files to Google Drive for use later if needed.\n",
- "\n",
- "A fair few steps, but nothing we can't handle!\n",
- "\n",
- "Plus, this is all good practice for dealing with and manipulating data, a very important skill in the machine learning engineers toolbox."
- ],
- "metadata": {
- "id": "1tHF13s8UbzK"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "from pathlib import Path\n",
- "from google.colab import drive\n",
- "\n",
- "# 1. Mount Google Drive (this will bring up a pop-up to sign-in/authenticate)\n",
- "# Note: This step is specifically for Google Colab, if you're working locally, you may need a different setup\n",
- "drive.mount(\"/content/drive\")\n",
- "\n",
- "# 2. Setup constants\n",
- "# Note: For constants like this, you'll often see them created as variables with all capitals\n",
- "TARGET_DRIVE_PATH = Path(\"drive/MyDrive/tensorflow/dog_vision_data\")\n",
- "TARGET_FILES = [\"images.tar\", \"annotation.tar\", \"lists.tar\"]\n",
- "TARGET_URL = \"http://vision.stanford.edu/aditya86/ImageNetDogs\"\n",
- "\n",
- "# 3. Setup local path\n",
- "local_dir = Path(\"dog_vision_data\")\n",
- "\n",
- "# 4. Check if the target files exist in Google Drive, if so, copy them to Google Colab\n",
- "if all((TARGET_DRIVE_PATH / file).is_file() for file in TARGET_FILES):\n",
- " print(f\"[INFO] Copying Dog Vision files from Google Drive to local directory...\")\n",
- " print(f\"[INFO] Source dir: {TARGET_DRIVE_PATH} -> Target dir: {local_dir}\")\n",
- " !cp -r {TARGET_DRIVE_PATH} .\n",
- " print(\"[INFO] Good to go!\")\n",
- "\n",
- "else:\n",
- " # 5. If the files don't exist in Google Drive, download them\n",
- " print(f\"[INFO] Target files not found in Google Drive.\")\n",
- " print(f\"[INFO] Downloading the target files... this shouldn't take too long...\")\n",
- " for file in TARGET_FILES:\n",
- " # wget is short for \"world wide web get\", as in \"get a file from the web\"\n",
- " # -nc or --no-clobber = don't download files that already exist locally\n",
- " # -P = save the target file to a specified prefix, in our case, local_dir\n",
- " !wget -nc {TARGET_URL}/{file} -P {local_dir} # the \"!\" means to execute the command on the command line rather than in Python\n",
- "\n",
- " print(f\"[INFO] Saving the target files to Google Drive, so they can be loaded later...\")\n",
- "\n",
- " # 6. Ensure target directory in Google Drive exists\n",
- " TARGET_DRIVE_PATH.mkdir(parents=True, exist_ok=True)\n",
- "\n",
- " # 7. Copy downloaded files to Google Drive (so we can use them later and not have to re-download them)\n",
- " !cp -r {local_dir}/* {TARGET_DRIVE_PATH}/"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "ZHOkXNmeDgZq",
- "outputId": "2a431b42-a84a-4870-f10c-0c8e48d1d975"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "Mounted at /content/drive\n",
- "[INFO] Copying Dog Vision files from Google Drive to local directory...\n",
- "[INFO] Source dir: drive/MyDrive/tensorflow/dog_vision_data -> Target dir: dog_vision_data\n",
- "[INFO] Good to go!\n"
- ]
- }
- ]
- },
- {
- "cell_type": "markdown",
- "source": [
- "Data downloaded!\n",
- "\n",
- "Nice work!\n",
- "\n",
- "Now if we get the contents of `local_dir` (`dog_vision_data`), what do we get?\n",
- "\n",
- "We can first make sure it exists with [`Path.exists()`](https://docs.python.org/3/library/pathlib.html#pathlib.Path.exists) and then we can iterate through its contents with [`Path.iterdir()`](https://docs.python.org/3/library/pathlib.html#pathlib.Path.iterdir) and print out the [`.name`](https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.name) attribute of each file."
- ],
- "metadata": {
- "id": "xJULPU7gbAgC"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "if local_dir.exists():\n",
- " print(str(local_dir) + \"/\")\n",
- " for item in local_dir.iterdir():\n",
- " print(\" \", item.name)"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "YRs1gwSLbMma",
- "outputId": "ce6cb83f-b2b2-4d67-ac4f-199796540980"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "stream",
- "name": "stdout",
- "text": [
- "dog_vision_data/\n",
- " annotation.tar\n",
- " images.tar\n",
- " lists.tar\n"
- ]
- }
- ]
- },
- {
- "cell_type": "markdown",
- "source": [
- "Excellent! That's exactly the format we wanted.\n",
- "\n",
- "Now you might've noticed that each file ends in `.tar`.\n",
- "\n",
- "What's this?\n",
- "\n",
- "Searching \"what is .tar?\", I found:\n",
- "\n",
- "> *In computing, tar is a computer software utility for collecting many files into one archive file, often referred to as a tarball, for distribution or backup purposes.*\n",
- ">\n",
- "> Source: [Wikipedia tar page](https://en.wikipedia.org/wiki/Tar_(computing)).\n",
- "\n",
- "Exploring a bit more, I found that the `.tar` format is similar to `.zip`, however, `.zip` offers compression, where as `.tar` mostly combines many files into one.\n",
- "\n",
- "So how do we \"untar\" the files in `images.tar`, `annotation.tar` and `lists.tar`?\n",
- "\n",
- "We can use the [`!tar`](https://www.gnu.org/software/tar/manual/tar.html) command (or just `tar` from outside of a Jupyter Cell)!\n",
- "\n",
- "Doing this will expand all of the files within each of the `.tar` archives.\n",
- "\n",
- "We'll also use a couple of flags to help us out:\n",
- "* The `-x` flag tells `tar` to extract files from an archive.\n",
- "* The `-f` flag specifies that the following argument is the name of the archive file.\n",
- "* You can combine flags by putting them together `-xf`.\n",
- "\n",
- "Let's try it out!"
- ],
- "metadata": {
- "id": "_6Cqw4Qfb3Cq"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "# Untar images\n",
- "# -x = extract files from the zipped file\n",
- "# -v = verbose\n",
- "# -z = decompress files\n",
- "# -f = tell tar which file to deal with\n",
- "!tar -xf dog_vision_data/images.tar\n",
- "!tar -xf dog_vision_data/annotation.tar\n",
- "!tar -xf dog_vision_data/lists.tar"
- ],
- "metadata": {
- "id": "SxChC5P7-5GP"
- },
- "execution_count": null,
- "outputs": []
- },
- {
- "cell_type": "markdown",
- "source": [
- "What new files did we get?\n",
- "\n",
- "We can check in Google Colab by inspecting the \"Files\" tab on the left.\n",
- "\n",
- "Or with Python by using [`os.listdir(\".\")`](https://docs.python.org/3/library/os.html#os.listdir) where `\".\"` means \"the current directory\"."
- ],
- "metadata": {
- "id": "lDpvKKOsjCKU"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "import os\n",
- "\n",
- "os.listdir(\".\") # \".\" stands for \"here\" or \"current directory\""
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "gzcNamgnjEbl",
- "outputId": "79949de2-7761-4f6d-d07a-0d9931077e78"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "execute_result",
- "data": {
- "text/plain": [
- "['.config',\n",
- " 'test_list.mat',\n",
- " 'dog_vision_data',\n",
- " 'Annotation',\n",
- " 'train_list.mat',\n",
- " 'drive',\n",
- " 'Images',\n",
- " 'file_list.mat',\n",
- " 'sample_data']"
- ]
- },
- "metadata": {},
- "execution_count": 8
- }
- ]
- },
- {
- "cell_type": "markdown",
- "source": [
- "Ooooh!\n",
- "\n",
- "Looks like we've got some new files!\n",
- "\n",
- "Specifically:\n",
- "\n",
- "* `train_list.mat` - a list of all the training set images.\n",
- "* `test_list.mat` - a list of all the testing set images.\n",
- "* `Images/` - a folder containing all of the images of dogs.\n",
- "* `Annotation/` - a folder containing all of the annotations for each image.\n",
- "* `file_list.mat` - a list of all the files (training and test list combined).\n",
- "\n",
- "Our next step is to go through them and see what we've got.\n",
- "\n"
- ],
- "metadata": {
- "id": "JMfbxmM7jn-u"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "## Exploring the data\n",
- "\n",
- "Once you've got a dataset, before building a model, it's wise to explore it for a bit to see what kind of data you're working with.\n",
- "\n",
- "* TK - things you should do when you start with a new dataset\n",
- "* visualize\n",
- "* check the distributions (e.g. number of samples per class)\n",
- "\n",
- "TK - daniel bourke tweet about abraham loss function - https://twitter.com/mrdbourke/status/1456087631641473033"
- ],
- "metadata": {
- "id": "3fRbMSbBU67C"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "### Discussing our target data format\n",
- "\n",
- "Since our goal is to build a computer vision model to classify dog breeds, we need a way to tell our model what breed of dog is in what image.\n",
- "\n",
- "A common data format for a classification problem is to have samples stored in folders named after their class name.\n",
- "\n",
- "For example:\n",
- "\n",
- "```\n",
- "images_split/\n",
- "├── train/\n",
- "│ ├── class_1/\n",
- "│ │ ├── train_image1.jpg\n",
- "│ │ ├── train_image2.jpg\n",
- "│ │ └── ...\n",
- "│ ├── class_2/\n",
- "│ │ ├── train_image1.jpg\n",
- "│ │ ├── train_image2.jpg\n",
- "│ │ └── ...\n",
- "└── test/\n",
- " ├── class_1/\n",
- " │ ├── test_image1.jpg\n",
- " │ ├── test_image2.jpg\n",
- " │ └── ...\n",
- " ├── class_2/\n",
- " │ ├── test_image1.jpg\n",
- " │ ├── test_image2.jpg\n",
- "...\n",
- "```\n",
- "\n",
- "In the case of dog images, we'd put all of the images labelled \"chihuahua\" in a folder called `chihuahua/` (and so on for all the other classes and images).\n",
- "\n",
- "We could split these folders so that training images go in `train/chihuahua/` and testing images go in `test/chihuahua/`.\n",
- "\n",
- "This is what we'll be working towards creating.\n",
- "\n",
- "> **Note:** This structure of folder format doesn't just work for only images, it works for text, audio and other kind of classification data too.\n"
- ],
- "metadata": {
- "id": "SBuYrnJZWecW"
- }
- },
- {
- "cell_type": "markdown",
- "source": [
- "\n",
- "### Exploring the file lists\n",
- "\n",
- "How about we check out the `train_list.mat`, `test_list.mat` and `full_list.mat` files?\n",
- "\n",
- "Searching online, for \"what is a .mat file?\", I found that [it's a MATLAB file](https://www.mathworks.com/help/matlab/import_export/mat-file-versions.html). Before Python became the default language for machine learning and deep learning, many models and datasets were built in MATLAB.\n",
- "\n",
- "Then I searched, \"how to open a .mat file with Python?\" and found an [answer on Stack Overflow](https://stackoverflow.com/questions/874461/read-mat-files-in-python) saying I could use the [`scipy` library](https://scipy.org/) (a scientific computing library).\n",
- "\n",
- "The good news is, Google Colab comes with `scipy` preinstalled.\n",
- "\n",
- "We can use the [`scipy.io.loadmat()`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.loadmat.html) method to open a `.mat` file."
- ],
- "metadata": {
- "id": "3VxsBnbEWW8G"
- }
- },
- {
- "cell_type": "code",
- "source": [
- "import scipy\n",
- "\n",
- "# Open lists of train and test .mat\n",
- "train_list = scipy.io.loadmat(\"train_list.mat\")\n",
- "test_list = scipy.io.loadmat(\"test_list.mat\")\n",
- "file_list = scipy.io.loadmat(\"file_list.mat\")\n",
- "\n",
- "# Let's inspect the output and type of the train_list\n",
- "train_list, type(train_list)"
- ],
- "metadata": {
- "colab": {
- "base_uri": "https://localhost:8080/"
- },
- "id": "PmVRq8MTLM5s",
- "outputId": "7094c433-1996-4021-8278-3679329d4a8f"
- },
- "execution_count": null,
- "outputs": [
- {
- "output_type": "execute_result",
- "data": {
- "text/plain": [
- "({'__header__': b'MATLAB 5.0 MAT-file, Platform: GLNXA64, Created on: Sun Oct 9 08:36:13 2011',\n",
- " '__version__': '1.0',\n",
- " '__globals__': [],\n",
- " 'file_list': array([[array(['n02085620-Chihuahua/n02085620_5927.jpg'], dtype='| \n", - " | class_name | \n", - "image_count | \n", - "
|---|---|---|
| 71 | \n", - "n02085936-Maltese_dog | \n", - "252 | \n", - "
| 32 | \n", - "n02088094-Afghan_hound | \n", - "239 | \n", - "
| 91 | \n", - "n02092002-Scottish_deerhound | \n", - "232 | \n", - "
| 117 | \n", - "n02112018-Pomeranian | \n", - "219 | \n", - "
| 35 | \n", - "n02107683-Bernese_mountain_dog | \n", - "218 | \n", - "
| \n", - " | class_name | \n", - "image_count | \n", - "
|---|---|---|
| 71 | \n", - "maltese_dog | \n", - "252 | \n", - "
| 32 | \n", - "afghan_hound | \n", - "239 | \n", - "
| 91 | \n", - "scottish_deerhound | \n", - "232 | \n", - "
| 117 | \n", - "pomeranian | \n", - "219 | \n", - "
| 35 | \n", - "bernese_mountain_dog | \n", - "218 | \n", - "
| \n", - " | image_count | \n", - "
|---|---|
| count | \n", - "120.000000 | \n", - "
| mean | \n", - "171.500000 | \n", - "
| std | \n", - "23.220898 | \n", - "
| min | \n", - "148.000000 | \n", - "
| 25% | \n", - "152.750000 | \n", - "
| 50% | \n", - "159.500000 | \n", - "
| 75% | \n", - "186.250000 | \n", - "
| max | \n", - "252.000000 | \n", - "
| \n", - " | class_name | \n", - "image_count | \n", - "
|---|---|---|
| 33 | \n", - "doberman | \n", - "3 | \n", - "
| 23 | \n", - "pembroke | \n", - "4 | \n", - "
| 61 | \n", - "brabancon_griffon | \n", - "4 | \n", - "
| 64 | \n", - "old_english_sheepdog | \n", - "4 | \n", - "
| 100 | \n", - "scotch_terrier | \n", - "5 | \n", - "
| ... | \n", - "... | \n", - "... | \n", - "
| 40 | \n", - "briard | \n", - "16 | \n", - "
| 88 | \n", - "collie | \n", - "16 | \n", - "
| 11 | \n", - "greater_swiss_mountain_dog | \n", - "17 | \n", - "
| 43 | \n", - "cocker_spaniel | \n", - "18 | \n", - "
| 90 | \n", - "boxer | \n", - "19 | \n", - "
120 rows × 2 columns
\n", - "| \n", - " | test_pred_label | \n", - "test_pred_prob | \n", - "test_pred_class_name | \n", - "test_truth_label | \n", - "test_truth_class_name | \n", - "correct | \n", - "
|---|---|---|---|---|---|---|
| 0 | \n", - "0 | \n", - "0.981926 | \n", - "affenpinscher | \n", - "0 | \n", - "affenpinscher | \n", - "1 | \n", - "
| 1 | \n", - "0 | \n", - "0.747863 | \n", - "affenpinscher | \n", - "0 | \n", - "affenpinscher | \n", - "1 | \n", - "
| 2 | \n", - "0 | \n", - "0.995609 | \n", - "affenpinscher | \n", - "0 | \n", - "affenpinscher | \n", - "1 | \n", - "
| 3 | \n", - "44 | \n", - "0.467855 | \n", - "flat_coated_retriever | \n", - "0 | \n", - "affenpinscher | \n", - "0 | \n", - "
| 4 | \n", - "0 | \n", - "0.997168 | \n", - "affenpinscher | \n", - "0 | \n", - "affenpinscher | \n", - "1 | \n", - "
| ... | \n", - "... | \n", - "... | \n", - "... | \n", - "... | \n", - "... | \n", - "... | \n", - "
| 8575 | \n", - "119 | \n", - "0.785783 | \n", - "yorkshire_terrier | \n", - "119 | \n", - "yorkshire_terrier | \n", - "1 | \n", - "
| 8576 | \n", - "102 | \n", - "0.735301 | \n", - "silky_terrier | \n", - "119 | \n", - "yorkshire_terrier | \n", - "0 | \n", - "
| 8577 | \n", - "102 | \n", - "0.828518 | \n", - "silky_terrier | \n", - "119 | \n", - "yorkshire_terrier | \n", - "0 | \n", - "
| 8578 | \n", - "119 | \n", - "0.940582 | \n", - "yorkshire_terrier | \n", - "119 | \n", - "yorkshire_terrier | \n", - "1 | \n", - "
| 8579 | \n", - "119 | \n", - "0.603093 | \n", - "yorkshire_terrier | \n", - "119 | \n", - "yorkshire_terrier | \n", - "1 | \n", - "
8580 rows × 6 columns
\n", - "| \n", - " | test_truth_class_name | \n", - "correct | \n", - "
|---|---|---|
| 62 | \n", - "keeshond | \n", - "1.000000 | \n", - "
| 10 | \n", - "bedlington_terrier | \n", - "1.000000 | \n", - "
| 30 | \n", - "chow | \n", - "0.989583 | \n", - "
| 92 | \n", - "saint_bernard | \n", - "0.985714 | \n", - "
| 2 | \n", - "african_hunting_dog | \n", - "0.985507 | \n", - "
| ... | \n", - "... | \n", - "... | \n", - "
| 76 | \n", - "miniature_poodle | \n", - "0.600000 | \n", - "
| 5 | \n", - "appenzeller | \n", - "0.588235 | \n", - "
| 104 | \n", - "staffordshire_bullterrier | \n", - "0.581818 | \n", - "
| 16 | \n", - "border_collie | \n", - "0.560000 | \n", - "
| 43 | \n", - "eskimo_dog | \n", - "0.440000 | \n", - "
120 rows × 2 columns
\n", - "| \n", - " | test_pred_label | \n", - "test_pred_prob | \n", - "test_pred_class_name | \n", - "test_truth_label | \n", - "test_truth_class_name | \n", - "correct | \n", - "
|---|---|---|---|---|---|---|
| 2727 | \n", - "75 | \n", - "0.993720 | \n", - "miniature_pinscher | \n", - "38 | \n", - "doberman | \n", - "0 | \n", - "
| 6884 | \n", - "54 | \n", - "0.993490 | \n", - "groenendael | \n", - "95 | \n", - "schipperke | \n", - "0 | \n", - "
| 5480 | \n", - "44 | \n", - "0.990781 | \n", - "flat_coated_retriever | \n", - "78 | \n", - "newfoundland | \n", - "0 | \n", - "
| 7630 | \n", - "4 | \n", - "0.988580 | \n", - "american_staffordshire_terrier | \n", - "104 | \n", - "staffordshire_bullterrier | \n", - "0 | \n", - "
| 4155 | \n", - "55 | \n", - "0.986820 | \n", - "ibizan_hound | \n", - "60 | \n", - "italian_greyhound | \n", - "0 | \n", - "
| ... | \n", - "... | \n", - "... | \n", - "... | \n", - "... | \n", - "... | \n", - "... | \n", - "
| 2644 | \n", - "63 | \n", - "0.882950 | \n", - "kelpie | \n", - "37 | \n", - "dingo | \n", - "0 | \n", - "
| 7934 | \n", - "73 | \n", - "0.882824 | \n", - "maltese_dog | \n", - "109 | \n", - "tibetan_terrier | \n", - "0 | \n", - "
| 1059 | \n", - "14 | \n", - "0.881923 | \n", - "bloodhound | \n", - "12 | \n", - "black_and_tan_coonhound | \n", - "0 | \n", - "
| 2047 | \n", - "86 | \n", - "0.879732 | \n", - "pembroke | \n", - "27 | \n", - "cardigan | \n", - "0 | \n", - "
| 4601 | \n", - "15 | \n", - "0.878603 | \n", - "bluetick | \n", - "67 | \n", - "labrador_retriever | \n", - "0 | \n", - "
100 rows × 6 columns
\n", - "