deploy: 65f8115

_sources/assignments/deep-learning/image-segmentation/comparing-edge-based-and-region-based-segmentation.ipynb

@@ -0,0 +1,332 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": [
+     "hide-input"
+    ]
+   },
+   "outputs": [],
+   "source": [
+    "# Install the necessary dependencies\n",
+    "\n",
+    "import os\n",
+    "import sys\n",
+    "!{sys.executable} -m pip install --quiet seaborn pandas scikit-learn numpy matplotlib jupyterlab_myst ipython skimage"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "tags": [
+     "remove-cell"
+    ]
+   },
+   "source": [
+    "---\n",
+    "license:\n",
+    "    code: MIT\n",
+    "    content: CC-BY-4.0\n",
+    "github: https://github.com/ocademy-ai/machine-learning\n",
+    "venue: By Ocademy\n",
+    "open_access: true\n",
+    "bibliography:\n",
+    "  - https://raw.githubusercontent.com/ocademy-ai/machine-learning/main/open-machine-learning-jupyter-book/references.bib\n",
+    "---"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Comparing edge-based and region-based segmentation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In this example, we will try how to segment objects from a background. We use the coins image from skimage.data, which shows several coins outlined against a darker background."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "import matplotlib.pyplot as plt\n",
+    "\n",
+    "from skimage import data\n",
+    "from skimage.exposure import histogram\n",
+    "\n",
+    "coins = data.coins()\n",
+    "hist, hist_centers = histogram(coins)\n",
+    "\n",
+    "fig, axes = plt.subplots(1, 2, figsize=(8, 3))\n",
+    "axes[0].imshow(coins, cmap=plt.cm.gray)\n",
+    "axes[0].axis('off')\n",
+    "axes[1].plot(hist_centers, hist, lw=2)\n",
+    "axes[1].set_title('histogram of gray values')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Thresholding"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "A simple way to segment the coins is to choose a threshold based on the histogram of gray values. Unfortunately, thresholding this image gives a binary image that either misses significant parts of the coins or merges parts of the background with the coins:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "fig, axes = plt.subplots(1, 2, figsize=(8, 3), sharey=True)\n",
+    "\n",
+    "axes[0].imshow(coins > 100, cmap=plt.cm.gray)\n",
+    "axes[0].set_title('coins > 100')\n",
+    "\n",
+    "axes[1].imshow(coins > 150, cmap=plt.cm.gray)\n",
+    "axes[1].set_title('coins > 150')\n",
+    "\n",
+    "for a in axes:\n",
+    "    a.axis('off')\n",
+    "\n",
+    "plt.tight_layout()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Edge-based segmentation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Next, we try to delineate the contours of the coins using edge-based segmentation. To do this, we first get the edges of features using the Canny edge-detector."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from skimage.feature import canny\n",
+    "\n",
+    "edges = canny(coins)\n",
+    "\n",
+    "fig, ax = plt.subplots(figsize=(4, 3))\n",
+    "ax.imshow(edges, cmap=plt.cm.gray)\n",
+    "ax.set_title('Canny detector')\n",
+    "ax.axis('off')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "These contours are then filled using mathematical morphology."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from scipy import ndimage as ndi\n",
+    "\n",
+    "fill_coins = ndi.binary_fill_holes(edges)\n",
+    "\n",
+    "fig, ax = plt.subplots(figsize=(4, 3))\n",
+    "ax.imshow(fill_coins, cmap=plt.cm.gray)\n",
+    "ax.set_title('filling the holes')\n",
+    "ax.axis('off')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Small spurious objects are easily removed by setting a minimum size for valid objects."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from skimage import morphology\n",
+    "\n",
+    "coins_cleaned = morphology.remove_small_objects(fill_coins, 21)\n",
+    "\n",
+    "fig, ax = plt.subplots(figsize=(4, 3))\n",
+    "ax.imshow(coins_cleaned, cmap=plt.cm.gray)\n",
+    "ax.set_title('removing small objects')\n",
+    "ax.axis('off')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "However, this method is not very robust, since contours that are not perfectly closed are not filled correctly, as is the case for one unfilled coin above."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Region-based segmentation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We therefore try a region-based method using the watershed transform. First, we find an elevation map using the Sobel gradient of the image."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from skimage.filters import sobel\n",
+    "\n",
+    "elevation_map = sobel(coins)\n",
+    "\n",
+    "fig, ax = plt.subplots(figsize=(4, 3))\n",
+    "ax.imshow(elevation_map, cmap=plt.cm.gray)\n",
+    "ax.set_title('elevation map')\n",
+    "ax.axis('off')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Next we find markers of the background and the coins based on the extreme parts of the histogram of gray values."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "markers = np.zeros_like(coins)\n",
+    "markers[coins < 30] = 1\n",
+    "markers[coins > 150] = 2\n",
+    "\n",
+    "fig, ax = plt.subplots(figsize=(4, 3))\n",
+    "ax.imshow(markers, cmap=plt.cm.nipy_spectral)\n",
+    "ax.set_title('markers')\n",
+    "ax.axis('off')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Finally, we use the watershed transform to fill regions of the elevation map starting from the markers determined above:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from skimage import segmentation\n",
+    "\n",
+    "segmentation_coins = segmentation.watershed(elevation_map, markers)\n",
+    "\n",
+    "fig, ax = plt.subplots(figsize=(4, 3))\n",
+    "ax.imshow(segmentation_coins, cmap=plt.cm.gray)\n",
+    "ax.set_title('segmentation')\n",
+    "ax.axis('off')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This last method works even better, and the coins can be segmented and labeled individually."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from skimage.color import label2rgb\n",
+    "\n",
+    "segmentation_coins = ndi.binary_fill_holes(segmentation_coins - 1)\n",
+    "labeled_coins, _ = ndi.label(segmentation_coins)\n",
+    "image_label_overlay = label2rgb(labeled_coins, image=coins, bg_label=0)\n",
+    "\n",
+    "fig, axes = plt.subplots(1, 2, figsize=(8, 3), sharey=True)\n",
+    "axes[0].imshow(coins, cmap=plt.cm.gray)\n",
+    "axes[0].contour(segmentation_coins, [0.5], linewidths=1.2, colors='y')\n",
+    "axes[1].imshow(image_label_overlay)\n",
+    "\n",
+    "for a in axes:\n",
+    "    a.axis('off')\n",
+    "\n",
+    "plt.tight_layout()\n",
+    "\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Acknowledgments\n",
+    "\n",
+    "Thanks to [Scikit-image](https://scikit-image.org/) for creating the open-source project [Comparing edge-based and region-based segmentation](https://scikit-image.org/docs/stable/auto_examples/applications/plot_coins_segmentation.html). This inspires the majority of the content in this chapter."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "open-machine-learning-jupyter-book",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.18"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}