notebooks/multi_material-lungman_phantom.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c81ec0d9-e184-4185-8e63-0b9ec47cf274",
   "metadata": {
    "id": "4DrP1n1sjg-e"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/TomographicImaging/gVXR-Tutorials/blob/main/notebooks/multi_material-lungman_phantom.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0fe41a6-8340-40b7-b4a5-a239daab3fb7",
   "metadata": {
    "id": "be4b4b61-7abb-440c-9605-486bce082901"
   },
   "outputs": [],
   "source": [
    "# -*- coding: utf-8 -*-\n",
    "#\n",
    "#  Copyright 2024 United Kingdom Research and Innovation\n",
    "#\n",
    "#  Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "#  you may not use this file except in compliance with the License.\n",
    "#  You may obtain a copy of the License at\n",
    "#\n",
    "#      http://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "#  Unless required by applicable law or agreed to in writing, software\n",
    "#  distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "#  See the License for the specific language governing permissions and\n",
    "#  limitations under the License.\n",
    "#\n",
    "#   Authored by:    Franck Vidal (UKRI-STFC)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ffcfed3-2145-4ba8-8715-17327e58b0d2",
   "metadata": {
    "id": "LitIbS2Yjg-g",
    "tags": []
   },
   "source": [
    "![gVXR](https://github.com/TomographicImaging/gVXR-Tutorials/blob/main/img/Logo-transparent-small.png?raw=1)\n",
    "\n",
    "# Multi-material example: Lungman phantom\n",
    "\n",
    "In this notebook you will use the knowledge learned in [first_xray_simulation.ipynb](first_xray_simulation.ipynb) to set some of the simulation parameters. \n",
    "We aim to **replicate a digital radiograph (DR)** of the [Lungman anthropomorphic chest phantom](https://doi.org/10.1117/1.JMI.5.1.013504) (Kyoto Kagaku, Tokyo, Japan) taken with a clinical X-ray machine by GE Healthcare (Chicago, Illinois, USA) at one of our local hospitals (Glan Clwyd). Parameters relevant to the simulation are extracted from the DICOM file, such as source-patient-distance and source-detector-distance.\n",
    "\n",
    "![Photograph of the Lungman phantom during the digital radiograph acquisition](https://github.com/TomographicImaging/gVXR-Tutorials/blob/main/img/lungman-photo.jpg?raw=1)\n",
    "\n",
    "Additionaly, we will **demonstrate how to use more than one material**. \n",
    "We scanned the [Lungman anthropomorphic chest phantom](https://doi.org/10.1117/1.JMI.5.1.013504) at the same hospital (Ysbyty Gwynedd) using a 128-slice Somatom Definition Edge scanner by Siemens Healthcare (Erlangen, Germany). \n",
    "The CT volume was then segmented into individual structures. \n",
    "Surface meshes were eventually extracted from the segmented CT scan. \n",
    "The corresponding data is available on [Zenodo](https://zenodo.org/records/10782644) (the notebook will download and extract the files automatically). \n",
    "The material property of each anatomical structure is defined as the average Hounsfield unit of that structure. \n",
    "Hounsfield values are then converted into material compositions (mixtures) and densities using [Schneider et al.'s method](https://doi.org/10.1088/0031-9155/45/2/314). This functionality is already built in gVXR.\n",
    "\n",
    "<div class=\"alert alert-block alert-warning\">\n",
    "    <b>Note:</b> Make sure the Python packages are already installed. See <a href=\"../README.md\">README.md</a> in the root directory of the repository. If you are running this notebook from Google Colab, please run the cell below to install the required packages.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1e174bf3-e7a7-4df9-a0ad-75baccc01d4c",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bSYJhBjCjg-h",
    "outputId": "443ab78d-3838-498c-dfe5-addd6f3cf186"
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "IN_COLAB = 'google.colab' in sys.modules\n",
    "\n",
    "if IN_COLAB:\n",
    "    !pip install gvxr SimpleITK k3d"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1961536e-deac-495f-8459-433f36efe3cd",
   "metadata": {
    "id": "77W1AXNQjg-h"
   },
   "source": [
    "## Aims of this session\n",
    "\n",
    "1. Practice, amongst other things, how to set simulation parameters related to the X-ray source and detector;\n",
    "2. Demonstrate how to load several STL files and use them all in the simulation (multi-material).\n",
    "\n",
    "![Screenshot of the 3D environment using K3D](https://github.com/TomographicImaging/gVXR-Tutorials/blob/main/notebooks/output_data/lungman/k3d_screenshot-cropped.png?raw=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea0ad5b2-dab2-4df9-9128-5d1292dc4bb7",
   "metadata": {
    "id": "rxBXwyIbjg-i"
   },
   "source": [
    "## Summary of the simulation parameters\n",
    "\n",
    "| Parameters | Values | Units | Function call |\n",
    "|------------|--------|-------|---------------|\n",
    "| Source to patient distance | 1751 | mm | `gvxr.setSourcePosition(...)` |\n",
    "| Source type (beam shape) | Point source (cone beam) | | `gvxr.usePointSource()` |\n",
    "| Beam spectrum | Monochromatic: 16,000 photons of 90 | keV | `gvxr.setMonoChromatic(...)` |\n",
    "| Source to detector distance | 1800 | mm | `gvxr.setDetectorPosition(...)` |\n",
    "| Detector orientation | [0, 0, -1] |  | `gvxr.setDetectorUpVector(...)` |\n",
    "| Detector resolution | 1871 &times; 1881 | pixels | `gvxr.setDetectorNumberOfPixels(...)` |\n",
    "| Pixel spacing | 0.2 &times; 0.2 | mm | `gvxr.setDetectorPixelSize(...)` |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4dea73c-4c57-4171-93d0-58f003b9b606",
   "metadata": {
    "id": "98anuGe-jg-i"
   },
   "source": [
    "## Import packages\n",
    "\n",
    "- `os` to create the output directory if needed\n",
    "- `matplotlib` to show 2D images\n",
    "- `SimpleITK` to load the DICOM file\n",
    "- `gvxr` to simulate X-ray images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "00443714-17d0-427f-9b11-e073415ae56a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os # Create the output directory if necessary\n",
    "import numpy as np # Who does not use Numpy?\n",
    "\n",
    "import matplotlib # To plot images\n",
    "import matplotlib.pyplot as plt # Plotting\n",
    "from matplotlib.colors import LogNorm # Look up table\n",
    "from matplotlib.colors import PowerNorm # Look up table\n",
    "\n",
    "font = {'family' : 'serif',\n",
    "         'size'   : 15\n",
    "       }\n",
    "matplotlib.rc('font', **font)\n",
    "\n",
    "# Uncomment the line below to use LaTeX fonts\n",
    "# matplotlib.rc('text', usetex=True)\n",
    "\n",
    "import SimpleITK as sitk\n",
    "\n",
    "import urllib.request\n",
    "import progressbar\n",
    "\n",
    "import zipfile\n",
    "import base64\n",
    "\n",
    "from gvxrPython3 import gvxr\n",
    "from gvxrPython3.utils import visualise"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35095b46-e3dd-4c8d-bc16-4bc442c49831",
   "metadata": {},
   "source": [
    "## Getting the data ready"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c5f1f8e-7b10-4f8a-a733-b1fb78d3a9f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "output_path = \"output_data/multi_material-lungman_phantom\"\n",
    "if not os.path.exists(output_path):\n",
    "    os.makedirs(output_path);\n",
    "\n",
    "lungman_path = os.path.join(\"output_data\", \"lungman\");\n",
    "mesh_path = os.path.join(lungman_path, \"MESHES\");\n",
    "if not os.path.exists(mesh_path):\n",
    "    os.makedirs(mesh_path);    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10b6b99a-8e4f-430a-b55b-562806a87401",
   "metadata": {},
   "source": [
    "Download the data from [Zenodo](https://zenodo.org/records/10782644)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0094acd7-bd53-4c8d-9526-41f17a1c3a23",
   "metadata": {},
   "outputs": [],
   "source": [
    "zip_file_url = \"https://zenodo.org/records/10782644/files/lungman_data.zip?download=1\";\n",
    "\n",
    "zip_fname = os.path.join(lungman_path, \"lungman_data.zip\");\n",
    "\n",
    "if not os.path.exists(zip_fname):\n",
    "    pbar = None;\n",
    "    def show_progress(block_num, block_size, total_size):\n",
    "        global pbar\n",
    "        if pbar is None:\n",
    "            pbar = progressbar.ProgressBar(maxval=total_size)\n",
    "            pbar.start()\n",
    "    \n",
    "        downloaded = block_num * block_size\n",
    "        if downloaded < total_size:\n",
    "            pbar.update(downloaded)\n",
    "        else:\n",
    "            pbar.finish()\n",
    "            pbar = None\n",
    "            \n",
    "    print(\"Download the file (%s) from %s\\n\" % (zip_fname, zip_file_url))\n",
    "    urllib.request.urlretrieve(zip_file_url, zip_fname, show_progress)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0607065-f6c8-4be2-a2a8-a25c754bdbcc",
   "metadata": {},
   "source": [
    "Extract the STL files from the ZIP file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92a7f679-0ebf-43fd-86d8-ad85ac10e163",
   "metadata": {},
   "outputs": [],
   "source": [
    "stl_fname_set = [];\n",
    "\n",
    "with zipfile.ZipFile(zip_fname) as z:\n",
    "    for fname in z.namelist():\n",
    "        if \".stl\" in fname:\n",
    "            stl_fname = os.path.join(lungman_path, fname);\n",
    "            stl_fname_set.append(stl_fname);\n",
    "            \n",
    "            if not os.path.exists(stl_fname):\n",
    "                print(\"Extract %s\" % stl_fname);\n",
    "                with open(stl_fname, 'wb') as f:\n",
    "                    f.write(z.read(fname));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be58c000-2b93-415c-b8bf-59b18fa70e3f",
   "metadata": {},
   "source": [
    "Extract the digital radiograph DICOM file from the ZIP file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31d50e3d-3285-4aca-afab-ca7d2b06728c",
   "metadata": {},
   "outputs": [],
   "source": [
    "input_DICOM_fname = \"CD3/DICOM/ST000000/SE000000/DX000000\"\n",
    "output_DICOM_fname = os.path.join(lungman_path, \"DX000000\");\n",
    "\n",
    "with zipfile.ZipFile(zip_fname) as z:\n",
    "    if not os.path.exists(output_DICOM_fname):\n",
    "        print(\"Extract %s\" % input_DICOM_fname);\n",
    "        with open(output_DICOM_fname, 'wb') as f:\n",
    "            f.write(z.read(input_DICOM_fname));"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20ed7e07-0fff-4767-a3da-c4457609ed13",
   "metadata": {},
   "source": [
    "Read the DICOM file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5ba8b953-7262-4473-a5b8-bd7f75ad76f8",
   "metadata": {},
   "outputs": [],
   "source": [
    "reader = sitk.ImageFileReader();\n",
    "reader.SetImageIO(\"GDCMImageIO\");\n",
    "reader.SetFileName(output_DICOM_fname);\n",
    "reader.LoadPrivateTagsOn();\n",
    "reader.ReadImageInformation();    \n",
    "volume = reader.Execute();\n",
    "raw_reference = sitk.GetArrayFromImage(volume)[0];"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22532801-7e38-4996-b41d-abee1508b78a",
   "metadata": {},
   "source": [
    "Extract information useful for the simulation from the DICOM file:\n",
    "\n",
    "- The image size and the physical pixel spacing (i.e. not taking into account the magnification),\n",
    "- The number of pixels,\n",
    "- The distance from the source to the detector, and\n",
    "- The distance from the source to the patient."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "74e1f051-1e0b-4ef9-ab2a-2ab76aeded7f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Extract the information from the DICOM header\n",
    "imager_pixel_spacing = np.array(volume.GetMetaData(\"0018|1164\").split(\"\\\\\")).astype(np.single);\n",
    "detector_element_spacing = np.array(volume.GetMetaData(\"0018|7022\").split(\"\\\\\")).astype(np.single);\n",
    "print(\"Imager Pixel Spacing (in mm): \", imager_pixel_spacing, \"(with magnification)\");\n",
    "print(\"Detector Element Spacing (in mm): \", detector_element_spacing, \"(without magnification)\");\n",
    "\n",
    "# Extract the number of pixels\n",
    "size = volume.GetSize()[0:2]\n",
    "print(\"Image size (in pixels): \", str(size[0]) + \" x \" + str(size[1]))\n",
    "\n",
    "# Extract the information from the DICOM header\n",
    "distance_source_to_detector = float(volume.GetMetaData(\"0018|1110\"))\n",
    "distance_source_to_patient = float(volume.GetMetaData(\"0018|1111\"))\n",
    "\n",
    "print(\"Distance Source to Detector: \", distance_source_to_detector, \"mm\")\n",
    "print(\"Distance Source to Patient: \", distance_source_to_patient, \"mm\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d2c1a6a-9e1b-4d39-ba9d-0aef9fdc4fb0",
   "metadata": {},
   "source": [
    "We also extract the visualisation window to show the image using the 'harder' window."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0e495232-82e0-4b55-a68c-757c4ae776c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "window_centre = int(volume.GetMetaData(\"0028|1050\").split(\"\\\\\")[1]) # Use 0 for normal, 1 for harder, 2 for softer\n",
    "window_width = int(volume.GetMetaData(\"0028|1051\").split(\"\\\\\")[1]) # Use 0 for normal, 1 for harder, 2 for softer\n",
    "\n",
    "print(\"Window Center used: \", window_centre)\n",
    "print(\"Window Width used: \", window_width)\n",
    "\n",
    "vmin = window_centre - window_width / 2\n",
    "vmax = window_centre + window_width / 2\n",
    "\n",
    "view_position = volume.GetMetaData(\"0018|5101\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8486eeda-a7d7-4a7b-b6e3-cbd6f2d6f71b",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure();\n",
    "plt.imshow(raw_reference, cmap=\"gray\", vmin=vmin, vmax=vmax,\n",
    "                             extent=[0,(raw_reference.shape[1]-1)*imager_pixel_spacing[0],0,(raw_reference.shape[0]-1)*imager_pixel_spacing[1]])\n",
    "plt.title(\"Digital radiograph of the Lungman phantom\")\n",
    "plt.xlabel(\"Pixel position\\n(in mm)\")\n",
    "plt.ylabel(\"Pixel position\\n(in mm)\")\n",
    "plt.colorbar()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b724fb07-e8d2-4cfa-aa55-accbe431c6cc",
   "metadata": {
    "id": "cvuPTNdejg-k"
   },
   "source": [
    "## 1. Create an OpenGL context\n",
    "\n",
    "The first step is to create the simulation environment, known here as \"OpenGL context\".\n",
    "`gvxr.createOpenGLContext` will try to find the most suitable environment possible regardless of the operating system. This is an alternative function to `gvxr.createNewContext` used in [test_installation.ipynb](test_installation.ipynb).\n",
    "\n",
    "---\n",
    "## Task:\n",
    "\n",
    "In the cell below, call `gvxr.createOpenGLContext` or `gvxr.createNewContext` to create the simulation environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0f64449d-55fc-4132-a843-d4a1d0394c76",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "O3l2mW4Fjg-k",
    "outputId": "d420b660-1997-460e-ecc0-45677479141d"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6cf2ce8c-97a4-4a11-9b5e-359bd49f88e5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# %load 'snippets/multi_material-lungman_phantom-01.py'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1c7650d-a26e-431d-b16e-dba0e50f9487",
   "metadata": {},
   "source": [
    "We increase the size of the visualisation framebuffer to generate higher resolution screenshots. It does not affect the simulation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "739f08cc-5824-4d4c-b4a2-0c7263310e3f",
   "metadata": {},
   "outputs": [],
   "source": [
    "gvxr.setWindowSize(1000, 1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51cc30fb-5a45-490d-992f-9a10561a9f49",
   "metadata": {
    "id": "TGJJ-VAUjg-k"
   },
   "source": [
    "## 2. Set the X-ray source\n",
    "\n",
    "We must set it's position and beam shape. \n",
    "\n",
    "- The distance from the source to the patient is 1751.0 mm.\n",
    "- As we are trying to replicate an actual acquisition, a cone-beam geometry should be used.\n",
    "\n",
    "---\n",
    "## Task:\n",
    "\n",
    "In the cell below, \n",
    "- set the source position so that the distance from the source to the patient is 1751.0 mm. Consider that the patient position is in [0, 0, 0];\n",
    "- make sure a point source is used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2ae7d25e-8acf-4065-ac5b-526e9f03193e",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "604a51c0-dde6-42f3-91d4-0dfedd79cb9d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# %load 'snippets/multi_material-lungman_phantom-02.py'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d9305d9-35f5-4f89-9656-c25e26c97971",
   "metadata": {
    "id": "faqHVUZDjg-l"
   },
   "source": [
    "## 3. Set the Spectrum\n",
    "\n",
    "We define here the number of photons and their kinetic energy.\n",
    "\n",
    "---\n",
    "## Task:\n",
    "\n",
    "In the cell below, use a monochromatic source that emits 16,000 photons of 90 keV per pixel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2cfca6e8-f695-46ef-84ea-83ceb454b11c",
   "metadata": {
    "id": "tA4L8T8Njg-l"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "22562134-6a2e-452f-aabe-003bb3b21cbc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# %load 'snippets/multi_material-lungman_phantom-03.py'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ffad669-2913-4ffe-84a5-620f45d295a4",
   "metadata": {
    "id": "ijli6jrCjg-l"
   },
   "source": [
    "## 4. Set the Detector\n",
    "\n",
    "A detector is defined by its position, orientation, pixel resolution and the space between the centre of two consecutive pixels along its two axes.\n",
    "\n",
    "---\n",
    "## Task:\n",
    "\n",
    "- Use the detector position.\n",
    "    - distance from the source to the patient is 1751.0 mm,\n",
    "    - distance from the source to the detector is 1800.0 mm,\n",
    "    - the patient position is in [0, 0, 0],\n",
    "    - therefore the distance from the patient to the detector is 1800 minus 1751 mm.\n",
    "- The up vector defining the detector orientation is (0, 0, 1).\n",
    "- There are 1871 &times; 1881 pixels.\n",
    "- The pixel spacing in mm is [0.2, 0.2]."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aef7bdff-7b9b-495a-96b8-200733bf9108",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "ntxrdc-4jg-l",
    "outputId": "f3bad6c0-7b8d-4944-c5ca-591bf103bbbe"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bda92451-38ad-4e2b-b276-7ba2fd9448a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# %load 'snippets/multi_material-lungman_phantom-04.py'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ab23d92-866e-4ecb-b896-277b85dbfdf7",
   "metadata": {
    "id": "kK3gju1xjg-m"
   },
   "source": [
    "## 5. Set the Sample\n",
    "\n",
    "A sample is define by its geometry (surface) and material composition. Note that you can transform (translate, scale and rotate) a sample."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d965accd-4d12-42e6-9c83-1ae458ddf22a",
   "metadata": {},
   "outputs": [],
   "source": [
    "gvxr.removePolygonMeshesFromXRayRenderer();\n",
    "\n",
    "gvxr.emptyMesh(\"lungman\");\n",
    "\n",
    "geometry_set = {\n",
    "    \"bronchioles\": {\"HU\": -419.57144, \"Colour\" : [0, 240, 240, 0.08]},\n",
    "    \"bronchus\": {\"HU\": -40.36795, \"Colour\" : [0, 62, 186, 0.4]},\n",
    "    \"trachea\": {\"HU\": -914.32916, \"Colour\" : [170, 85, 255, 0.4]},\n",
    "    \"diaphram\": {\"HU\": -12.778751, \"Colour\" : [255, 85, 127, 1]},\n",
    "    \"skin\": {\"HU\": -12.121676, \"Colour\" : [125, 125, 125, 0.17]},\n",
    "    \"heart\": {\"HU\": 28.384626, \"Colour\" : [255, 0, 0, 1]},\n",
    "    \"sheets_low\": {\"HU\": -158.2706, \"Colour\" : [193, 193, 193, 1]},\n",
    "    \"sheets_med\": {\"HU\": 203.39578, \"Colour\" : [193, 193, 193, 1]},\n",
    "    \"sheets_high\": {\"HU\": 324.9135, \"Colour\" : [193, 193, 193, 1]},\n",
    "    \"tumours_630HU\": {\"HU\": -658.61346, \"Colour\" : [138, 0, 0, 1]},\n",
    "    \"tumours_100HU\": {\"HU\": 83.32481, \"Colour\" : [255, 85, 0, 1]},\n",
    "    \"spine-hard-650\": {\"HU\": 857.8602, \"Colour\" : [255, 255, 127, 1]},\n",
    "    \"spine-soft-650\": {\"HU\": 375.58865, \"Colour\" : [255, 255, 127, 1]},\n",
    "    \"scaps-hard-550\": {\"HU\": 709.09717, \"Colour\" : [255, 255, 127, 1]},\n",
    "    \"scaps-soft-550\": {\"HU\": 372.82138, \"Colour\" : [255, 255, 127, 1]},\n",
    "    \"sternum-hard-550\": {\"HU\": 789.6037, \"Colour\" : [255, 255, 127, 1]},\n",
    "    \"sternum-soft-550\": {\"HU\": 378.79736, \"Colour\" : [255, 255, 127, 1]},\n",
    "    \"clavicle-hard-700\": {\"HU\": 778.28, \"Colour\" : [255, 255, 127, 1]},\n",
    "    \"clavicle-soft-700\": {\"HU\": 261.89047, \"Colour\" : [255, 255, 127, 1]},\n",
    "}\n",
    "\n",
    "for label in geometry_set:\n",
    "    if \"sheet\" not in label:\n",
    "        gvxr.loadMeshFile(label, os.path.join(mesh_path, label + \".stl\"), \"mm\", True, \"root\");\n",
    "        gvxr.setHounsfieldUnit(label, round(geometry_set[label][\"HU\"]));\n",
    "        gvxr.setColour(label,\n",
    "            geometry_set[label][\"Colour\"][0] / 255.0,\n",
    "            geometry_set[label][\"Colour\"][1] / 255.0,\n",
    "            geometry_set[label][\"Colour\"][2] / 255.0,\n",
    "            geometry_set[label][\"Colour\"][3]);\n",
    "\n",
    "        # We translate the geometry to make sure the detector is not within the patient\n",
    "        gvxr.translateNode(label, 0, -118, 0, \"mm\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "318ccf85-51b9-4df4-9b3c-384c362bf341",
   "metadata": {
    "id": "FWhKJqMBjg-n"
   },
   "source": [
    "## 6. Compute the corresponding X-ray image.\n",
    "\n",
    "It is possible to compute, retrieve and save an X-ray image as well as the path length of X-ray through an object.\n",
    "\n",
    "---\n",
    "## Tasks:\n",
    "\n",
    "- Compute an X-ray image and store it in a local variable called `x_ray_image`\n",
    "- Make sure to convert it into a Numpy array, and\n",
    "- Divide the pixel values by `gvxr.getTotalEnergyWithDetectorResponse()` (mock flat-field correction, this way the pixels values with be in the range [0, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "02e8f49d-86d2-4e4e-9360-9f4f6274bce4",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a780a562-7f2e-4b8e-88a3-f5828907f7fd",
   "metadata": {},
   "outputs": [],
   "source": [
    "# %load 'snippets/multi_material-lungman_phantom-05.py'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79f657ce-5159-4168-9642-c19e47f16ded",
   "metadata": {
    "id": "yMOMh4hTjg-n"
   },
   "source": [
    "## Visualisation of the results\n",
    "\n",
    "Keep in mind that clinical devices implement built-in image post-processing algorithms that change the pixel values. Specifics of these algorithms are unknown. We post-process our simulated X-ray image to display the image in a similar fashion using:\n",
    "\n",
    "$$(-\\log((Img + shift_1) \\times scale_1) + shift_2) \\times scale_2$$\n",
    "\n",
    "Values of $shift_1$, $scale_1$, $shift_2$ and $scale_2$ have been calibrated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6e6614f1-1919-402f-89ea-e1bd9796dc0a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Crop the image\n",
    "y_min_id = 200\n",
    "y_max_id = raw_reference.shape[0] - 250\n",
    "x_min_id = 50\n",
    "x_max_id = raw_reference.shape[1] - 100\n",
    "\n",
    "cropped_x_ray_image = x_ray_image[y_min_id:y_max_id, x_min_id:x_max_id]\n",
    "cropped_raw_reference = raw_reference[y_min_id:y_max_id, x_min_id:x_max_id]\n",
    "\n",
    "# Change the dynamic range to match the one of the experimental data\n",
    "log_epsilon = 1e-9;\n",
    "cropped_x_ray_image += 1.955133958883307132e-04;\n",
    "cropped_x_ray_image *= 9.301220532307965186e+03;\n",
    "cropped_x_ray_image[cropped_x_ray_image < log_epsilon] = log_epsilon;\n",
    "cropped_x_ray_image = -np.log(cropped_x_ray_image);\n",
    "cropped_x_ray_image += 1.451272580374132559e+01;\n",
    "cropped_x_ray_image *= 3.990621875946345654e+00;\n",
    "\n",
    "mean_test = np.mean(cropped_x_ray_image)\n",
    "stddev_test = np.std(cropped_x_ray_image)\n",
    "\n",
    "mean_ref = np.mean(cropped_raw_reference)\n",
    "stddev_ref = np.std(cropped_raw_reference)\n",
    "\n",
    "cropped_x_ray_image -= mean_test\n",
    "cropped_x_ray_image /= stddev_test\n",
    "\n",
    "cropped_x_ray_image *= stddev_ref\n",
    "cropped_x_ray_image += mean_ref"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "665db86a-40a2-499c-9e16-b49de17ee0ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot the two images side-by-side\n",
    "fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(20, 20))\n",
    "\n",
    "im1 = axes.flat[0].imshow(cropped_raw_reference, cmap=\"gray\", vmin=vmin, vmax=vmax,\n",
    "    extent=[0,(cropped_raw_reference.shape[1]-1)*imager_pixel_spacing[0],0,(cropped_raw_reference.shape[0]-1)*imager_pixel_spacing[1]])\n",
    "axes.flat[0].set_title(\"Ground truth\")\n",
    "\n",
    "\n",
    "im2 = axes.flat[1].imshow(cropped_x_ray_image, cmap=\"gray\", vmin=vmin, vmax=vmax,\n",
    "    extent=[0,(cropped_x_ray_image.shape[1]-1)*imager_pixel_spacing[0],0,(cropped_x_ray_image.shape[0]-1)*imager_pixel_spacing[1]])\n",
    "axes.flat[1].set_title(\"gVirtualXRay\")\n",
    "\n",
    "axes.flat[0].set_xlabel(\"Pixel position\\n(in mm)\")\n",
    "axes.flat[1].set_xlabel(\"Pixel position\\n(in mm)\")\n",
    "axes.flat[0].set_ylabel(\"Pixel position\\n(in mm)\")\n",
    "\n",
    "cb_ax = fig.add_axes([0.925, 0.35, 0.02, 0.275])\n",
    "cbar = fig.colorbar(im1, cax=cb_ax)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94a225c6-93ae-4fb2-b6df-37062ba3ab37",
   "metadata": {
    "id": "mbGmmxmyjg-n",
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "It may be useful to visualise the 3D environment to ascertain everything is as expected. It can be done in a window or offscreen with a screenshot.\n",
    "We use both the functionality built in gVXR and K3D"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "69cae938-32a7-45a6-9522-c63efb2ca3f9",
   "metadata": {},
   "outputs": [],
   "source": [
    "def cropImage(img):\n",
    "    image_data_greyscale = np.array(img).max(axis=2)\n",
    "\n",
    "    non_empty_columns = np.where(image_data_greyscale.min(axis=0)<255)[0]\n",
    "    non_empty_rows = np.where(image_data_greyscale.min(axis=1)<255)[0]\n",
    "\n",
    "    # In case no cropping is possible\n",
    "    if non_empty_columns.shape[0] == 0 or non_empty_rows.shape[0] == 0:\n",
    "        return img\n",
    "\n",
    "    cropBox = (min(non_empty_rows), max(non_empty_rows), min(non_empty_columns), max(non_empty_columns))\n",
    "    \n",
    "    return np.array(img)[cropBox[0]:cropBox[1]+1, cropBox[2]:cropBox[3]+1 , :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96072cfd-6191-4d1c-a402-c6626ebb8200",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 833
    },
    "id": "s7Nnf1cijg-n",
    "outputId": "ba135996-0b19-4c46-e123-94bdbbd1a924"
   },
   "outputs": [],
   "source": [
    "# This image can be used in a research paper to illustrate the simulation environment, in which case you may want to change the background colour to white with:\n",
    "gvxr.setWindowBackGroundColour(1.0, 1.0, 1.0);\n",
    "\n",
    "# Update the visualisation window\n",
    "gvxr.displayScene();\n",
    "\n",
    "# Take the screenshot and save it in a file\n",
    "screenshot = cropImage(255 * np.array(gvxr.takeScreenshot())).astype(np.uint8);\n",
    "plt.imsave(os.path.join(output_path, \"screenshot-01.png\"), np.array(screenshot));\n",
    "\n",
    "# or display it using Matplotlib\n",
    "plt.figure(figsize=(10, 10));\n",
    "plt.imshow(screenshot);\n",
    "plt.title(\"Screenshot of the X-ray simulation environment\");\n",
    "plt.axis('off');\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bc1d51d-5ede-41b8-861a-4c99db0d0621",
   "metadata": {},
   "source": [
    "We rotate and zoom out to make a nicer screenshot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12a58ef3-a32f-44d1-afce-fe648c5dd413",
   "metadata": {},
   "outputs": [],
   "source": [
    "gvxr.setZoom(1900)\n",
    "gvxr.setSceneRotationMatrix([\n",
    "    -0.8833954334259033, 0.017649494111537933, 0.46829527616500854, 0.0,\n",
    "    -0.46784770488739014, 0.024374328553676605, -0.883472204208374, 0.0, \n",
    "    -0.02700728178024292, -0.9995452165603638, -0.01327541097998619, 0.0,\n",
    "    0.0, 0.0, 0.0, 1.0]);\n",
    "\n",
    "# Update the visualisation window\n",
    "gvxr.displayScene();\n",
    "\n",
    "# Take the screenshot and save it in a file\n",
    "screenshot = cropImage(255 * np.array(gvxr.takeScreenshot())).astype(np.uint8);\n",
    "plt.imsave(os.path.join(output_path, \"screenshot-02.png\"), np.array(screenshot));\n",
    "\n",
    "# or display it using Matplotlib\n",
    "plt.figure(figsize=(10, 10));\n",
    "plt.imshow(screenshot);\n",
    "plt.title(\"Screenshot of the X-ray simulation environment\");\n",
    "plt.axis('off');\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb4ac50f-1bd7-4ee8-8e73-14afcc150034",
   "metadata": {},
   "source": [
    "We use k3D if possible. It's a nice 3D visualisation framework for Jupyter notebooks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c6958eb-6bb5-449c-bd60-3c810828578b",
   "metadata": {},
   "outputs": [],
   "source": [
    "plot = visualise(use_log=True, use_negative=True, sharpen_ksize=2, sharpen_alpha=1.0);\n",
    "\n",
    "if plot:\n",
    "    plot.display();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba4c6d06-9c35-4b42-a375-23b2c010e58c",
   "metadata": {},
   "outputs": [],
   "source": [
    "if plot:\n",
    "    plot.fetch_screenshot();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee6e2b76-bcd7-4327-b9a3-335d9d2daa2f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# if plot:\n",
    "#     k3d_screenshot = plot.screenshot;\n",
    "#     data = base64.b64decode(k3d_screenshot);\n",
    "#     with open(os.path.join(lungman_path, \"k3d_screenshot.png\"), \"wb\") as fp:\n",
    "#         fp.write(data);\n",
    "#         fp.flush();\n",
    "#         fp.close();"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ce68bc6-9fab-4818-b09f-5159873fae2c",
   "metadata": {
    "id": "1vrbJFWzjg-p"
   },
   "source": [
    "# Cleaning up\n",
    "\n",
    "Once we have finished it is good practice to clean up the OpenGL contexts and windows with the following command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f8713a20-0639-4ebf-be91-34c76d362e1c",
   "metadata": {
    "id": "2ldO2muTjg-p"
   },
   "outputs": [],
   "source": [
    "gvxr.terminate();"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}