templates/1_set_up_toffy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ec35cb18-c771-48e6-859f-e10e074545d6",
   "metadata": {},
   "source": [
    "# Setting up toffy"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e36293c5-aa89-4029-a3fa-e8ea841bb8b5",
   "metadata": {},
   "source": [
    "There are three parts to this notebook.\n",
    "1. The first part creates the necessary folders that toffy is expecting, and **only needs to be run the first time you install it on a new CAC**. \n",
    "2. The second part updates the co-registration parameters between the slide image (optical image) and the stage coordinates. This **needs to be run when Ionpath changes the co-registration, which usually happens after a column realignment**\n",
    "3. The third part generates a tuning curve to correct for shifts in instrument sensitivity, and **only needs to be run once per instrument**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c65c5bf-c7c1-43a7-882e-4134d9de0007",
   "metadata": {},
   "outputs": [],
   "source": [
    "# imports\n",
    "import sys\n",
    "sys.path.append('../')\n",
    "\n",
    "from datetime import datetime\n",
    "import json\n",
    "import numpy as np\n",
    "import os\n",
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "from toffy import tiling_utils, normalize\n",
    "from tmi import io_utils"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb3f2573-2c06-4212-8abe-7fb41b83c878",
   "metadata": {},
   "source": [
    "## 1. Create necessary folders\n",
    "\n",
    "**If this is the first time toffy has been installed on your CAC, run the following cell to generate the folders necessary.**\n",
    "\n",
    "\n",
    "More information on the uses and locations of the directories in toffy can be found in the [README](https://github.com/angelolab/toffy#directory-structure)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e62bf338-873f-4c3f-a58c-6f908e6bdb24",
   "metadata": {},
   "outputs": [],
   "source": [
    "folders = ['D:\\\\Extracted_Images', 'D:\\\\Rosetta_Compensated_Images', 'D:\\\\Normalized_Images', 'D:\\\\Cohorts',\n",
    "           'C:\\\\Users\\\\Customer.ION\\\\Documents\\\\run_metrics', 'C:\\\\Users\\\\Customer.ION\\\\Documents\\\\watcher_logs',\n",
    "           'C:\\\\Users\\\\Customer.ION\\\\Documents\\\\tiled_run_jsons', \n",
    "           'C:\\\\Users\\\\Customer.ION\\\\Documents\\\\autolabeled_tma_jsons', \n",
    "           'C:\\\\Users\\\\Customer.ION\\\\Documents\\\\panel_files', 'C:\\\\Users\\\\Customer.ION\\\\Documents\\\\normalization_curve', \n",
    "           'C:\\\\Users\\\\Customer.ION\\\\Documents\\\\rosetta_testing',\n",
    "           'C:\\\\Users\\\\Customer.ION\\\\Documents\\\\rosetta_matrices']\n",
    "\n",
    "for folder in folders:\n",
    "    if not os.path.exists(folder):\n",
    "        os.makedirs(folder)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edb13b4a-9589-4c2d-814c-0cae835cc255",
   "metadata": {},
   "source": [
    "## 2. Update coregistration parameters\n",
    "In order to display the location of FOVs on the slide image, we need to map the location of pixels in the image to stage coordinates on the sldie. This notebook allows you to generate this information, which can change over time for a number of reasons. We use the ficudial marks, the dots on each side of the slide, to accomplish this. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44bf4948-eebe-43a9-8e0e-a993dc8e726b",
   "metadata": {},
   "source": [
    "### Set stage and optical coordinates of the fiducials"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d3cc736-faba-41e7-a406-ff900074d7fe",
   "metadata": {},
   "source": [
    "The code below will automatically prompt you to specify the coordinates of the top 3 fiducial pairs in both stage coordinate and corresponding optical coordinate values.\n",
    "\n",
    "For each fiducial:\n",
    "\n",
    "1: Use the pop-out window to open the slide image on the commercial instrument viewer.\n",
    "\n",
    "2: Open the developer console. Right click anywhere on the slide image, this menu should open:\n",
    "\n",
    "<div>\n",
    "    <img src=\"img/inspect_menu.png\" width=\"150\" height=\"137\"/>\n",
    "</div>\n",
    "\n",
    "\n",
    "Click `Inspect`. The developer console should appear on the right side:\n",
    "\n",
    "<div>\n",
    "    <img src=\"img/developer_console.png\" width=\"254\" height=\"240\"/>\n",
    "</div>\n",
    "\n",
    "3: In the console (above the tabs `What's New` and `Console`), type `logger.level=4` and hit `Enter`\n",
    "\n",
    "<div>\n",
    "    <img src=\"img/logger_level.png\" width=\"253\" height=\"76\"/>\n",
    "</div>\n",
    "\n",
    "\n",
    "4: Click on one of the slide image fiducials. Retrieve the stage coordinates on top of the image next to `Target Point`:\n",
    "\n",
    "<div>\n",
    "    <img src=\"img/stage_coords.png\" width=\"164\" height=\"21\"/>\n",
    "</div>\n",
    "\n",
    "\n",
    "\n",
    "Retrieve the corresponding optical coordinates from the console:\n",
    "\n",
    "<div>\n",
    "    <img src=\"img/optical_coords.png\" width=\"254\" height=\"51\"/>\n",
    "</div>\n",
    "\n",
    "\n",
    "\n",
    "5: For each fiducial, we prompt you for the stage x-coordinate, stage y-coordinate, optical x-coordinate, and optical y-coordinate values separately. Enter the correct value for the coordinate and hit `Enter` to continue.\n",
    "\n",
    "6: The co-registration parameter settings will be saved to the `toffy` folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64d6d2f5-ac4c-4917-a874-b8310f75f5e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "fiducial_info = tiling_utils.read_fiducial_info()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce74c91b-86ef-4569-87ac-b8553ecb23bc",
   "metadata": {},
   "source": [
    "### Generate new co-registration parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3db446dd-2575-420b-a00e-874802b7a3ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "coreg_params = tiling_utils.generate_coreg_params(fiducial_info)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7df88849-a7ba-400c-92bd-88e165a5b515",
   "metadata": {},
   "source": [
    "### Save co-registration settings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f126bc6a-4695-4722-9d73-b9dc66bb7195",
   "metadata": {},
   "outputs": [],
   "source": [
    "tiling_utils.save_coreg_params(coreg_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14c82566-d6f5-4096-a249-92fae371ab39",
   "metadata": {},
   "source": [
    "## 3. Generate sensitivity tuning curve\n",
    "The sensitivity of the detector will slowly decay over time, decreasing the counts recorded for the same amount of true signal. The Median Pulse Height (MPH) can be used as metric to evaluate the sensitivity of the detector.\n",
    "\n",
    "We create a tuning curve to relate MPH to the percentage of max signal to understand their relationship.\n",
    "\n",
    "Your curve should look like the image below. It's okay if your values are a bit different, but the shape of the curve should be qualitatively the same. The curve will be saved in the `sweep_path` folder you define below.\n",
    "<figure>\n",
    "    <img src=\"img/tuning_curve.png\" width=\"500\" style=\"display=block; margin:auto\"/>\n",
    "    <figcaption> The x axis is the Detector Gain, y axis is the % of Max Signal </figcaption>\n",
    "</figure>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4170c03-1619-479c-99f0-ea03ac13d76b",
   "metadata": {},
   "source": [
    "### Identify detector sweep\n",
    "The first step is selecting a detector sweep. The goal is for this sweep to cover the range of values most often seen during image acqusition. Therefore, it's best to pick a sweep where the suggested change in voltage following the sweep was less than 50V."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "525de367-672a-416c-9c1a-1cb20a397cda",
   "metadata": {},
   "outputs": [],
   "source": [
    "# pick a name for the sweep, such as the date it was run\n",
    "sweep_name = '20220417_pmma'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e661bba8-6657-4f9f-ba8b-ebb9eab874bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a new folder with the sweep name\n",
    "normalization_dir = 'C:\\\\Users\\\\Customer.ION\\\\Documents\\\\normalization_curve'\n",
    "sweep_path = os.path.join(normalization_dir, sweep_name)\n",
    "if not os.path.exists(sweep_path):\n",
    "    os.makedirs(sweep_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9cab012d-9092-4136-b81b-9c8b3a969e15",
   "metadata": {},
   "source": [
    "Now, copy all of the FOVs from the sweep into the newly created folder, which can be found in *C:\\\\Users\\\\Customer.ION\\\\Documents\\\\normalization_curve*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86941169-09a1-43fe-9299-f9f1ca8766b3",
   "metadata": {},
   "source": [
    "### Create tuning curve\n",
    "We'll then use these FOVs in order to create the curve. **Sometimes FOV channel counts which are too low or too high can interfere with the curve generation, so you can adjust the default range below and run the cell multiple times.** Two tuning curves will be produced, one including all of the data and one excluding values outside of the count range. \n",
    "\n",
    "A barplot will be produced below so you can determine where the channel counts seem to level off; this will indicate what the upper threshold for the count range should be. If you would like to simply use the curve with no values exlcuded, set `count_range=None` and run the cell again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e7d0aa94-a481-4628-b41a-fa2b74d489d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# define masses to use\n",
    "normalize.create_tuning_function(sweep_path=sweep_path, count_range=(0, 3000000))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "toffy38",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.15 (default, Nov 11 2022, 16:55:28) \n[Clang 14.0.0 (clang-1400.0.29.202)]"
  },
  "vscode": {
   "interpreter": {
    "hash": "5a55edcf046d85fe15e2be55257d5f0a71929bdfb61b433d935bb794f8010f3b"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}