GettingStarted/FindingDocs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to Find DM Stack Documentation\n",
    "\n",
    "<br>Author(s): **Phil Marshall** ([@drphilmarshall](https://github.com/LSSTScienceCollaborations/StackClub/issues/new?body=@drphilmarshall))\n",
    "<br>Maintainer(s): **Alex Drlica-Wagner** ([@kadrlica](https://github.com/LSSTScienceCollaborations/StackClub/issues/new?body=@kadrlica))\n",
    "<br>Level: **Introductory**\n",
    "<br>Last Verified to Run: **2021-09-03**\n",
    "<br>Verified Stack Release: **w_2021_33**\n",
    "\n",
    "### Learning Objectives:\n",
    "\n",
    "In this notebook we will look at a few different ways to find the documentation on a given DM Stack function or class. \n",
    "After working through this tutorial you should be able to: \n",
    "1. Use the jupyter notebook built-in functions to read the docstrings of Stack classes and functions \n",
    "2. Use the `where_is` Stack Club utility to locate DM Stack web documentation.\n",
    "\n",
    "### Logistics\n",
    "This notebook is intended to be run at `lsst-lsp-stable.ncsa.illinois.edu` or `data.lsst.cloud` from a local git clone of the [StackClub](https://github.com/LSSTScienceCollaborations/StackClub) repo.\n",
    "\n",
    "### Set-up"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Site, host, and stack version\n",
    "! echo $EXTERNAL_INSTANCE_URL\n",
    "! echo $HOSTNAME\n",
    "! eups list -s | grep lsst_distrib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll need the `stackclub` package to be installed. If you are not developing this package, and you have permission to write to your base python site-packages, you can install it using `pip`, like this:\n",
    "```\n",
    "pip install git+git://github.com/LSSTScienceCollaborations/StackClub.git#egg=stackclub\n",
    "```\n",
    "If you are developing the `stackclub` package (eg by adding modules to it to support the Stack Club tutorial that you are writing), you'll need to make a local, editable installation, like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! cd .. && python setup.py -q develop --user && cd -"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `stackclub` package will be installed in your user site directory under `$HOME/.local/lib`. If you don't have any user packages previously installed, this directory won't exist and it won't be added to your system path. To add it, we need to import and reload the `site` module (see [here](https://stackoverflow.com/a/25384923/4075339) for more details). If this doesn't work, try restarting your kernel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import site\n",
    "from importlib import reload\n",
    "reload(site)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When editing the `stackclub` package files, we want the latest version to be imported when we re-run the import command. To enable this, we need the %autoreload magic command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inline Notebook Help\n",
    "\n",
    "Command line tasks have usage information - try running them with no arguments, or `--help`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! imageDifference.py --help"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The pipeline task python code also contains useful docstrings, accessible in various ways:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from lsst.pipe.tasks.imageDifference import ImageDifferenceTask"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "help(ImageDifferenceTask)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can follow up on the methods and attributes listed in the `help()` output, with further `help()` commands:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "help(ImageDifferenceTask.getName)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `help()` function mostly prints out the `__doc__` attribute:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(ImageDifferenceTask.getName.__doc__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Jupyter/IPython `?` magic command gives a different, condensed view that may sometimes be helpful:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "? ImageDifferenceTask"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Online Resources: Searchable GitHub-hosted Source Code\n",
    "\n",
    "All the DM code is housed in GitHub repositories in the `lsst` organization.\n",
    "It's nice to provide hyperlinks to the code you are demonstrating, so people can quickly go read the source. We can construct the GitHub URL from the module name, using the `stackclub.where_is` utility."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from stackclub import where_is"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from lsst.pipe.tasks.imageDifference import ImageDifferenceTask\n",
    "where_is(ImageDifferenceTask)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, `where_is` looks for the named object in the source code on GitHub. You can specify this behavior explitly with the `in_the` kwarg:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from lsst.daf.persistence import Butler\n",
    "where_is(Butler.get, in_the='source')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> In case you're interested in what the `where_is` function is doing, paste the following into a python cell: \n",
    "```\n",
    "%load ../stackclub/where_is\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "GitHub search is pretty powerful. Here's an example, using the search string `user:lsst ImageDifferenceTask` and selecting \"Code\" results (in python):\n",
    "\n",
    "https://github.com/search?l=Python&q=user%3Alsst+ImageDifferenceTask&type=Code\n",
    "\n",
    "You can also generate search strings like this one with `where_is`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "where_is(Butler, in_the='repo')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, here's how to generate a search within the LSST DM technotes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "where_is(ImageDifferenceTask, in_the='technotes')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "In this tutorial we have explored two general ways to read more about the DM Stack code objects: the built-in notebook `help` and magic '?' commands, and the `stackclub.where_is`  utility for locating the relevant part of the Stack source code. \n",
    "\n",
    "Both of the above methods focus on the python code, which for many purposes will be sufficient. However, to understand the Stack's C++ primitives, we'll need to dig deeper into the DM Stack's [doxygen documentation](http://doxygen.lsst.codes/stack/doxygen/x_masterDoxyDoc/), as linked from https://pipelines.lsst.io. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "LSST",
   "language": "python",
   "name": "lsst"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}