GettingStarted/ImportTricks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Import Tricks: Remote Modules, and Importing Notebooks\n",
    "<br>Author(s): **Phil Marshall** ([@drphilmarshall](https://github.com/LSSTScienceCollaborations/StackClub/issues/new?body=@drphilmarshall)) \n",
    "<br>Maintainer(s): **Alex Drlica-Wagner** ([@kadrlica](https://github.com/LSSTScienceCollaborations/StackClub/issues/new?body=@kadrlica)) \n",
    "<br>Level: **Introductory**\n",
    "<br>Last Verified to Run: **2021-09-03**\n",
    "<br>Verified Stack Release: **w_2021_33**\n",
    "\n",
    "### Learning Objectives:\n",
    "\n",
    "After working through this tutorial you should be able to: \n",
    "1. Use the `stackclub.wimport` function to import a python module from the web;\n",
    "2. Import a notebook as a module, following an `import stackclub`\n",
    "3. Understand the current limitations of these utilities\n",
    "\n",
    "### Logistics\n",
    "This notebook is intended to be run at `lsst-lsp-stable.ncsa.illinois.edu` or `data.lsst.cloud` from a local git clone of the [StackClub](https://github.com/LSSTScienceCollaborations/StackClub) repo.\n",
    "\n",
    "### Set-up\n",
    "\n",
    "You can find the Stack version that this notebook is running by using eups list -s on the terminal command line:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Site, host, and stack version\n",
    "! echo $EXTERNAL_INSTANCE_URL\n",
    "! echo $HOSTNAME\n",
    "! eups list -s | grep lsst_distrib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll need the `stackclub` package to be installed. If you are not developing this package, you can install it using `pip`, like this:\n",
    "```\n",
    "pip install git+git://github.com/LSSTScienceCollaborations/StackClub.git#egg=stackclub\n",
    "```\n",
    "If you are developing the `stackclub` package (eg by adding modules to it to support the Stack Club tutorial that you are writing, you'll need to make a local, editable installation. In the top level folder of the `StackClub` repo, do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! cd .. && python setup.py -q develop --user && cd -"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `stackclub` package will have been installed in your `--user` install directory under `$HOME/.local/lib`. If you don't have any user packages previously installed, this directory won't exist and won't be in your system path. To add it, we need to import and reload the `site` module (see [here](https://stackoverflow.com/a/25384923/4075339) for more details). If this doesn't work, try restarting your kernel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import site\n",
    "from importlib import reload\n",
    "reload(site)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When editing the `stackclub` package files, we want the latest version to be imported when we re-run the import command. To enable this, we need the %autoreload magic command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "For this tutorial we'll need the following modules:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import stackclub"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Importing Python Modules from the Web\n",
    "\n",
    "Sometimes we may want to import a python module without installing an entire package - eg. from a GitHub gist. We can do that by first downloading it and then importing it: this is what the `stackclub.wimport` function does."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "# %load -n stackclub.wimport"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For example, suppose the ``stackclub`` library did _not_ include the `where_is` module: we could still download it and import it, like this:    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "where_is_url = \"https://github.com/LSSTScienceCollaborations/StackClub/raw/master/stackclub/where_is.py\"\n",
    "so = stackclub.wimport(where_is_url, vb=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(so)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example, `so` is an imported module - so we can invoke it's functions as normal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from lsst.daf.persistence import Butler\n",
    "so.where_is(Butler.get, in_the='source')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mpl_url = \"https://matplotlib.org/mpl_examples/lines_bars_and_markers/fill_demo_features.py\"\n",
    "mpl = stackclub.wimport(mpl_url, vb=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's another example - a simple python gist:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing Notebooks as Modules\n",
    "\n",
    "Sometimes we will come across Jupyter notebooks that contain functions and classes that we can re-use. Rather than duplicating the code, we can import the other notebook (ie, run it), and then call the function or class as it is. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ability to import notebooks as modules is enabled by the `import stackclub` statement, which sets up a new \"loader\" that can handle Jupyter notebooks. Here's a demo, using the `HelloWorld` notebook that is also in this folder:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import stackclub\n",
    "import HelloWorld as Yo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Yo.take_that_first_baby_step(and_follow_up=\"I'm using code imported from a notebook!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Current Limitations\n",
    "\n",
    "At present, it is not possible to `wimport` a Jupyter notebook. But this would be very useful functionality to have, indeed!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "You should now be able to import and use remote modules with the `stackclub.wimport` function, and import local notebooks (in the current working directory) as modules."
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "LSST",
   "language": "python",
   "name": "lsst"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "livereveal": {
   "scroll": true,
   "start_slideshow_at": "selected"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}