Measurement/AsteroidLightCurve.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Asteroid Light Curve\n",
    "\n",
    "<br>Owner(s): **Bryce Kalmbach** ([@jbkalmbach](https://github.com/LSSTScienceCollaborations/StackClubCourse/issues/new?body=@jbkalmbach))\n",
    "<br>Last Verified to Run: **2021-04-04**\n",
    "<br>Verified Stack Release: **v21.0.0**\n",
    "\n",
    "This notebook shows \n",
    "\n",
    "### Learning Objectives:\n",
    "\n",
    "After working through this lesson you should be able to: \n",
    "1. Load difference images and source catalogs using the Butler.\n",
    "2. Use an Exposure Object to get a WCS for an image.\n",
    "3. Use the WCS to find objects in the image based upon their ra, dec.\n",
    "4. Use Astropy to match an object with known astrometry to the source catalog\n",
    "4. Use a photoCalib object to get calibrated photometry from detected sources in the catalog.\n",
    "5. Build a light curve using the DM stack on real data!\n",
    "\n",
    "### Logistics\n",
    "This notebook is intended to be runnable on `lsst-lsp-stable.ncsa.illinois.edu` from a local git clone of https://github.com/LSSTScienceCollaborations/StackClubCourse.\n",
    "\n",
    "\n",
    "#### Further Resources\n",
    "This notebook uses methods from these other Stack Club notebooks:\n",
    "\n",
    "[Low-Surface Brightness Source Detection](https://github.com/LSSTScienceCollaborations/StackClub/blob/master/SourceDetection/LowSurfaceBrightness.ipynb)\n",
    "\n",
    "as well as previous notebooks in the Stack Club Course.\n",
    "\n",
    "#### Data Credit\n",
    "The image data in this notebook is DECam data from the HiTS survey processed by Meredith Rawls ([@mrawls](https://github.com/mrawls)) (original dataset location: `/project/mrawls/hits2015/rerun/cw_2020_04`)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Set-up\n",
    "\n",
    "You can find the Stack version that this notebook is running by using `eups list -s` on the terminal command line:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# What version of the Stack am I using?\n",
    "! echo $HOSTNAME\n",
    "! eups list lsst_distrib -s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will need the following packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import lsst.daf.persistence as dafPersist\n",
    "import lsst.afw.image as afwImage\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import lsst.geom\n",
    "\n",
    "from astropy.coordinates import SkyCoord\n",
    "from astropy import units as u\n",
    "from astropy.time import Time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Enable interactive, in-notebook plotting with matplotlib\n",
    "%matplotlib ipympl\n",
    "\n",
    "# Some other options if the above doesn't work\n",
    "#%matplotlib widget\n",
    "#%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import lsst.afw.display as afw_display\n",
    "afw_display.setDefaultBackend('matplotlib')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Look at difference images\n",
    "\n",
    "Our first task is (not surprisingly) to point the butler at the location of the repository we want to use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_dir = '/project/stack-club/decam_hits_2015_subset/'\n",
    "butler = dafPersist.Butler(data_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We happen to know _a priori_ that there are difference images in this dataset, but what are the actual datasets called when we access them through the butler? We can remind ourselves with `getDatasetTypes`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_types = butler.getDatasetTypes()\n",
    "diff_data_types = [x for x in data_types if x.startswith('deepDiff') ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list(np.sort(list(diff_data_types)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also use `queryMetadata` to see what visits are available."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "butler.queryMetadata('deepDiff_differenceExp', ('visit'), dataId={'filter': 'g'})[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Images and Source Catalogs\n",
    "\n",
    "Now we are ready to load some data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "visit_num = 410929\n",
    "ccd_num = 9"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First let's load the calexp and get the `maskedImage`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "calexp = butler.get('calexp', {'visit': visit_num, 'ccdnum': ccd_num, 'filter': 'g'})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Two ways to get the `maskedImage` out of the `Exposure` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "calexp_im = calexp.getMaskedImage()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Alternate Way\n",
    "calexp_im = calexp.maskedImage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the source catalog (`src`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "calexp_src_cat = butler.get('src', {'visit': visit_num, 'ccdnum': ccd_num, 'filter': 'g'})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the difference image itself"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "diffexp = butler.get('deepDiff_differenceExp', {'visit': visit_num, 'ccdnum': ccd_num, 'filter':'g'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "diffexp_src_cat = butler.get('deepDiff_diaSrc', {'visit': visit_num, 'ccdnum': ccd_num, 'filter': 'g'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "diffexp_im = diffexp.getMaskedImage()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get Visit Info\n",
    "Since we want to get light curves in the end we'll need to learn how to get the time of the visits. Let's use `getInfo` found in the `Exposure` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "exp_visit_info = calexp.getInfo().getVisitInfo()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "visit_date = exp_visit_info.getDate()\n",
    "print(visit_date)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "visit_date_python = exp_visit_info.getDate().toPython()\n",
    "print(visit_date_python)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "visit_date_astropy = Time(visit_date_python)\n",
    "print(visit_date_astropy.mjd)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot `calexp` and `diffExp` side-by-side"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure()#figsize=(16, 14))\n",
    "display = []\n",
    "\n",
    "fig.add_subplot(1,2,1)\n",
    "display.append(afw_display.Display(frame=fig))\n",
    "display[0].scale('linear', 'zscale')\n",
    "display[0].mtv(calexp_im)\n",
    "\n",
    "fig.add_subplot(1,2,2)\n",
    "display.append(afw_display.Display(frame=fig))\n",
    "display[1].scale('linear', 'zscale')\n",
    "#display[1].setMaskTransparency(10)\n",
    "display[1].mtv(diffexp_im)\n",
    "\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What's in the mask?\n",
    "\n",
    "First, we get the mask plane from the difference image. Then we ask what items it contains, and what colors are used to display them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mask = diffexp_im.mask"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mask.getMaskPlaneDict().items()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mask = diffexp.getMask()\n",
    "for mask_name, mask_bit in mask.getMaskPlaneDict().items():\n",
    "    print('{:20}: {}'.format(mask_name, display[1].getMaskPlaneColor(mask_name)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also mouse over the difference image mask and matplotlib will display the mask plane bit."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading and using the Exposure WCS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using an asteroid finding algorithm called [KBMOD](https://github.com/dirac-institute/kbmod) we found Kuiper Belt Objects in the 2015 HiTS data that we are using here. In this folder, we've provided astrometry for the two objects we found in this field. Let's use `2015 DQ249` as an example and build its light curve.\n",
    "\n",
    "The astrometry for these objects is provided in text files easily loaded in a `pandas.DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hits_object_df = pd.read_csv('NotebookData/hits_kbmod_2015_DQ249_coords.dat', delimiter=' ')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hits_object_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's use the information for visit number 410985 and find the asteroid in our difference image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hits_visit = hits_object_df.loc[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the visit data. From our KBMOD search, I already know it is in CCD #9 so we can start with the data id already complete."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_id = {'visit': hits_visit['visit'], 'ccdnum': 9, 'filter':'g'}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "calexp = butler.get('calexp', data_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "calexp_src_cat = butler.get('src', data_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "diffexp = butler.get('deepDiff_differenceExp', data_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "diffexp_src_cat = butler.get('deepDiff_diaSrc', data_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "diffexp_src_df = diffexp_src_cat.asAstropy().to_pandas()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Brief Aside: There are lots of columns in the source catalog including quality flags. Here we show some potentially useful quality flags."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "[x for x in diffexp_src_df.columns if x.startswith('base_PixelFlags')]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Find object with WCS\n",
    "\n",
    "To find the object we will load the image's WCS and use it to convert the ra, dec to pixel location."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "wcs = diffexp.getWcs()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "wcs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use astropy SkyCoords to translate ra, dec into radians."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obj_pos = SkyCoord('%i %i %f %i %i %f' % (hits_visit['ra_hour'],\n",
    "                                          hits_visit['ra_min'],\n",
    "                                          hits_visit['ra_sec'],\n",
    "                                          hits_visit['dec_deg'],\n",
    "                                          hits_visit['dec_min'],\n",
    "                                          hits_visit['dec_sec']),\n",
    "                   unit=(u.hourangle, u.degree))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obj_pos.ra.deg, obj_pos.dec.deg"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use `lsst.geom` package to create `SpherePoint` that describes a position on the sky.\n",
    "\n",
    "`lsst.geom` also has units that we can provide."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obj_pos_lsst = lsst.geom.SpherePoint(obj_pos.ra.deg, obj_pos.dec.deg, lsst.geom.degrees)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can use `skyToPixel` to convert our ra, dec coordinates to pixel coordinates in the image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_pix, y_pix = wcs.skyToPixel(obj_pos_lsst)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also double check that our object is on this CCD by using `getDimensions` and comparing to the pixel location the WCS gives us."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(x_pix, y_pix)\n",
    "print(diffexp.getDimensions())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Draw Postage Stamps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Use `Factory` method to create cutouts\n",
    "\n",
    "One way to create a postage stamp is using the `Factory` method. To use this we need to create a bounding box with `lsst.geom`. \n",
    "\n",
    "The `origin` argument in the call to `Factory` specifies that image pixel origin for our bounding box will be local to the cutout. (For more info on `afwImage.LOCAL` vs `afwImage.PARENT` see [here](https://pipelines.lsst.io/v/d-2018-07-09/modules/lsst.afw.image/indexing-conventions.html).)\n",
    "\n",
    "If `deep` is set then the cutout will copy the data rather than using a reference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_half_width = 40\n",
    "y_half_width = 40"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define bounding box for cutout\n",
    "bbox = lsst.geom.Box2I()\n",
    "bbox.include(lsst.geom.Point2I(x_pix - x_half_width, y_pix - y_half_width))\n",
    "bbox.include(lsst.geom.Point2I(x_pix + x_half_width, y_pix + y_half_width))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Generate cutouts with Factory\n",
    "calexp_cutout = calexp.Factory(calexp, bbox, origin=afwImage.LOCAL, deep=False)\n",
    "diffexp_cutout = diffexp.Factory(diffexp, bbox, origin=afwImage.LOCAL, deep=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is a new and easier way to get a cutout!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "calexp_cutout = calexp.getCutout(obj_pos_lsst, size=lsst.geom.Extent2I(80, 80))\n",
    "diffexp_cutout = diffexp.getCutout(obj_pos_lsst, size=lsst.geom.Extent2I(80, 80))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(10, 5))\n",
    "stamp_display = []\n",
    "\n",
    "fig.add_subplot(1,2,1)\n",
    "stamp_display.append(afw_display.Display(frame=fig))\n",
    "stamp_display[0].scale('linear', 'zscale')\n",
    "stamp_display[0].mtv(calexp_cutout.maskedImage)\n",
    "\n",
    "#stamp_display[0].dot('o', x_pix, y_pix, size=4)\n",
    "for src in calexp_src_cat:\n",
    "    stamp_display[0].dot('o', src.getX(), src.getY(), ctype='cyan', size=4)\n",
    "plt.title('Calexp Image and Source Catalog')\n",
    "    \n",
    "fig.add_subplot(1,2,2)\n",
    "stamp_display.append(afw_display.Display(frame=fig))\n",
    "stamp_display[1].scale('linear', 'zscale')\n",
    "stamp_display[1].mtv(diffexp_cutout.maskedImage)\n",
    "\n",
    "#stamp_display[1].dot('o', x_pix, y_pix, size=4)\n",
    "for src in diffexp_src_cat:\n",
    "    stamp_display[1].dot('o', src.getX(), src.getY(), ctype='cyan', size=4)\n",
    "plt.title('Diffexp Image and Source Catalog')\n",
    "\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise 1\n",
    "\n",
    "In this folder there is astrometry for another asteroid that was found in the same field on the first night of HiTS observations. Load `hits_kbmod_2014_XW40_coords.dat` into a dataframe and try to recreate the plot above for one of the visits. \n",
    "\n",
    "Hint: If you run into an error trying to create a cutout it may be because while the asteroid is in the same field it may not fall on the same CCD. To find the correct CCD use the `wcs.skyToPixel` function and this [map of the DECam focal plane](http://www.ctio.noao.edu/noao/sites/default/files/DECam/DECamOrientation.png)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get Photometry for the Source"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Source matching with Astropy\n",
    "\n",
    "We are going to use Astropy and the `match_to_catalog_sky` method to match our asteroid to the closest source in difference exposure source catalog. We already loaded our asteroid ra, dec into an Astropy `SkyCoord` object above and called it `obj_pos`. A `SkyCoord` object can hold more than one set of coordinates though. So, we will load the coordinates from the source catalog into a `SkyCoord` object called `visit_coords`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "visit_coords = SkyCoord(diffexp_src_cat['coord_ra']*u.rad, diffexp_src_cat['coord_dec']*u.rad)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# obj_pos is only one row: The location of our asteroid in this visit.\n",
    "obj_pos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# visit_coords is many rows: The location of all detected sources in the source catalog\n",
    "visit_coords[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we want to find the closest match to `obj_pos` in `visit_coords` so we use `matchToCatalogSky` with `visit_coords` as the argument. What we get back are the index in `visit_coords` of the closest match and the 2-dimensional separation on the sky to that match. (If we had distance information we could use this to also get a closest 3-dimensional match.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "idx, sep2d, sep3d = obj_pos.match_to_catalog_sky(visit_coords)\n",
    "print(idx, sep2d.arcsec)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get Photometry for our matched source out of the Source Catalog"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the source catalog directly to get instrumental fluxes and errors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obj_instFlux = diffexp_src_cat.getPsfInstFlux()[idx]\n",
    "print(obj_instFlux)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obj_instFlux_err = diffexp_src_cat.getPsfInstFluxErr()[idx]\n",
    "print(obj_instFlux_err)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But what if we want calibrated fluxes and magnitudes along with the errors?\n",
    "\n",
    "Use `photoCalib` product."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "deepDiff_photoCalib = diffexp.getPhotoCalib()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obj_g_flux = deepDiff_photoCalib.instFluxToNanojansky(obj_instFlux, obj_instFlux_err)\n",
    "print(obj_g_flux)\n",
    "\n",
    "# Access flux and error separately\n",
    "print(obj_g_flux.value, obj_g_flux.error)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obj_g_mag = deepDiff_photoCalib.instFluxToMagnitude(obj_instFlux, obj_instFlux_err)\n",
    "print(obj_g_mag)\n",
    "\n",
    "# Access flux and error separately\n",
    "print(obj_g_mag.value, obj_g_mag.error)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a light curve for the asteroid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a function to load in coordinates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def return_obj_skycoord(visit_data):\n",
    "    obj_pos = SkyCoord('%i %i %f %i %i %f' % (visit_data['ra_hour'],\n",
    "                                              visit_data['ra_min'],\n",
    "                                              visit_data['ra_sec'],\n",
    "                                              visit_data['dec_deg'],\n",
    "                                              visit_data['dec_min'],\n",
    "                                              visit_data['dec_sec']),\n",
    "                       unit=(u.hourangle, u.degree))\n",
    "    \n",
    "    return obj_pos"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loop through each visit and gather necessary data\n",
    "\n",
    "To do this we need to load in the `differenceExp` and get the time of the observation. Then we get the source catalog and find the closest match to the known position of our asteroid coordinates. \n",
    "\n",
    "To make sure we only keep good matches in our light curve we set a threshold of 1 arcsec on our matches. If the closest detected object is more than 1 arcsec we will move on to the next visit without a flux measurement.\n",
    "\n",
    "If we do have a good match then we will use the `photoCalib` to get a calibrated flux measurement for our light curve."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "visit_time = []\n",
    "visit_flux = []\n",
    "visit_flux_err = []\n",
    "visit_mag = []\n",
    "\n",
    "for obs_idx in range(len(hits_object_df)):\n",
    "    \n",
    "    # Load data\n",
    "    hits_visit = hits_object_df.iloc[obs_idx]\n",
    "    data_id = {'visit': hits_visit['visit'], 'ccdnum': 9, 'filter':'g'}\n",
    "    diffexp = butler.get('deepDiff_differenceExp', data_id)\n",
    "    diffexp_src_cat = butler.get('deepDiff_diaSrc', data_id)\n",
    "    exp_visit_info = diffexp.getInfo().getVisitInfo()\n",
    "    \n",
    "    # Get Times\n",
    "    visit_date_python = exp_visit_info.getDate().toPython()\n",
    "    visit_date_astropy = Time(visit_date_python)\n",
    "    \n",
    "    # Match to Difference Image Source Catalog\n",
    "    obj_pos = return_obj_skycoord(hits_visit)\n",
    "    visit_coords = SkyCoord(diffexp_src_cat['coord_ra']*u.rad,\n",
    "                            diffexp_src_cat['coord_dec']*u.rad)\n",
    "    match_idx, match_sep2d, _ = obj_pos.match_to_catalog_sky(visit_coords)\n",
    "    \n",
    "    # Only keep matches with 1 arcsecond. Otherwise skip this visit.\n",
    "    if match_sep2d.arcsec > 1.0:\n",
    "        print('No close matches for visit %i. Distance to closest match: %.2f arcsec' % (hits_visit['visit'], match_sep2d.arcsec))\n",
    "        continue\n",
    "    else:\n",
    "        print('Match within %.2f arcsec for visit %i' % (match_sep2d.arcsec, hits_visit['visit']))\n",
    "        \n",
    "    # Load Flux for matched object\n",
    "    visit_time.append(visit_date_astropy.mjd)\n",
    "    inst_flux = diffexp_src_cat.getPsfInstFlux()[match_idx]\n",
    "    inst_flux_err = diffexp_src_cat.getPsfInstFluxErr()[match_idx]\n",
    "    deepDiff_photoCalib = diffexp.getPhotoCalib()\n",
    "    obj_flux = deepDiff_photoCalib.instFluxToNanojansky(inst_flux, inst_flux_err)\n",
    "    visit_flux.append(obj_flux.value)\n",
    "    visit_flux_err.append(obj_flux.error)\n",
    "    \n",
    "    # For Exercise 2\n",
    "    # Load Magnitude for matched object or interesting columns from source catalog or visit information here.\n",
    "\n",
    "visit_time = np.array(visit_time)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure()\n",
    "plt.errorbar(visit_time - visit_time[0], visit_flux, yerr=visit_flux_err, marker='o', lw=1,  elinewidth=2)\n",
    "plt.xlabel('Time from First Observation (Days)')\n",
    "plt.ylabel('Flux (nanojansky)')\n",
    "plt.title('2015 DQ249 Light Curve in HiTS')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And there is our light curve!\n",
    "\n",
    "It seems that a few of our measurements did not have matching sources in the source catalog. KBMOD is designed to find objects that are not likely to be above the standard 5-sigma detection threshold in a single measurement so this is not surprising. An advanced exercise would be to rerun source detection with a lower threshold to try and get measurements for those visits and fill in the light curve!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise 2\n",
    "Now that we have the basic infrastructure in place go back and make some other plots over time. An easy first step would be to make this plot with the magnitude of the source instead of the flux. Other ideas are to plot values from other columns in the source catalog or plot properties of the exposures from the `exp_visit_info` object. Or remake this plot with the astrometry from `2014_XW40`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "LSST",
   "language": "python",
   "name": "lsst"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}