kis_drms_access.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Directly accessing SDO data using the DRMS\n",
    "\n",
    "\n",
    "Install `drms` Python module from [PyPI](https://pypi.python.org/pypi/drms):\n",
    "\n",
    "    pip install drms\n",
    "\n",
    "Resources:\n",
    "\n",
    "- HMI Science Nugget: http://hmi.stanford.edu/hminuggets/?p=1757\n",
    "- Documentation: http://drms.readthedocs.io/\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create DRMS client instance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import drms\n",
    "from astropy.io import fits\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "client = drms.Client('kis')\n",
    "client"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## List available series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "client.series('hmi\\.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Accessing metadata #1\n",
    "\n",
    "Querying all metadata for HMI magnetograms from 2011 to 2015 using one magnetogram per day: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "keys = client.query('hmi.m_720s[2011.01.01-2015.01.01@1d]', key=drms.const.all)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "keys"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "keys.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "si = client.info('hmi.m_720s')\n",
    "si.keywords.loc['DSUN_OBS']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#print(keys['T_REC'])\n",
    "t_rec = drms.to_datetime(keys['T_REC'])\n",
    "#print(t_rec)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(2, 1, sharex=True)\n",
    "\n",
    "ax[0].plot(t_rec, keys['DSUN_OBS'])\n",
    "ax[0].set_ylabel('Distance [m]')\n",
    "\n",
    "ax[1].plot(t_rec, keys['CRLT_OBS'])\n",
    "ax[1].set_ylabel('$B_0$ angle [deg]')\n",
    "\n",
    "ax[1].set_xlabel('Date')\n",
    "plt.setp(ax[1].get_xticklabels(), rotation=30)\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Accessing metadata #2\n",
    "\n",
    "Querying all metadata for 5 days of HMI Dopplergrams, separated by 15 minutes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "keys = client.query('hmi.v_45s[2013.02.01/5d@15m]', key=drms.const.all)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "keys.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "t_rec = drms.to_datetime(keys['T_REC'])\n",
    "\n",
    "fig, ax = plt.subplots(3, 1, sharex=True)\n",
    "\n",
    "ax[0].plot(t_rec, keys['DSUN_OBS'])\n",
    "ax[0].set_ylabel('Distance [m]')\n",
    "\n",
    "ax[1].plot(t_rec, keys['OBS_VR'])\n",
    "ax[1].set_ylabel('$v_r$ [m/s]')\n",
    "\n",
    "ax[2].plot(t_rec, keys['OBS_VN'])\n",
    "ax[2].set_ylabel('$v_n$ [m/s]')\n",
    "\n",
    "ax[2].set_xlabel('Date')\n",
    "plt.setp(ax[2].get_xticklabels(), rotation=30)\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Accessing data files\n",
    "\n",
    "The following only works if you have direct access to the KIS NFS, e.g., when running Python on one of our servers.\n",
    "\n",
    "Querying metadata and FITS file locations for HMI Dopplergrams, magnetograms, continuum intensity maps:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "keys_dopp, segs_dopp = client.query('hmi.v_45s[2013.01.01_TAI/1d@1h]', key=drms.const.all, seg=drms.const.all)\n",
    "keys_cont, segs_cont = client.query('hmi.ic_720s[2013.01.01_TAI/1d@1h]', key=drms.const.all, seg=drms.const.all)\n",
    "keys_mag, segs_mag = client.query('hmi.m_720s[2013.01.01_TAI/1d@1h]', key=drms.const.all, seg=drms.const.all)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the record time as table indices:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "keys_dopp.index = segs_dopp.index = drms.to_datetime(keys_dopp['T_REC'])\n",
    "keys_cont.index = segs_cont.index = drms.to_datetime(keys_cont['T_REC'])\n",
    "keys_mag.index = segs_mag.index = drms.to_datetime(keys_mag['T_REC'])\n",
    "segs_dopp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reading FITS files from NFS:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "fname_cont = segs_cont['continuum']['2013-01-01 15:00:00']\n",
    "fname_mag = segs_mag['magnetogram']['2013-01-01 15:00:00']\n",
    "print('Filenames:', fname_cont, fname_mag, sep='\\n')\n",
    "\n",
    "print('Reading continuum intensity map...')\n",
    "cont = fits.getdata(fname_cont)\n",
    "print('Continuum:', cont.shape, cont.dtype)\n",
    "\n",
    "print('Reading magnetogram...')\n",
    "mag = fits.getdata(fname_mag)\n",
    "print('Magnetogram:', mag.shape, mag.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Some simple plots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(2, 1, sharex=True, sharey=True, figsize=(6, 9))\n",
    "\n",
    "im_cont = ax[0].imshow(cont, cmap='gray')\n",
    "fig.colorbar(im_cont, ax=ax[0])\n",
    "\n",
    "im_mag = ax[1].imshow(mag, cmap='gray', vmin=-500, vmax=500)\n",
    "fig.colorbar(im_mag, ax=ax[1])\n",
    "\n",
    "fig.tight_layout()\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "ax[0].set_xlim(1600, 2800)\n",
    "ax[1].set_ylim(700, 1900)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}