Cython_demo_notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Cython Demo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import timeit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# There are two packages, one containing regular Python modules and\n",
    "# the other containing corresponding Cython modules"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cythonization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Let's create a C extension from the `hello` module\n",
    "! rm -f awesome_cython_stuff/hello.c awesome_cython_stuff/hello*.so awesome_cython_stuff/hello.html\n",
    "! ls awesome_cython_stuff/hello*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# The Cython module contains code that is unmodified Python code (in this case)\n",
    "! more awesome_cython_stuff/hello.pyx"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## C Extensions\n",
    "To generate the C extension, there are a couple ways:\n",
    "1. Run `cython` manually to generate the `.c` file and then run `gcc` to generate the `.so`:\n",
    "  ```\n",
    "  cython -a awesome_cython_stuff/hello.pyx\n",
    "  ```\n",
    "  ```\n",
    "  gcc -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing -I${PYTHON_HEADER_DIR} -o awesome_cython_stuff/hello.so awesome_cython_stuff/hello.c\n",
    "  ```\n",
    "\n",
    "2. Run `setup.py` (using `distutils.extension.Extension` and `Cython.Distutils.build_ext` or `Cython.Build.cythonize`, etc. to generate the extensions):\n",
    "   ```\n",
    "   python setup.py install\n",
    "   ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Example setup.py\n",
    "! head -25 setup.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# After generating the C extension, there will be a .c file and a .so file,\n",
    "# the latter being the more important since that is what actually gets\n",
    "# imported"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "ROOTENV=$(conda info | grep \"root environment :\" | awk '{print $4}')\n",
    "PYTHON_HEADER_DIR=${ROOTENV}/pkgs/python-3.4.3-2/include/python3.4m\n",
    "cython -a awesome_cython_stuff/hello.pyx\n",
    "gcc -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing -I${PYTHON_HEADER_DIR} -o awesome_cython_stuff/hello.so awesome_cython_stuff/hello.c\n",
    "ls awesome_cython_stuff/hello*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hello World in Python vs. Cython"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from regular_old_yet_fine_python_stuff.hello import say_hello as say_hello_python\n",
    "from awesome_cython_stuff.hello import say_hello as say_hello_cython"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "say_hello_python()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "say_hello_cython()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Let's see what kind of difference there is in terms of speed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "t = timeit.Timer(\"say_hello_python()\", \"from regular_old_yet_fine_python_stuff.hello import say_hello as say_hello_python\")\n",
    "print(\"Python function: {} seconds\".format(t.timeit(100000)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "t = timeit.Timer(\"say_hello_cython()\", \"from awesome_cython_stuff.hello import say_hello as say_hello_cython\")\n",
    "print(\"Cython function: {} seconds\".format(t.timeit(100000)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# The C extension version is *usually* at least marginally faster even though\n",
    "# the original code is unchanged"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## \"Great Circle\" Function in Python vs. Cython"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# The \"great circle\" function calculates the distance between two points on\n",
    "# the surface of the earth\n",
    "# Source: http://blog.perrygeo.net/2008/04/19/a-quick-cython-introduction/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Python version\n",
    "! cat regular_old_yet_fine_python_stuff/great_circle.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Cython version (with modified Python code)\n",
    "# All that's different is we're using C data types\n",
    "! cat awesome_cython_stuff/great_circle.pyx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from regular_old_yet_fine_python_stuff.great_circle import great_circle as great_circle_python\n",
    "from awesome_cython_stuff.great_circle import great_circle as great_circle_cython\n",
    "\n",
    "lon1, lat1, lon2, lat2 = -72.345, 34.323, -61.823, 54.826\n",
    "args = \"lon1, lat1, lon2, lat2\"\n",
    "\n",
    "print(\"great_circle_python({1}) = {0}\".format(great_circle_python(lon1, lat1, lon2, lat2),\n",
    "                                              args))\n",
    "print(\"great_circle_cython({1}) = {0}\".format(great_circle_cython(lon1, lat1, lon2, lat2),\n",
    "                                              args))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Test\n",
    "num = 100000\n",
    "\n",
    "t1 = timeit.Timer(\"great_circle_python(%f, %f, %f, %f)\" % (lon1, lat1, lon2, lat2), \n",
    "                  \"from regular_old_yet_fine_python_stuff.great_circle import great_circle as great_circle_python\")\n",
    "t2 = timeit.Timer(\"great_circle_cython(%f, %f, %f, %f)\" % (lon1, lat1, lon2, lat2), \n",
    "                  \"from awesome_cython_stuff.great_circle import great_circle as great_circle_cython\")\n",
    "print(\"Pure python function: {} seconds\".format(t1.timeit(num)))\n",
    "print(\"Cython function: {} seconds\".format(t2.timeit(num)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# The C extension function is faster by a small, but consistent amount\n",
    "# (though the difference decreases as num increases)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Let's try to speed up the C extension by using more C code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "! cat awesome_cython_stuff/great_circle_2.pyx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Now, it does not use the Python `math` library at all to do the calculations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from awesome_cython_stuff.great_circle_2 import great_circle_2\n",
    "\n",
    "print(\"great_circle_2({1}) = {0}\".format(great_circle_2(lon1, lat1, lon2, lat2),\n",
    "                                         args))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Test\n",
    "t3 = timeit.Timer(\"great_circle_2(%f, %f, %f, %f)\" % (lon1, lat1, lon2, lat2), \n",
    "                  \"from awesome_cython_stuff.great_circle_2 import great_circle_2\")\n",
    "print(\"Pure python function: {} seconds\".format(t1.timeit(num)))\n",
    "print(\"Cython function #2: {} seconds\".format(t3.timeit(num)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Now the difference really shows as the Cython module is 10x faster"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# End"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}