6-compilers.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Compilers: Numba, Cython, ~~pybind11~~"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```\n",
    "conda install numba cython\n",
    "```\n",
    "\n",
    "_(and numpy, matplotlib, dask from before)_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Speeding things up through parallel processing is called \"horizontal scaling.\" Often, analysis code can also be accelerated on a single thread, known as \"vertical scaling.\"\n",
    "\n",
    "Horizontal and vertical scaling are complementary."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Let's illustrate this with the fractal example from last time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy\n",
    "import time\n",
    "\n",
    "def run_numpy(height, width, maxiterations=20):\n",
    "    y, x = numpy.ogrid[-1:0:height*1j, -1.5:0:width*1j]  # ask Numpy to make an x, y grid for us\n",
    "    c = x + y*1j                                         # c is a constant: a grid of complex coordinates\n",
    "    z = c\n",
    "    fractal = numpy.zeros(c.shape, dtype=numpy.int32) + maxiterations   # the fractal image starts as \"20\" everywhere\n",
    "    for i in range(maxiterations):\n",
    "        z = z**2 + c                                     # iteratively apply z -> z**2 + c\n",
    "        diverge = numpy.absolute(z) > 2                  # define \"divergence\" by |z| > 2\n",
    "        divnow = diverge & (fractal == maxiterations)    # the pixels that are diverging in this iteration\n",
    "        fractal[divnow] = i                              # the fractal image is a plot of the iteration number\n",
    "        z[diverge] = 2                                   # clamp to 2 so they don't diverge too much\n",
    "    return fractal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Stare at this code: it performs operations across the whole grid, identifies pixels that have diverged, and repeats everything 20 times, even though the parts that have already diverged are \"done.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "More significant than the unnecessary work, though, is the memory movement. Each `z**2 + c` creates new intermediate arrays, moving a lot of memory, flushing CPU caches. Nowadays, mathematical operations are much faster than memory movement."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "As a reminder, this took 35 seconds to run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "starttime = time.time()\n",
    "fractal = run_numpy(8000, 12000)\n",
    "time.time() - starttime"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "If we weren't using Numpy, we'd write the algorithm differently: we'd deal with one pixel at a time. Once the pixel has diverged, we'd move on to the next, saving some work. But more importantly, we make only one pass over the image, avoiding repeated memory access."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def run_python(height, width, maxiterations=20):\n",
    "    y, x = numpy.ogrid[-1:0:height*1j, -1.5:0:width*1j]\n",
    "    c = x + y*1j\n",
    "    fractal = numpy.zeros(c.shape, dtype=numpy.int32) + maxiterations\n",
    "    for h in range(height):\n",
    "        for w in range(width):\n",
    "            z = c[h, w]\n",
    "            for i in range(maxiterations):\n",
    "                z = z**2 + c[h, w]\n",
    "                if abs(z) > 2:\n",
    "                    fractal[h, w] = i\n",
    "                    break\n",
    "    return fractal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Before I run this, I'm going to drop the number of pixels from 8000×12000 to 800×1200, a factor of 100.\n",
    "\n",
    "We don't want to wait 900 seconds (15 minutes)!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "starttime = time.time()\n",
    "fractal = run_python(800, 1200)\n",
    "time.time() - starttime"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(12, 8))\n",
    "ax.imshow(fractal)\n",
    "# ax.imshow(fractal[-200:, :300])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "It works, but it's _super slow!_ This is how an efficient algorithm would go, but stepping through each pixel in Python code kills performance due to all the type-checking, numeric boxing, and virtualization that Python does."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "At this point, we'd normally start thinking about compiled code. And we should: compilation is exactly how to avoid all the aforementioned issues."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "However, we _don't_ need to rewrite our code in another language."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "import numba\n",
    "\n",
    "@numba.jit\n",
    "def run_numba_loop(height, width, maxiterations, c, fractal):\n",
    "    for h in range(height):\n",
    "        for w in range(width):\n",
    "            z = c[h, w]\n",
    "            for i in range(maxiterations):\n",
    "                z = z**2 + c[h, w]\n",
    "                if abs(z) > 2:\n",
    "                    fractal[h, w] = i\n",
    "                    break\n",
    "    return fractal\n",
    "\n",
    "def run_numba(height, width, maxiterations=20):\n",
    "    y, x = numpy.ogrid[-1:0:height*1j, -1.5:0:width*1j]\n",
    "    c = x + y*1j\n",
    "    fractal = numpy.zeros(c.shape, dtype=numpy.int32) + maxiterations\n",
    "    return run_numba_loop(height, width, maxiterations, c, fractal)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "starttime = time.time()\n",
    "fractal = run_numba(8000, 12000, maxiterations=20)\n",
    "time.time() - starttime"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Numba is a \"just in time\" compiler (JIT) for numeric Python. That is, it compiles the Python code as soon as it knows the data types of the inputs, just before execution. (Remember that the compilation time is included in the measurement— it's small compared to 10 seconds, though.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Numba knew to compile the `run_numba_loop` function because it was preceeded by the decorator ` @numba.jit`. It is now a wrapped function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run_numba_loop"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Its \"overloads\" are the saved, compiled functions for each signature. There's only one so far: `int, int, int, array(complex), array(int)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "run_numba_loop.overloads"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Notice that we split the process into `run_numba`, a plain function, and `run_numba_loop`, a JIT-compiled function. Not all Python can be compiled, or we'd be doing it all the time! Python is a highly dynamic language (did you know you can change an object's class after it's created?), so there will always be things Python can do that a compiled language can't do. There will always be data types Numba doesn't recognize."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Numba lists the [Python language features](https://numba.pydata.org/numba-doc/latest/reference/pysupported.html) and [Numpy types and functions](https://numba.pydata.org/numba-doc/latest/reference/numpysupported.html) that it recognizes on its website. This is a growing list, bit it will never converge to the entirety of Python and all its libraries."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "For the most part, Numba recognizes numbers and arrays, and even if it can handle a given language feature (iterators, classes), it will perform best on simple loops and straightforward code. Generally, you only want to wrap the most arithmetically intense part of your calculation.\n",
    "\n",
    "In the above example, I didn't include the array-creation steps because the first one was an unsupported function (`numpy.ogrid`)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "One of the early reasons for Numba's existence was to write new Numpy universal functions (\"ufuncs\")."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@numba.vectorize\n",
    "def as_ufunc(c, maxiterations):\n",
    "    z = c\n",
    "    for i in range(maxiterations):\n",
    "        z = z**2 + c\n",
    "        if abs(z) > 2:\n",
    "            return i\n",
    "    return maxiterations\n",
    "\n",
    "def run_numba_2(height, width, maxiterations=20):\n",
    "    y, x = numpy.ogrid[-1:0:height*1j, -1.5:0:width*1j]\n",
    "    c = x + y*1j\n",
    "    return as_ufunc(c, maxiterations)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "starttime = time.time()\n",
    "fractal = run_numba_2(8000, 12000, maxiterations=20)\n",
    "time.time() - starttime"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "type(as_ufunc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "This is only possible if the process we want to apply is elementwise— we do an independent thing to each element, and the output shape is the same as the input shape— because that's what a ufunc does. The function definition is much simpler since the input argument `c` is now a (complex) number, rather than an array. We don't need to write the for loops."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "It even has the funky ufunc methods, like `.at` and `.reduce`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = numpy.arange(0, 2, 0.1) * 1j\n",
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "as_ufunc.at(a, [0, 2, 4, 6, 8, 10, 12, 14], 20)\n",
    "a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Remember when I said horizontal and vertical scaling are complementary? I didn't say that they're multiplicative because it's sometimes much better than that. Let's put this Numba-compiled ufunc into Dask:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import dask.array\n",
    "\n",
    "def run_dask(height, width, maxiterations=20):\n",
    "    y, x = numpy.ogrid[-1:0:height*1j, -1.5:0:width*1j]\n",
    "    c = dask.array.from_array(x + y*1j, chunks=(100, 12000))\n",
    "    return as_ufunc(c, maxiterations)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "starttime = time.time()\n",
    "fractal = run_dask(8000, 12000, maxiterations=20).compute()\n",
    "time.time() - starttime"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "It took\n",
    "\n",
    "   * 35 seconds to run in Numpy on 1 core.\n",
    "   * 21 seconds to run in Numpy on 12 cores with Dask.\n",
    "   * 10 seconds to run as a Numba-compiled ufunc on 1 core.\n",
    "   * 3.7 seconds to run as a Numba-compiled ufunc on 12 cores with Dask.\n",
    "\n",
    "Dask multiprocessing scales better with the Numba-compiled ufunc because it's a much simpler computation graph. Dask can't see inside `as_ufunc` to worry about interdependencies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "c = dask.array.from_array(numpy.array([[0j, 1j], [0j, 1j]]), chunks=(1, 1))\n",
    "as_ufunc(c, 20).visualize()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Remember how it used to look? How its complexity scaled with the number of iterations? Now all of the complexity of our algorithm is internal to `as_ufunc`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "(Incidentally, the reason it's not scaling beyond 3 cores is likely memory bandwidth: the above example was fetching memory at 1.5 GB/sec, which I've found to be an approximate limit on all systems I've encountered except for Knight's Landing's MCDRAM.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "I presented Numba first because it involves the least change to your code— the orthodox mantra is to get your code working first, profile it to find the slowest parts, and only accelerate those parts. Numba lets you do that with the least effort."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "But sometimes you need something more: features that are only available in C++, for instance. Python is unable to express some concepts related to performance tuning (deliberately: to keep the language simple) and compilers aren't magical— Numba can't always guess what you mean."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Cython is a halfway language, part Python and part C/C++. It translates to C or C++ and uses a conventional compiler to turn into a Python extension module. They also have a Jupyter extension, which I'll use for this demo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext Cython"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The following cell creates C++ code from Python, compiles it, and loads the resulting Python module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%cython --cplus\n",
    "import numpy\n",
    "\n",
    "def run_cython(height, width, maxiterations=20):\n",
    "    y, x = numpy.ogrid[-1:0:height*1j, -1.5:0:width*1j]\n",
    "    c = x + y*1j\n",
    "    fractal = numpy.zeros(c.shape, dtype=numpy.int32) + maxiterations\n",
    "    for h in range(height):\n",
    "        for w in range(width):\n",
    "            z = c[h, w]\n",
    "            for i in range(maxiterations):\n",
    "                z = z**2 + c[h, w]\n",
    "                if abs(z) > 2:\n",
    "                    fractal[h, w] = i\n",
    "                    break\n",
    "    return fractal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "starttime = time.time()\n",
    "fractal = run_cython(800, 1200)     # note: small grid; 100× smaller problem\n",
    "time.time() - starttime"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "But the resulting _compiled_ module runs almost as slowly as Python itself: 7 sec vs 9.5 sec (note: we're using the smaller grid again, so this is hundreds of times slower than Numpy or Numba). Why is that? Isn't this compiled?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The issue is that Cython does nothing about all the runtime type-checking of Python objects. Numba replaced Python objects with raw numbers, where possible, which makes the real difference."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Cython chose to cover the entire Python language and make naive translations by default. Numba chose to make optimized translations by default but not cover the entire Python language."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "To get optimizations, we have to introduce C++ by hand."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%cython --cplus --annotate\n",
    "import cython\n",
    "import numpy      # load Python interface to Numpy\n",
    "cimport numpy     # load C++ interface to Numpy (types end in _t)\n",
    "\n",
    "@cython.boundscheck(False) # turn off bounds-checking\n",
    "@cython.wraparound(False)  # turn off negative index wrapping (e.g. -1 for last element)\n",
    "def run_cython(int height, int width, int maxiterations=20):\n",
    "    y, x = numpy.ogrid[-1:0:height*1j, -1.5:0:width*1j]\n",
    "    c = x + y*1j\n",
    "    fractal = numpy.zeros(c.shape, dtype=numpy.int32) + maxiterations\n",
    "\n",
    "    cdef numpy.ndarray[numpy.complex128_t, ndim=2, mode=\"c\"] c_array = c\n",
    "    cdef numpy.ndarray[numpy.int32_t,      ndim=2, mode=\"c\"] fractal_array = fractal\n",
    "    cdef numpy.complex128_t z\n",
    "    \n",
    "    for h in range(height):\n",
    "        for w in range(width):\n",
    "            z = c_array[h, w]\n",
    "            for i in range(maxiterations):\n",
    "                z = z**2 + c_array[h, w]\n",
    "                if abs(z) > 2:\n",
    "                    fractal_array[h, w] = i\n",
    "                    break\n",
    "    return fractal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "(Still using the small grid; still unable to scale to native speeds.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "starttime = time.time()\n",
    "fractal = run_cython(800, 1200)     # note: small grid; 100× smaller problem\n",
    "time.time() - starttime"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "In addition to importing Python libraries, Cython can include C++ headers. A hidden feature in the `cdef extern` syntax for including C++ allows you to write literal C++ in your Cython."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%cython --cplus -c-O3\n",
    "import numpy\n",
    "\n",
    "cdef extern from *:\n",
    "    \"\"\"\n",
    "    #include <complex>\n",
    "    void quick(int height, int width, int maxiterations, double* c, int* fractal) {\n",
    "        for (int h = 0;  h < height;  h++) {\n",
    "            for (int w = 0;  w < width;  w++) {\n",
    "                double creal = c[2 * (h + height*w)];\n",
    "                double cimag = c[2 * (h + height*w) + 1];\n",
    "                std::complex<double> ci = std::complex<double>(creal, cimag);\n",
    "                std::complex<double> z = ci;\n",
    "                for (int i = 0;  i < maxiterations;  i++) {\n",
    "                    z = z * z + ci;\n",
    "                    if (std::abs(z) > 2) {\n",
    "                        fractal[h + height*w] = i;\n",
    "                        break;\n",
    "                    }\n",
    "                }\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "    \"\"\"\n",
    "    void quick(int height, int width, int maxiterations, double* c, int* fractal)\n",
    "\n",
    "def run_cython(int height, int width, int maxiterations=20):\n",
    "    y, x = numpy.ogrid[-1:0:height*1j, -1.5:0:width*1j]\n",
    "    c = x + y*1j\n",
    "    fractal = numpy.zeros(c.shape, dtype=numpy.int32) + maxiterations\n",
    "    quick(height, width, maxiterations, <double*>(<size_t>c.ctypes.data), <int*>(<size_t>fractal.ctypes.data))\n",
    "    return fractal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Now we can use the full grid. This is\n",
    "\n",
    "   * 2× slower than Numba,\n",
    "   * 2× faster than Numpy, and\n",
    "   * 45× faster than Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "starttime = time.time()\n",
    "fractal = run_cython(8000, 12000)\n",
    "time.time() - starttime"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Although Cython was originally intended as a code optimizer (you can see that it's difficult to use it that way!), it has come to be used to _bind_ C++ libraries as Python extensions, since it can speak both languages."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Today, there's another alternative: pybind11 is a C++ header for binding to Python (coming from the other direction, from C++ to Python)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "I would recommend:\n",
    "\n",
    "   * **Numba** for accelerating small bits of numerical code.\n",
    "   * **Cython** for mixing C++ into a mostly Python script: for instance, to access C++ only libraries.\n",
    "   * **pybind11** for wrapping C++ cleanly as Python modules: for instance, you're distributing a C++ library for use in Python."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "(I've dropped pybind11 content because I'm sure we'd run out of time.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Last minute addition: since you've installed Pandas, you have NumExpr. This is a very easy way to accelerate \"one in, one out\" formulae. Our fractal is more complex than that, but many formulas aren't."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numexpr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = numpy.arange(1e6)\n",
    "b = numpy.arange(1e6)\n",
    "numexpr.evaluate(\"sin(a) + arcsinh(b)\")"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}