Nu_Psi_Nu_2nd_Sep_2020_Student.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Nu Psi Nu 2nd Sep 2020_Student.ipynb",
      "provenance": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VeEcZDhu4sRm",
        "colab_type": "text"
      },
      "source": [
        "# 0.Introduction"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wCIOtNqe4zs2",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "\n",
        "*   Python is a great general-purpose programming language on its own, but with the help of a few popular libraries (numpy, scipy, matplotlib) it becomes a **powerful** environment for scientific computing.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mJmgK-T776Db",
        "colab_type": "text"
      },
      "source": [
        "# 1.Numpy"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H_TkhoogcThr",
        "colab_type": "text"
      },
      "source": [
        "## Outline\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0j4o3z49F382",
        "colab_type": "text"
      },
      "source": [
        "* Arrays\n",
        "* Array indexing\n",
        "* Array math\n",
        "* **Documentation**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "58QdX8BLL9hZ",
        "colab": {}
      },
      "source": [
        "import numpy as np"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "DDx6v1EdL9hb"
      },
      "source": [
        "## Arrays"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "f-Zv3f7LL9hc"
      },
      "source": [
        "* A numpy array is a **grid** of values, all of the **same type**, and is **indexed by a tuple of nonnegative integers**.\n",
        "* We can initialize numpy arrays from **nested Python lists**, and access elements using **square brackets**."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yC4imbmZdtQz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "a = np.array([1, 2, 3])  # Create a rank 1 array\n",
        "print(type(a), a.shape, a[0], a[1], a[2])\n",
        "a[0] = 5                 # Change an element of the array\n",
        "print(a)\n",
        "\n",
        "b = np.array([[1,2,3],[4,5,6]])   # Create a rank 2 array\n",
        "print(type(b), b.shape, b[0], b[1], b[1][0], b[1,0])\n",
        "print(b)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FJruTtHlfu3g",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "a = np.zeros((2,2))  # Create an array of all zeros\n",
        "print(a,'\\n')\n",
        "\n",
        "b = np.ones((2,2))   # Create an array of all ones\n",
        "print(b,'\\n')\n",
        "\n",
        "c = np.full((2,2), 7) # Create a constant array\n",
        "print(c,'\\n')\n",
        "\n",
        "d = np.eye(2)        # Create a 2x2 identity matrix\n",
        "print(d,'\\n')\n",
        "\n",
        "e = np.random.random((2,2)) # Create an array filled with random values\n",
        "print(e,'\\n')\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "agnkxymegy0t",
        "colab_type": "text"
      },
      "source": [
        "## Array indexing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U17f-IXFg5Dp",
        "colab_type": "text"
      },
      "source": [
        "* **Slicing**: Numpy arrays can be sliced. Since arrays may be multidimensional, you must specify a slice for each dimension of the array.\n",
        "* A slice of an array is a **view** into the same data, so modifying it **will modify the original array**."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xqHh5YMrfxvc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Create the following rank 2 array with shape (3, 4)\n",
        "# [[ 1  2  3  4]\n",
        "#  [ 5  6  7  8]\n",
        "#  [ 9 10 11 12]]\n",
        "a = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])\n",
        "\n",
        "# Use slicing to pull out the subarray consisting of the first 2 rows\n",
        "# and columns 1 and 2; b is the following array of shape (2, 2):\n",
        "# [[2 3]\n",
        "#  [6 7]]\n",
        "b = a[:2, 1:3]\n",
        "print(b)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q2s8z7e5fzUm",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(a[0, 1])\n",
        "b[0, 0] = 77    # b[0, 0] is the same piece of data as a[0, 1]\n",
        "print(a[0, 1]) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cK4EXGZGjeHL",
        "colab_type": "text"
      },
      "source": [
        "* Boolean array indexing: This type of indexing is used to select the elements of an array that **satisfy some condition**.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z-OFEWocjkhW",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "a = np.array([[1,2], [3, 4], [5, 6]])\n",
        "\n",
        "bool_idx = (a > 2)  # Find the elements of a that are bigger than 2;\n",
        "                    # this returns a numpy array of Booleans of the same\n",
        "                    # shape as a, where each slot of bool_idx tells\n",
        "                    # whether that element of a is > 2.\n",
        "\n",
        "print(bool_idx)\n",
        "\n",
        "# We use boolean array indexing to construct a rank 1 array\n",
        "# consisting of the elements of a corresponding to the True values\n",
        "# of bool_idx\n",
        "print(a[bool_idx])\n",
        "\n",
        "# We can do all of the above in a single concise statement:\n",
        "print(a[a > 2])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YTJsVAUMly_J",
        "colab_type": "text"
      },
      "source": [
        "## Array math"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qJn8TzAHl3va",
        "colab_type": "text"
      },
      "source": [
        "* Basic mathematical functions operate **elementwise** on arrays, and are available both **as operator overloads and as functions**."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f5HW9X1BmA52",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = np.array([[1,2],[3,4]], dtype=np.float64)\n",
        "y = np.array([[5,6],[7,8]], dtype=np.float64)\n",
        "\n",
        "# Elementwise sum; both produce the array\n",
        "print(x + y)\n",
        "print(np.add(x, y),'\\n')\n",
        "\n",
        "# Elementwise difference; both produce the array\n",
        "print(x - y)\n",
        "print(np.subtract(x, y),'\\n')\n",
        "\n",
        "# Elementwise product; both produce the array\n",
        "print(x * y)\n",
        "print(np.multiply(x, y),'\\n')\n",
        "\n",
        "# Elementwise division; both produce the array\n",
        "# [[ 0.2         0.33333333]\n",
        "#  [ 0.42857143  0.5       ]]\n",
        "print(x / y)\n",
        "print(np.divide(x, y),'\\n')\n",
        "\n",
        "# Elementwise square root; produces the array\n",
        "# [[ 1.          1.41421356]\n",
        "#  [ 1.73205081  2.        ]]\n",
        "print(np.sqrt(x),'\\n')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s2nq3QCcmUDe",
        "colab_type": "text"
      },
      "source": [
        "* Matrix Operations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "P1IXM02BmZkx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = np.array([[1,2],[3,4]])\n",
        "y = np.array([[5,6],[7,8]])\n",
        "\n",
        "# Inner product of vectors;\n",
        "print(np.dot(x, y))\n",
        "print(x @ y,'\\n')\n",
        "\n",
        "# Transpose of a Matrix\n",
        "print(x)\n",
        "print(\"transpose\\n\", x.T)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Wen9YhwNn6l9",
        "colab_type": "text"
      },
      "source": [
        "## [Documentation](https://numpy.org/doc/stable/reference/)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j3QqsdgF76Av",
        "colab_type": "text"
      },
      "source": [
        "# 2.SciPy"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IWO8Fjop55c3",
        "colab_type": "text"
      },
      "source": [
        "* As non-professional programmers, scientists often tend to re-invent the wheel, which leads to buggy, non-optimal, difficult-to-share and unmaintainable code. By contrast, Scipy’s routines are optimized and tested, and should therefore be used when possible."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O7AfhaMC748D",
        "colab_type": "text"
      },
      "source": [
        "[Documentation](https://docs.scipy.org/doc/scipy/reference/api.html)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vm4_OK2GKfYd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#STEP1: Find SVD function's documentation from the link above\n",
        "#STEP2: define a arbitary 3x2 matrix\n",
        "#STEP3: use the SVD function decompose the matrix into its three svd components\n",
        "#STEP4: Convert the middle component to a diagonal matrix from the array\n",
        "#STEP5: Reconstruct original matrix by multiply all the components\n",
        "#STEP6: Print Original and reconstructed matrix and check if both are same "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PpCrXnD976Fb",
        "colab_type": "text"
      },
      "source": [
        "# 3.Numba"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TaE-cH33-to8",
        "colab_type": "text"
      },
      "source": [
        "## Introduction"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L82P3O1Q75-K",
        "colab_type": "text"
      },
      "source": [
        "* Numba provides the ability to speed up applications with high performance functions written directly in Python.\n",
        "* With a few simple annotations, array-oriented and math-heavy Python code can be just-in-time (JIT) optimized to achieve performance similar to C, C++ and Fortran, without having to switch languages or Python interpreters.\n",
        "* Numba works at the function level.\n",
        "* Numba’s main features are:\n",
        "    * On-the-fly code generation (at import time or runtime, at the user’s preference)\n",
        "    * Native code generation for the CPU (default) and GPU hardware\n",
        "    * Integration with the Python scientific software stack (thanks to NumPy)\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6zC5KAbK_JZT",
        "colab_type": "text"
      },
      "source": [
        "## Just-in-time Compiling"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FyHVicDe_u2t",
        "colab_type": "text"
      },
      "source": [
        "* Numba’s central feature is the numba.jit() decoration. Using this decorator, it is possible to mark a function for optimization by Numba’s JIT compiler.\n",
        "* Decorators are a way to uniformly modify functions in a particular way. You can think of them as functions that take functions as input and produce a function as output. See the [Python reference documentation](https://docs.python.org/3/reference/compound_stmts.html#function-definitions) for a more information.\n",
        "* The returned value is bound to the function name instead of the function object. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Z1EGYyWZAteC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Python implementation of bubblesort for NumPy arrays.\n",
        "def bubblesort(X):\n",
        "    #REMOVED\n",
        "    for end in range(N, 1, -1):\n",
        "        for i in range(end - 1):\n",
        "            #REMOVED\n",
        "            if cur > X[i + 1]:\n",
        "                tmp = X[i]\n",
        "                X[i] = X[i + 1]\n",
        "                #REMOVED"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0XoDNUFFBYDK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "# Data\n",
        "original = np.arange(0.0, 10.0, 0.01, dtype='f4')\n",
        "shuffled = original.copy()\n",
        "np.random.shuffle(shuffled)\n",
        "\n",
        "# Testing\n",
        "sorted = shuffled.copy()\n",
        "bubblesort(sorted)\n",
        "print(np.array_equal(sorted, original))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dUfuGQGnCH0Q",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Time Profiling\n",
        "%timeit -n 1 -r 1 sorted[:] = shuffled[:]; bubblesort(sorted) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "52RCqTGNCdDC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Lets JIT it\n",
        "from numba import jit\n",
        "@jit\n",
        "def bubblesort_numba(X):\n",
        "    #COPY CODE FROM bubblesort(X) HERE\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aiJobj_wCtG4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Time Profiling\n",
        "%timeit -n 1 -r 1 sorted[:] = shuffled[:]; bubblesort_numba(sorted)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MWe3zR6gLfaO",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Time Profiling\n",
        "%timeit -n 10 -r 1 sorted[:] = shuffled[:]; bubblesort_numba(sorted)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8aqOhwX__zZh",
        "colab_type": "text"
      },
      "source": [
        "* Using the decorator in this way will defer compilation until the first function execution, so the first execution will be significantly slower.\n",
        "* Numba will infer the argument types at call time, and generate optimized code based on this information. Numba will also be able to compile separate specializations depending on the input types."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0zJEVihvF8n7",
        "colab_type": "text"
      },
      "source": [
        "## Function Signatures"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G521gZ9nGG_g",
        "colab_type": "text"
      },
      "source": [
        "* It is also possible to specify the signature of the Numba function.\n",
        "* This can produce slightly faster code as the compiler does not need to infer the types.\n",
        "* However the function is no longer able to accept other types.\n",
        "* This is useful if you want fine-grained control over types chosen by the compiler (for example, to use single-precision floats)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k1TsVwKdGHpH",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "13675400-1f3c-426f-da17-2becba675b7a"
      },
      "source": [
        "from numba import jit, int32, float64\n",
        "\n",
        "@jit(float64(int32, int32)) #f8(i4, i4)\n",
        "def f(x, y):\n",
        "    # A somewhat trivial example\n",
        "    return (x + y) / 3.14\n",
        "\n",
        "f(1, 3)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.2738853503184713"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 75
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wTrSbjW3JcUt",
        "colab_type": "text"
      },
      "source": [
        "Challenge : See if function signarures can improve performance for the bubblesort_numba function we had above. Report the difference in fastest run time according to timeit."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KEQVBGW-J_7h",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Lets Sign it and Define it\n",
        "#IMPORT TYPES HERE \n",
        "#ADD DECORATOR WITH FUNCTION SIGNATURE HERE - Hint : Input is an array and output is void, for array of int32 you can write 'int32[:]' as type\n",
        "def bubblesort_numba_defined(X):\n",
        "    #COPY CODE FROM bubblesort(X) HERE"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "v2bXSQNnKsS8",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "cc4f4912-cc2e-4657-962b-3385f7070d0c"
      },
      "source": [
        "# Time Profiling\n",
        "#ADD TIMEIT STATEMENT HERE"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1 loop, best of 1: 1.81 ms per loop\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qKSNzgyyRobA",
        "colab_type": "text"
      },
      "source": [
        "## Compilation Modes"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bDTd3HO1RqrQ",
        "colab_type": "text"
      },
      "source": [
        "* Numba has two compilation modes: nopython mode and object mode.\n",
        "* In nopython mode, the Numba compiler will generate code that does not access the Python C API. This mode produces the highest performance code, but requires that the native types of all values in the function can be inferred.\n",
        "* In object mode, the Numba compiler generates code that handles all values as Python objects and uses the Python C API to perform all operations on those objects. Code compiled in object mode will often run no faster than Python interpreted code.\n",
        "* Numba will by default automatically use object mode if nopython mode cannot be used for some reason. Rather than fall back to object mode, it is sometimes preferrable to generate an error instead. By adding the nopython=True keyword, it is possible to force Numbe to do this:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7awqQuoeTDXh",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#STEP1: COPY BLOCK FROM ABOVE\n",
        "#STEP2: import Decimal from decimal\n",
        "#STEP3: ADD LINE \"val = Decimal(100)\" as the first line of the function definition\n",
        "#STEP4: TIMEIT\n",
        "#STEP5: Add \"val = val + 1\" as the first line of innermost for loop\n",
        "#STEP6: TIMEIT"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KKs1TCYrWr5Q",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Time Profiling\n",
        "#ADD TIMEIT STATEMENT HERE"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sKkLiLjosqvC",
        "colab_type": "text"
      },
      "source": [
        "# Detailed Guides"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "541B6xJps0S1",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "\n",
        "*   [Numpy](https://cs231n.github.io/python-numpy-tutorial/#numpy)\n",
        "*   [SciPy](https://scipy-lectures.org/intro/scipy.html)\n",
        "*   [Numba](https://nyu-cds.github.io/python-numba/)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zM4HjZEFTkYD",
        "colab_type": "text"
      },
      "source": [
        "# Self-Promotion"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k4E0h3w0Toaa",
        "colab_type": "text"
      },
      "source": [
        "* LinkedIn: www.linkedin.com/in/ping-jeet\n",
        "* Email: ping.jeet@gmail.com"
      ]
    }
  ]
}