PythonWorkshop2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4A7IZqAlgwIH"
   },
   "source": [
    "## Getting Started\n",
    "\n",
    "Lets start at the very beginning. You can use Python as a calculator. This gives you an idea of how to do arithmetic (and what the arithmetic operators are) in Python."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "F0rZHyfEgc20"
   },
   "source": [
    "![](https://github.com/univai-pyprep-c1/BasicPython/raw/9a66b164175e3032fb88db4112754c1cb07ac4d8//getting_started.slides.dir/8.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "siqC1BNaggkD",
    "outputId": "4d2b00bf-b5c1-4160-a7e0-10deced5bd2b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "1 + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "WWns1uqOhH-L",
    "outputId": "0fc197c2-e463-4d11-f213-c68f8a323c7f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1024"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "2**10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "_9OTo_JWhKZx",
    "outputId": "9776ff9d-d782-4733-c6ec-84a687532b7b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.5"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "5/2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "yyOpHZluhLrN",
    "outputId": "0077e001-d8d0-4388-8e8c-4c9b98b1c3c4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "5//2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hIubnDszhNCF"
   },
   "source": [
    "### Variables\n",
    "\n",
    "Variables are labels for values\n",
    "\n",
    "- Start with a letter or underscore\n",
    "- Can contain only alpha-numeric characters and underscore\n",
    "- Are case sensitive"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "RvjT6_VchV9A",
    "outputId": "1d1d9075-a487-451a-cd4a-c62d5e072a35"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hello\n"
     ]
    }
   ],
   "source": [
    "# CANT DO THIS 9variable = \"hello\"\n",
    "# print 9variable\n",
    "\n",
    "Var = \"hello\"\n",
    "#print (var) # will give an error\n",
    "print (Var)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "znklPOb_hxSJ"
   },
   "source": [
    "What happened above? In the computer's memory, a location was found, and then filled with the word \"hello\". Then a variable Var was created and was used to label this memory location.\n",
    "\n",
    "![](https://github.com/univai-pyprep-c1/BasicPython/raw/9a66b164175e3032fb88db4112754c1cb07ac4d8//images/labelmem.jpg)\n",
    "\n",
    "A variable is literally a label. You can think of it as a post-it, or sticky note, or a pointer. Do not think of it as a box in which a value is stored."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "znklPOb_hxSJ"
   },
   "source": [
    "### Value Types\n",
    "\n",
    "Variables point to values that can be of multiple types. See below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "40Uwt7THhsy1",
    "outputId": "d24f0df9-abde-415d-bfb4-1992f2bf6e64"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7 <class 'int'>\n",
      "7.01 <class 'float'>\n",
      "Hello World! <class 'str'>\n",
      "True <class 'bool'>\n"
     ]
    }
   ],
   "source": [
    "var = 7 # an integer\n",
    "print (var, type(var))\n",
    "var = 7.01 # a float, something with decimals\n",
    "print (var, type(var))\n",
    "var = \"Hello World!\" # a string, a set of characters\n",
    "print (var, type(var))\n",
    "var = True # a boolean, that is, a true or false value\n",
    "print (var, type(var))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "D_Br-QKdikLB"
   },
   "source": [
    "### Variable location and garbage collection\n",
    "\n",
    "In the above, we just take the variable var and point it to different pieces of memory holding different values. This is perhaps more clear from a diagram.\n",
    "\n",
    "![](https://github.com/univai-pyprep-c1/BasicPython/raw/9a66b164175e3032fb88db4112754c1cb07ac4d8//images/labelmemmany.jpg)\n",
    "\n",
    "What happens to the old memory locations. They are now bereft: no variable points to them. Python will detect this and recycle that memory. This is called garbage collection."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nrjc0oIHizIv"
   },
   "source": [
    "There are comparison operators, which can be used to make decisions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "H7alzW88igui",
    "outputId": "f2acfd80-3053-4c2a-820b-cb04377821c8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "False\n",
      "True\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "a = \"hgi\"\n",
    "b = \"hello\"\n",
    "c = \"hi\"\n",
    "d = \"hello\"\n",
    "print (a==c)\n",
    "print (b==d)\n",
    "var1 = 5\n",
    "var2 = 3\n",
    "print (var1 < var2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IrYxzwOJjDAU"
   },
   "source": [
    "The first comparison compares the contents of the memory for a and c and finds that both are different, giving us False. Conversely, the second comparison gives us True. We can utilize such comparisons in \"decision statements\". The third is a numerical comparison. The fact that these comparisons give us a boolean value can be used in the decision-making:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "X-i2giKai-Y7",
    "outputId": "5c532b0d-e05d-4e90-9f1f-2e7227be272a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First variable is lesser than the second variable\n"
     ]
    }
   ],
   "source": [
    "#Simple If conditions\n",
    "var1 = 5\n",
    "var2 = 10\n",
    "\n",
    "if var1 == var2:\n",
    "    print(\"The values are equal\")\n",
    "if var1 < var2:\n",
    "    print(\"First variable is lesser than the second variable\")\n",
    "if var1 > var2:\n",
    "    print(\"Second variable is lesser than the first variable\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8rvW-j1QjYtU"
   },
   "source": [
    "Notice how python dispenses with brackets, replacing them by a colon and an indented next line. The indentation tells us that the code below runs when the condition holds. Python uses this colon-indentation for many things, such as for loops for iteration and loops in general, for conditionals, for function and class definition, etc.\n",
    "\n",
    "This conditional is such a common idiom that there is a better way to write it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "kS9VtABSjUiE",
    "outputId": "1f7f74e8-ce71-4b12-aad0-3df2e616d2ad"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First variable is lesser than the second variable\n"
     ]
    }
   ],
   "source": [
    "#An alternative way to code the previous (If-Else)\n",
    "var1 = 5\n",
    "var2 = 10\n",
    "\n",
    "if (var1 == var2):\n",
    "    print(\"The values are equal\")\n",
    "elif (var1 < var2):\n",
    "    print(\"First variable is lesser than the second variable\")  \n",
    "else:\n",
    "    print(\"Second variable is lesser than the first variable\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "927rJoinjjun"
   },
   "source": [
    "### Functions\n",
    "\n",
    "A function is a set of statements that take inputs, do some specific computation and produces output.\n",
    "\n",
    "There are many ways to define functions:\n",
    "\n",
    "- Functions can be built-in to python, or imported from an existing Python Library\n",
    "- A function can be user defined\n",
    "- A function can be anonymous\n",
    "- functions can belong to objects. More on this later.\n",
    "\n",
    "properties of functions:\n",
    "\n",
    "- A function can be called from other functions\n",
    "- Can return data\n",
    "\n",
    "![](https://github.com/univai-pyprep-c1/BasicPython/raw/9a66b164175e3032fb88db4112754c1cb07ac4d8//functions.slides.dir/2.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "zKtATFxFkJ4m",
    "outputId": "a0b84532-1f20-4f78-a419-1c786ad95183"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "15"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Built-in functions\n",
    "\n",
    "var1 = -15\n",
    "abs(var1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B92MLiNpkSou"
   },
   "source": [
    "Here are two different ways of importing from a module, which is a library of python functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "x6RN4wQBkQIB",
    "outputId": "4915c52c-e590-4ca8-ef4b-cf1361277e75"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.0"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from math import sqrt\n",
    "sqrt(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "UgFRAMpmkWPh",
    "outputId": "577bf8c0-ab3e-45fb-aee5-2cebd0813d40"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "os.cpu_count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bmyWGBnWp7OM"
   },
   "source": [
    "## EXERCISE 1: A Simple Calculator\n",
    "\n",
    "Here is a fun little calculator which exercises what you have learnt so far.\n",
    "\n",
    "It has only 4 operations - Addition, Subtraction, Multiplication and Divison\n",
    "\n",
    "Notice the use of no-operation `pass`. It allows us to incrementally code by doing nothing to start, and put in functionality over time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "XiVTLZDnqB5I",
    "outputId": "6d7152b9-88fd-4429-d681-d961a143e8b8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Select operation\n",
      "\n",
      "1.Addition\n",
      "2.Subtraction\n",
      "3.Multiplication\n",
      "4.Division\n"
     ]
    },
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "\n",
      "Enter choice(1/2/3/4):  \n",
      "\n",
      "Enter first number:  1\n",
      "\n",
      "Enter second number:  2\n"
     ]
    }
   ],
   "source": [
    "print(\"Select operation\\n\")\n",
    "print(\"1.Addition\")\n",
    "print(\"2.Subtraction\")\n",
    "print(\"3.Multiplication\")\n",
    "print(\"4.Division\")\n",
    "\n",
    "# Something NEW: Take input from the user \n",
    "choice = input(\"\\nEnter choice(1/2/3/4): \")\n",
    "\n",
    "# We convert the input to a floating-pount(or real) number.\n",
    "\n",
    "num1 = float(input(\"\\nEnter first number: \"))\n",
    "num2 = float(input(\"\\nEnter second number: \"))\n",
    "\n",
    "if choice == '1':\n",
    "    # your code here\n",
    "    pass\n",
    "elif choice == '2':\n",
    "    # your code here\n",
    "    pass\n",
    "elif choice == '3':\n",
    "    # your code here\n",
    "    pass\n",
    "elif choice == '4': # could use else \n",
    "    if(num2==0):\n",
    "        print (\"Invalid input\")\n",
    "    else:\n",
    "        print(num1,\"/\",num2,\"=\", num1/num2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7mIkf6kRlvy5"
   },
   "source": [
    "## Listiness.. or things that behave like lists\n",
    "\n",
    "Python puts great stock in the idea of having protocols or mechanisms of behavior, and identifying cases in which this behavior is common.\n",
    "\n",
    "One of the most important ideas is that of things that behave like a list of items.\n",
    "\n",
    "These include lists, strings, and files. Many other data structures in Python are made to behave like lists as well, so that their content can be iterated through, in addition to their own native behavior.\n",
    "\n",
    "![](https://github.com/univai-pyprep-c1/BasicPython/raw/9a66b164175e3032fb88db4112754c1cb07ac4d8//listiness.slides.dir/3.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "vXqQam55l9JI",
    "outputId": "a80de3ee-4a5a-4500-f4d9-d714870b5e65"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wednesday\n"
     ]
    }
   ],
   "source": [
    "# CREATING A LIST\n",
    "\n",
    "# A list is made from zero or more elements, separated by commas, and surrounded by square\n",
    "empty_list = []\n",
    "working_days = ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday']\n",
    "print (working_days[2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MLZNK3_Rmao4"
   },
   "source": [
    "![](https://github.com/univai-pyprep-c1/BasicPython/raw/9a66b164175e3032fb88db4112754c1cb07ac4d8//listiness.slides.dir/4.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "sKftKsU5mKS8",
    "outputId": "42abc8ff-52b4-42d5-c1cf-5e8d1613ba64"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 2, 3, 4]\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "lst = list(range(1,5))\n",
    "print(lst)\n",
    "print (lst[-1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "czQ1Vd07mhQr",
    "outputId": "a595ce2c-d2f0-4447-f256-1ff9fda91fa1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 3]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lst[0:3] # from 0, dont include whats at index 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "tXi6tdKGmkNH",
    "outputId": "ac7b93e1-276e-4e54-b60f-fa15cd9caa1a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[2, 3]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lst[-3:-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Ed82ITi8mpJ-",
    "outputId": "8570bdf0-2133-4604-f6d3-8e255cc36377"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1, 2, 3, 4, 5], ['PP', 'BDA', 'USP', 'WTA'], '6th sem'] 3\n"
     ]
    }
   ],
   "source": [
    "# A list can have another list, or anything else as its element\n",
    "numbers = [1, 2, 3, 4, 5]\n",
    "courses = ['PP', 'BDA', \"USP\", 'WTA'] \n",
    "new_list = [numbers, courses, '6th sem'] \n",
    "print (new_list, len(new_list))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "S3n0bTcUmt34",
    "outputId": "b06b407b-621c-4c15-9c0a-0a81c4627bf6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Membership (using 'in' operator)\n",
    "'PP' in courses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "vZuQnv1TmyfA",
    "outputId": "d5cca06b-5385-46e7-b040-081bb0495619"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['a', 'b', 'c', 'd']\n"
     ]
    }
   ],
   "source": [
    "# append(x) - Adds a new element 'x' at the end of the list\n",
    "alist = ['a', 'b', 'c'] \n",
    "alist.append('d') \n",
    "print (alist)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MerhRza8nIoG"
   },
   "source": [
    "Adding lists produces a bigger list! This is an example of a programming technique called operator overloading"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "x7v1WGqVnCr9",
    "outputId": "e730d76a-20bf-4d45-9182-77dcab87f37b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 3, 'a', 'b', 'c']"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "first_list = [1, 2, 3]\n",
    "second_list = ['a', 'b', 'c']\n",
    "first_list + second_list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FLKWEot9nNFZ"
   },
   "source": [
    "## Iteration\n",
    "\n",
    "for loops are fundamental in any language to go over lists.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "wwyBVIB_nHuM",
    "outputId": "b7a4e344-cd96-47e7-ee52-8d0ffe763ff1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4\n",
      "7\n",
      "2\n",
      "6\n",
      "3\n",
      "9\n"
     ]
    }
   ],
   "source": [
    "num = [4, 7, 2, 6, 3, 9]\n",
    "for ele in num:\n",
    "    print(ele)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "qEPSTJTqnX90",
    "outputId": "25f213e2-2e1a-40a7-d639-20070c9f6b09"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4\n",
      "2\n",
      "6\n"
     ]
    }
   ],
   "source": [
    "for ele in num:\n",
    "    if ele % 2 == 0: #even numbers only\n",
    "        print(ele)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QqUiaw7Xneav"
   },
   "source": [
    "There is a short-cut iteration syntax called a list comprehension, often used to construct new lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "uCDuB1PvncOP",
    "outputId": "7bad03d8-f06d-4a84-d5fc-c19aa5204e21"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4, 7, 2, 6, 3, 9]"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list_with_same_as_num = [e for e in num]\n",
    "list_with_same_as_num"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "iFlgf3MmnmN-",
    "outputId": "7642798e-536a-4529-c432-34ccedb466c1"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[16, 49, 4, 36, 9, 81]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "squared_list = [e*e for e in num]\n",
    "squared_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "qrTcWIwgnib-",
    "outputId": "a5275655-25f9-439f-dd33-b63e38dd359e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4, 2, 6]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list_with_evens = [e for e in num if e % 2 == 0]\n",
    "list_with_evens"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tuples\n",
    "\n",
    "They are a fast kind of sequence that functions much like a list - they have elements which are indexed starting at 0. They work exactly like lists, except that tuples can't be changed in place!! This means they are immutable, and this guarantee gives them their speed. Thus, unlike lists, they cannot be grown or shrunk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tuple"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = 1,2,3,4 # or z = (1, 2, 3, 4)\n",
    "type(z) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1, 2, 3, 4)"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "'tuple' object does not support item assignment",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-116-2ac06652192b>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mz\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m5\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: 'tuple' object does not support item assignment"
     ]
    }
   ],
   "source": [
    "z[2] = 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Files\n",
    "\n",
    "The built-in open() function creates a Python file object, which serves as a link to a file residing on your machine. After calling 'open()', strings of data can be transferred to and from the associated external file by calling the returned file object's methods.\n",
    "\n",
    "At this point, you can read data from the file as a whole (read(), or n bytes at a time, read(n). You can read a line at a time with readline(), and all the lines into a list of strings with readlines(). Similar methods exist for writing.\n",
    "\n",
    "You must close the file after you finish using it.\n",
    "\n",
    "![](https://github.com/univai-pyprep-c1/BasicPython/raw/9a66b164175e3032fb88db4112754c1cb07ac4d8//listiness.slides.dir/10.png)\n",
    "\n",
    "But as you might have expected, you can treat a file just like a list even more idiomatically, as we shall see."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<< Obama,Romney,States,Votes\n",
      " >>\n",
      "<< 0.0,1.0,Alabama,9\n",
      " >>\n",
      "<< 0.0,1.0,Alaska,3\n",
      " >>\n",
      "<< 0.062,0.938,Arizona,11\n",
      " >>\n",
      "<< 0.0,1.0,Arkansas,6\n",
      " >>\n",
      "<< 1.0,0.0,California,55\n",
      " >>\n",
      "<< 0.807,0.193,Colorado,9\n",
      " >>\n",
      "<< 1.0,0.0,Connecticut,7\n",
      " >>\n",
      "<< 1.0,0.0,Delaware,3\n",
      " >>\n",
      "<< 1.0,0.0,District of Columbia,3\n",
      " >>\n"
     ]
    }
   ],
   "source": [
    "fd = open(\"data/predictwise.csv\")\n",
    "counter = 0\n",
    "for line in fd:\n",
    "    if counter < 10: # print first 10 lines, there are 51!\n",
    "        print(\"<<\", line, \">>\")\n",
    "    counter = counter + 1 # also writeable as counter += 1\n",
    "fd.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tzPQVSS5oytQ"
   },
   "source": [
    "Notice that the newlines remain. You can use the string method `strip` to remove them. There are many such **methods**, which are functions that belong to strings, or in general, to \"objects\".\n",
    "\n",
    "### Strings \n",
    "This would be a good time to introduce strings as objects. And the key thing to note here is that a lot of what you learn about lists applies to them: they are iterable, and they have a length! But they have one critical additional property: they are immutable, that is they cant be changed!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is a string 16\n",
      "s\n",
      "T\n",
      "h\n",
      "i\n",
      "s\n",
      " \n",
      "i\n",
      "s\n",
      " \n",
      "a\n",
      " \n",
      "s\n",
      "t\n",
      "r\n",
      "i\n",
      "n\n",
      "g\n"
     ]
    }
   ],
   "source": [
    "var = \"This is a string\"\n",
    "print (var, len(var))\n",
    "print (var[3]) # the s of This\n",
    "for char in var:\n",
    "    print(char)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 198
    },
    "id": "vpcUBz3-n29I",
    "outputId": "51f9b16a-efb2-43eb-9737-743dbe504917"
   },
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "'str' object does not support item assignment",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-39-afd63de1e61e>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mvar\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"t\"\u001b[0m \u001b[0;31m# this will fail because immutability\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: 'str' object does not support item assignment"
     ]
    }
   ],
   "source": [
    "var[3] = \"t\" # this will fail because immutability"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "VBx5pYO9n5_b",
    "outputId": "e33968f7-ee21-4036-95eb-bb23faa09ec9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is a string\n",
      "s a string\n",
      "hi\n",
      "This is a strin\n",
      "i sa\n"
     ]
    }
   ],
   "source": [
    "# String slicing\n",
    "print(var)\n",
    "print (var[6:])\n",
    "print (var[1:3])\n",
    "print (var[:-1])\n",
    "print (var[2:10:2]) #the last parameter is for chagning the step size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://github.com/univai-pyprep-c1/BasicPython/raw/9a66b164175e3032fb88db4112754c1cb07ac4d8//listiness.slides.dir/7.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now go look up the documentation of the `strip` method. It removes whitespace \"around\" non-whitespace characters in a string. We can thus use it to get the  lines in our file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<< Obama,Romney,States,Votes >>\n",
      "<< 0.0,1.0,Alabama,9 >>\n",
      "<< 0.0,1.0,Alaska,3 >>\n",
      "<< 0.062,0.938,Arizona,11 >>\n",
      "<< 0.0,1.0,Arkansas,6 >>\n",
      "<< 1.0,0.0,California,55 >>\n",
      "<< 0.807,0.193,Colorado,9 >>\n",
      "<< 1.0,0.0,Connecticut,7 >>\n",
      "<< 1.0,0.0,Delaware,3 >>\n",
      "<< 1.0,0.0,District of Columbia,3 >>\n"
     ]
    }
   ],
   "source": [
    "fd = open(\"data/predictwise.csv\")\n",
    "counter = 0\n",
    "for line in fd:\n",
    "    if counter < 10: # print first 10 lines, there are lots!\n",
    "        print(\"<<\", line.strip(), \">>\")\n",
    "    counter = counter + 1 # also writeable as counter += 1\n",
    "fd.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rzdMCkOBpMFx"
   },
   "source": [
    "### What about writing?\n",
    "\n",
    "Lets write the first ten lines out..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "o9smIsSapKSy",
    "outputId": "6f1a0432-ee67-4064-dc9f-a8b086c6ce51"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<< Obama,Romney,States,Votes >>\n",
      "<< 0.0,1.0,Alabama,9 >>\n",
      "<< 0.0,1.0,Alaska,3 >>\n",
      "<< 0.062,0.938,Arizona,11 >>\n",
      "<< 0.0,1.0,Arkansas,6 >>\n",
      "<< 1.0,0.0,California,55 >>\n",
      "<< 0.807,0.193,Colorado,9 >>\n",
      "<< 1.0,0.0,Connecticut,7 >>\n",
      "<< 1.0,0.0,Delaware,3 >>\n",
      "<< 1.0,0.0,District of Columbia,3 >>\n",
      "10\n"
     ]
    }
   ],
   "source": [
    "fd = open(\"data/predictwise.csv\")\n",
    "fd2 = open(\"data/myown.csv\", \"w\")\n",
    "counter = 0\n",
    "for line in fd:\n",
    "    if counter < 10: # print first 10 lines\n",
    "        print(\"<<\", line.strip(), \">>\")\n",
    "        fd2.write(line)\n",
    "    else:\n",
    "        break # break out of for loop\n",
    "    counter = counter + 1 # also writeable as counter += 1\n",
    "fd.close()\n",
    "fd2.close()\n",
    "print(counter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LE-9eqnOqMYU"
   },
   "source": [
    "## EXERCISE 2: Read a file and parse words from it\n",
    "\n",
    "Read Julius Caesar. Get each line. Remove newline characters from each line. Split the line to get the words from the line. Lowercase them. Print the first 1000 words, lower-cased.\n",
    "\n",
    "Is this really a good way to get words. Can you suggest, a better way, at the cost of more memory?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "wYKpT5bDpQmn",
    "outputId": "e88aebf2-0371-474a-a7ec-f626f46c441d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['the', 'tragedy', 'of', 'julius', 'caesar', 'by', 'william', 'shakespeare', 'contents', 'act', 'i', 'scene', 'i.', 'rome.', 'a', 'street.', 'scene', 'ii.', 'the', 'same.', 'a', 'public', 'place.', 'scene', 'iii.', 'the', 'same.', 'a', 'street.', 'act', 'ii', 'scene', 'i.', 'rome.', 'brutusâ€™', 'orchard.', 'scene', 'ii.', 'a', 'room', 'in', 'caesarâ€™s', 'palace.', 'scene', 'iii.', 'a', 'street', 'near', 'the', 'capitol.', 'scene', 'iv.', 'another', 'part', 'of', 'the', 'same', 'street,', 'before', 'the', 'house', 'of', 'brutus.', 'act', 'iii', 'scene', 'i.', 'rome.', 'before', 'the', 'capitol;', 'the', 'senate', 'sitting.', 'scene', 'ii.', 'the', 'same.', 'the', 'forum.', 'scene', 'iii.', 'the', 'same.', 'a', 'street.', 'act', 'iv', 'scene', 'i.', 'a', 'room', 'in', 'antonyâ€™s', 'house.', 'scene', 'ii.', 'before', 'brutusâ€™', 'tent,', 'in', 'the', 'camp', 'near', 'sardis.', 'scene', 'iii.', 'within', 'the', 'tent', 'of', 'brutus.', 'act', 'v', 'scene', 'i.', 'the', 'plains', 'of', 'philippi.', 'scene', 'ii.', 'the', 'same.', 'the', 'field', 'of', 'battle.', 'scene', 'iii.', 'another', 'part', 'of', 'the', 'field.', 'scene', 'iv.', 'another', 'part', 'of', 'the', 'field.', 'scene', 'v.', 'another', 'part', 'of', 'the', 'field.', 'dramatis', 'personã¦', 'julius', 'caesar', 'octavius', 'caesar,', 'triumvir', 'after', 'his', 'death.', 'marcus', 'antonius,', 'â€\\x9d', 'â€\\x9d', 'â€\\x9d', 'm.', 'aemilius', 'lepidus,', 'â€\\x9d', 'â€\\x9d', 'â€\\x9d', 'cicero,', 'publius,', 'popilius', 'lena,', 'senators.', 'marcus', 'brutus,', 'conspirator', 'against', 'caesar.', 'cassius,', 'â€\\x9d', 'â€\\x9d', 'â€\\x9d', 'casca,', 'â€\\x9d', 'â€\\x9d', 'â€\\x9d', 'trebonius,', 'â€\\x9d', 'â€\\x9d', 'â€\\x9d', 'ligarius,â€\\x9d', 'â€\\x9d', 'â€\\x9d', 'decius', 'brutus,', 'â€\\x9d', 'â€\\x9d', 'â€\\x9d', 'metellus', 'cimber,', 'â€\\x9d', 'â€\\x9d', 'â€\\x9d', 'cinna,', 'â€\\x9d', 'â€\\x9d', 'â€\\x9d', 'flavius,', 'tribune', 'marullus,', 'tribune', 'artemidorus,', 'a', 'sophist', 'of', 'cnidos.', 'a', 'soothsayer', 'cinna,', 'a', 'poet.', 'another', 'poet.', 'lucilius,', 'titinius,', 'messala,', 'young', 'cato,', 'and', 'volumnius,', 'friends', 'to', 'brutus', 'and', 'cassius.', 'varro,', 'clitus,', 'claudius,', 'strato,', 'lucius,', 'dardanius,', 'servants', 'to', 'brutus', 'pindarus,', 'servant', 'to', 'cassius', 'calphurnia,', 'wife', 'to', 'caesar', 'portia,', 'wife', 'to', 'brutus', 'the', 'ghost', 'of', 'caesar', 'senators,', 'citizens,', 'soldiers,', 'commoners,', 'messengers,', 'and', 'servants.', 'scene:', 'rome,', 'the', 'conspiratorsâ€™', 'camp', 'near', 'sardis,', 'and', 'the', 'plains', 'of', 'philippi.', 'act', 'i', 'scene', 'i.', 'rome.', 'a', 'street.', 'enter', 'flavius,', 'marullus', 'and', 'a', 'throng', 'of', 'citizens.', 'flavius.', 'hence!', 'home,', 'you', 'idle', 'creatures,', 'get', 'you', 'home.', 'is', 'this', 'a', 'holiday?', 'what,', 'know', 'you', 'not,', 'being', 'mechanical,', 'you', 'ought', 'not', 'walk', 'upon', 'a', 'labouring', 'day', 'without', 'the', 'sign', 'of', 'your', 'profession?', 'speak,', 'what', 'trade', 'art', 'thou?', 'carpenter.', 'why,', 'sir,', 'a', 'carpenter.', 'marullus.', 'where', 'is', 'thy', 'leather', 'apron', 'and', 'thy', 'rule?', 'what', 'dost', 'thou', 'with', 'thy', 'best', 'apparel', 'on?', 'you,', 'sir,', 'what', 'trade', 'are', 'you?', 'cobbler.', 'truly,', 'sir,', 'in', 'respect', 'of', 'a', 'fine', 'workman,', 'i', 'am', 'but,', 'as', 'you', 'would', 'say,', 'a', 'cobbler.', 'marullus.', 'but', 'what', 'trade', 'art', 'thou?', 'answer', 'me', 'directly.', 'cobbler.', 'a', 'trade,', 'sir,', 'that', 'i', 'hope', 'i', 'may', 'use', 'with', 'a', 'safe', 'conscience,', 'which', 'is', 'indeed,', 'sir,', 'a', 'mender', 'of', 'bad', 'soles.', 'marullus.', 'what', 'trade,', 'thou', 'knave?', 'thou', 'naughty', 'knave,', 'what', 'trade?', 'cobbler.', 'nay,', 'i', 'beseech', 'you,', 'sir,', 'be', 'not', 'out', 'with', 'me;', 'yet,', 'if', 'you', 'be', 'out,', 'sir,', 'i', 'can', 'mend', 'you.', 'marullus.', 'what', 'meanâ€™st', 'thou', 'by', 'that?', 'mend', 'me,', 'thou', 'saucy', 'fellow!', 'cobbler.', 'why,', 'sir,', 'cobble', 'you.', 'flavius.', 'thou', 'art', 'a', 'cobbler,', 'art', 'thou?', 'cobbler.', 'truly,', 'sir,', 'all', 'that', 'i', 'live', 'by', 'is', 'with', 'the', 'awl;', 'i', 'meddle', 'with', 'no', 'tradesmanâ€™s', 'matters,', 'nor', 'womenâ€™s', 'matters,', 'but', 'withal', 'i', 'am', 'indeed,', 'sir,', 'a', 'surgeon', 'to', 'old', 'shoes:', 'when', 'they', 'are', 'in', 'great', 'danger,', 'i', 'recover', 'them.', 'as', 'proper', 'men', 'as', 'ever', 'trod', 'upon', 'neatâ€™s', 'leather', 'have', 'gone', 'upon', 'my', 'handiwork.', 'flavius.', 'but', 'wherefore', 'art', 'not', 'in', 'thy', 'shop', 'today?', 'why', 'dost', 'thou', 'lead', 'these', 'men', 'about', 'the', 'streets?', 'cobbler.', 'truly,', 'sir,', 'to', 'wear', 'out', 'their', 'shoes,', 'to', 'get', 'myself', 'into', 'more', 'work.', 'but', 'indeed,', 'sir,', 'we', 'make', 'holiday', 'to', 'see', 'caesar,', 'and', 'to', 'rejoice', 'in', 'his', 'triumph.', 'marullus.', 'wherefore', 'rejoice?', 'what', 'conquest', 'brings', 'he', 'home?', 'what', 'tributaries', 'follow', 'him', 'to', 'rome,', 'to', 'grace', 'in', 'captive', 'bonds', 'his', 'chariot', 'wheels?', 'you', 'blocks,', 'you', 'stones,', 'you', 'worse', 'than', 'senseless', 'things!', 'o', 'you', 'hard', 'hearts,', 'you', 'cruel', 'men', 'of', 'rome,', 'knew', 'you', 'not', 'pompey?', 'many', 'a', 'time', 'and', 'oft', 'have', 'you', 'climbâ€™d', 'up', 'to', 'walls', 'and', 'battlements,', 'to', 'towers', 'and', 'windows,', 'yea,', 'to', 'chimney', 'tops,', 'your', 'infants', 'in', 'your', 'arms,', 'and', 'there', 'have', 'sat', 'the', 'livelong', 'day', 'with', 'patient', 'expectation,', 'to', 'see', 'great', 'pompey', 'pass', 'the', 'streets', 'of', 'rome.', 'and', 'when', 'you', 'saw', 'his', 'chariot', 'but', 'appear,', 'have', 'you', 'not', 'made', 'an', 'universal', 'shout,', 'that', 'tiber', 'trembled', 'underneath', 'her', 'banks', 'to', 'hear', 'the', 'replication', 'of', 'your', 'sounds', 'made', 'in', 'her', 'concave', 'shores?', 'and', 'do', 'you', 'now', 'put', 'on', 'your', 'best', 'attire?', 'and', 'do', 'you', 'now', 'cull', 'out', 'a', 'holiday?', 'and', 'do', 'you', 'now', 'strew', 'flowers', 'in', 'his', 'way,', 'that', 'comes', 'in', 'triumph', 'over', 'pompeyâ€™s', 'blood?', 'be', 'gone!', 'run', 'to', 'your', 'houses,', 'fall', 'upon', 'your', 'knees,', 'pray', 'to', 'the', 'gods', 'to', 'intermit', 'the', 'plague', 'that', 'needs', 'must', 'light', 'on', 'this', 'ingratitude.', 'flavius.', 'go,', 'go,', 'good', 'countrymen,', 'and,', 'for', 'this', 'fault', 'assemble', 'all', 'the', 'poor', 'men', 'of', 'your', 'sort,', 'draw', 'them', 'to', 'tiber', 'banks,', 'and', 'weep', 'your', 'tears', 'into', 'the', 'channel,', 'till', 'the', 'lowest', 'stream', 'do', 'kiss', 'the', 'most', 'exalted', 'shores', 'of', 'all.', '[_exeunt', 'citizens._]', 'see', 'whether', 'their', 'basest', 'metal', 'be', 'not', 'movâ€™d;', 'they', 'vanish', 'tongue-tied', 'in', 'their', 'guiltiness.', 'go', 'you', 'down', 'that', 'way', 'towards', 'the', 'capitol;', 'this', 'way', 'will', 'i.', 'disrobe', 'the', 'images,', 'if', 'you', 'do', 'find', 'them', 'deckâ€™d', 'with', 'ceremonies.', 'marullus.', 'may', 'we', 'do', 'so?', 'you', 'know', 'it', 'is', 'the', 'feast', 'of', 'lupercal.', 'flavius.', 'it', 'is', 'no', 'matter;', 'let', 'no', 'images', 'be', 'hung', 'with', 'caesarâ€™s', 'trophies.', 'iâ€™ll', 'about', 'and', 'drive', 'away', 'the', 'vulgar', 'from', 'the', 'streets;', 'so', 'do', 'you', 'too,', 'where', 'you', 'perceive', 'them', 'thick.', 'these', 'growing', 'feathers', 'pluckâ€™d', 'from', 'caesarâ€™s', 'wing', 'will', 'make', 'him', 'fly', 'an', 'ordinary', 'pitch,', 'who', 'else', 'would', 'soar', 'above', 'the', 'view', 'of', 'men,', 'and', 'keep', 'us', 'all', 'in', 'servile', 'fearfulness.', '[_exeunt._]', 'scene', 'ii.', 'the', 'same.', 'a', 'public', 'place.', 'enter,', 'in', 'procession,', 'with', 'music,', 'caesar;', 'antony,', 'for', 'the', 'course;', 'calphurnia,', 'portia,', 'decius,', 'cicero,', 'brutus,', 'cassius', 'and', 'casca;', 'a', 'great', 'crowd', 'following,', 'among', 'them', 'a', 'soothsayer.', 'caesar.', 'calphurnia.', 'casca.', 'peace,', 'ho!', 'caesar', 'speaks.', '[_music', 'ceases._]', 'caesar.', 'calphurnia.', 'calphurnia.', 'here,', 'my', 'lord.', 'caesar.', 'stand', 'you', 'directly', 'in', 'antoniusâ€™', 'way,', 'when', 'he', 'doth', 'run', 'his', 'course.', 'antonius.', 'antony.', 'caesar,', 'my', 'lord?', 'caesar.', 'forget', 'not', 'in', 'your', 'speed,', 'antonius,', 'to', 'touch', 'calphurnia;', 'for', 'our', 'elders', 'say,', 'the', 'barren,', 'touched', 'in', 'this', 'holy', 'chase,', 'shake', 'off', 'their', 'sterile', 'curse.', 'antony.', 'i', 'shall', 'remember.']\n"
     ]
    }
   ],
   "source": [
    "# your code here\n",
    "## Read a file, parse lines, and get all words\n",
    "\n",
    "# make a list with all words in documents\n",
    "# the words can occur more than once\n",
    "wordlist = []  \n",
    "fd = open(\"data/Julius Caesar.txt\")\n",
    "lines = fd.readlines()\n",
    "fd.close()\n",
    "# strip newline characters and other whitespace off the edges\n",
    "cleaned_lines = [line.strip() for line in lines] \n",
    "# make a list of lists. \n",
    "# each inner list if the list of words on that line\n",
    "list_of_lines_words = [line.split() for line in lines]\n",
    "# Take each list of words, and get all the words\n",
    "for lines_words in list_of_lines_words:\n",
    "    wordlist = wordlist + [l.lower() for l in lines_words] # update the wordlist using the new list.\n",
    "print(wordlist[:1000]) # first 1000 words"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data\n",
    "\n",
    "Now that we know a bit about files, and a bit about reading files in as strings, let us see how we might get data from these files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['0.0', '1.0', 'Alabama', '9'],\n",
       " ['0.0', '1.0', 'Alaska', '3'],\n",
       " ['0.062', '0.938', 'Arizona', '11'],\n",
       " ['0.0', '1.0', 'Arkansas', '6'],\n",
       " ['1.0', '0.0', 'California', '55'],\n",
       " ['0.807', '0.193', 'Colorado', '9'],\n",
       " ['1.0', '0.0', 'Connecticut', '7'],\n",
       " ['1.0', '0.0', 'Delaware', '3'],\n",
       " ['1.0', '0.0', 'District of Columbia', '3'],\n",
       " ['0.72', '0.28', 'Florida', '29'],\n",
       " ['0.004', '0.996', 'Georgia', '16'],\n",
       " ['1.0', '0.0', 'Hawaii', '4'],\n",
       " ['0.0', '1.0', 'Idaho', '4'],\n",
       " ['1.0', '0.0', 'Illinois', '20'],\n",
       " ['0.036000000000000004', '0.9640000000000001', 'Indiana', '11'],\n",
       " ['0.8370000000000001', '0.163', 'Iowa', '6'],\n",
       " ['0.0', '1.0', 'Kansas', '6'],\n",
       " ['0.0', '1.0', 'Kentucky', '8'],\n",
       " ['0.0', '1.0', 'Louisiana', '8'],\n",
       " ['1.0', '0.0', 'Maine', '4'],\n",
       " ['1.0', '0.0', 'Maryland', '10'],\n",
       " ['1.0', '0.0', 'Massachusetts', '11'],\n",
       " ['0.987', '0.013000000000000001', 'Michigan', '16'],\n",
       " ['0.982', '0.018000000000000002', 'Minnesota', '10'],\n",
       " ['0.0', '1.0', 'Mississippi', '6'],\n",
       " ['0.07400000000000001', '0.9259999999999999', 'Missouri', '10'],\n",
       " ['0.046', '0.9540000000000001', 'Montana', '3'],\n",
       " ['0.0', '1.0', 'Nebraska', '5'],\n",
       " ['0.851', '0.149', 'Nevada', '6'],\n",
       " ['0.857', '0.14300000000000002', 'New Hampshire', '4'],\n",
       " ['0.998', '0.002', 'New Jersey', '14'],\n",
       " ['0.985', '0.015', 'New Mexico', '5'],\n",
       " ['1.0', '0.0', 'New York', '29'],\n",
       " ['0.349', '0.6509999999999999', 'North Carolina', '15'],\n",
       " ['0.025', '0.975', 'North Dakota', '3'],\n",
       " ['0.89', '0.11', 'Ohio', '18'],\n",
       " ['0.0', '1.0', 'Oklahoma', '7'],\n",
       " ['0.976', '0.024', 'Oregon', '7'],\n",
       " ['0.978', '0.022000000000000002', 'Pennsylvania', '20'],\n",
       " ['1.0', '0.0', 'Rhode Island', '4'],\n",
       " ['0.0', '1.0', 'South Carolina', '9'],\n",
       " ['0.001', '0.9990000000000001', 'South Dakota', '3'],\n",
       " ['0.001', '0.9990000000000001', 'Tennessee', '11'],\n",
       " ['0.0', '1.0', 'Texas', '38'],\n",
       " ['0.0', '1.0', 'Utah', '6'],\n",
       " ['1.0', '0.0', 'Vermont', '3'],\n",
       " ['0.7979999999999999', '0.20199999999999999', 'Virginia', '13'],\n",
       " ['0.9990000000000001', '0.001', 'Washington', '12'],\n",
       " ['0.002', '0.998', 'West Virginia', '5'],\n",
       " ['0.925', '0.075', 'Wisconsin', '10'],\n",
       " ['0.0', '1.0', 'Wyoming', '3']]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fd = open(\"data/predictwise.csv\")\n",
    "counter = 0\n",
    "data = []\n",
    "for line in fd:\n",
    "    linedata = line.strip().split(',')\n",
    "    counter += 1\n",
    "    if counter > 1: # dont get header line\n",
    "        data.append(linedata)\n",
    "fd.close()\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It might be easier to arrange this data in multiple arrays, after converting types:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[9,\n",
       " 3,\n",
       " 11,\n",
       " 6,\n",
       " 55,\n",
       " 9,\n",
       " 7,\n",
       " 3,\n",
       " 3,\n",
       " 29,\n",
       " 16,\n",
       " 4,\n",
       " 4,\n",
       " 20,\n",
       " 11,\n",
       " 6,\n",
       " 6,\n",
       " 8,\n",
       " 8,\n",
       " 4,\n",
       " 10,\n",
       " 11,\n",
       " 16,\n",
       " 10,\n",
       " 6,\n",
       " 10,\n",
       " 3,\n",
       " 5,\n",
       " 6,\n",
       " 4,\n",
       " 14,\n",
       " 5,\n",
       " 29,\n",
       " 15,\n",
       " 3,\n",
       " 18,\n",
       " 7,\n",
       " 7,\n",
       " 20,\n",
       " 4,\n",
       " 9,\n",
       " 3,\n",
       " 11,\n",
       " 38,\n",
       " 6,\n",
       " 3,\n",
       " 13,\n",
       " 12,\n",
       " 5,\n",
       " 10,\n",
       " 3]"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fd = open(\"data/predictwise.csv\")\n",
    "counter = 0\n",
    "obamaprobs = []\n",
    "romneyprobs = []\n",
    "states = []\n",
    "electoral_votes = []\n",
    "for line in fd:\n",
    "    linedata = line.strip().split(',')\n",
    "    counter += 1\n",
    "    if counter > 1: # dont get header line\n",
    "        obamaprobs.append(float(linedata[0]))\n",
    "        romneyprobs.append(float(linedata[1]))\n",
    "        states.append(linedata[2])\n",
    "        electoral_votes.append(int(linedata[3]))\n",
    "fd.close()\n",
    "electoral_votes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Numpy\n",
    "\n",
    "The entire process of doing this by yourself is fraught with edge cases and errors: what if a electoral vote value was done in characters? What if a state had a comma? So we prefer to use robust built-in libraries to do the work for us. Here we'll introduce `numpy`, and another workshop will introduce Pandas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "data2 = np.genfromtxt(fname='data/predictwise.csv', \n",
    "                   delimiter=',',\n",
    "                   skip_header=1,\n",
    "                   dtype=(float, float, '|S32', int)\n",
    "        )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([(0.   , 1.   , b'Alabama',  9), (0.   , 1.   , b'Alaska',  3),\n",
       "       (0.062, 0.938, b'Arizona', 11), (0.   , 1.   , b'Arkansas',  6),\n",
       "       (1.   , 0.   , b'California', 55), (0.807, 0.193, b'Colorado',  9),\n",
       "       (1.   , 0.   , b'Connecticut',  7),\n",
       "       (1.   , 0.   , b'Delaware',  3),\n",
       "       (1.   , 0.   , b'District of Columbia',  3),\n",
       "       (0.72 , 0.28 , b'Florida', 29), (0.004, 0.996, b'Georgia', 16),\n",
       "       (1.   , 0.   , b'Hawaii',  4), (0.   , 1.   , b'Idaho',  4),\n",
       "       (1.   , 0.   , b'Illinois', 20), (0.036, 0.964, b'Indiana', 11),\n",
       "       (0.837, 0.163, b'Iowa',  6), (0.   , 1.   , b'Kansas',  6),\n",
       "       (0.   , 1.   , b'Kentucky',  8), (0.   , 1.   , b'Louisiana',  8),\n",
       "       (1.   , 0.   , b'Maine',  4), (1.   , 0.   , b'Maryland', 10),\n",
       "       (1.   , 0.   , b'Massachusetts', 11),\n",
       "       (0.987, 0.013, b'Michigan', 16), (0.982, 0.018, b'Minnesota', 10),\n",
       "       (0.   , 1.   , b'Mississippi',  6),\n",
       "       (0.074, 0.926, b'Missouri', 10), (0.046, 0.954, b'Montana',  3),\n",
       "       (0.   , 1.   , b'Nebraska',  5), (0.851, 0.149, b'Nevada',  6),\n",
       "       (0.857, 0.143, b'New Hampshire',  4),\n",
       "       (0.998, 0.002, b'New Jersey', 14),\n",
       "       (0.985, 0.015, b'New Mexico',  5), (1.   , 0.   , b'New York', 29),\n",
       "       (0.349, 0.651, b'North Carolina', 15),\n",
       "       (0.025, 0.975, b'North Dakota',  3), (0.89 , 0.11 , b'Ohio', 18),\n",
       "       (0.   , 1.   , b'Oklahoma',  7), (0.976, 0.024, b'Oregon',  7),\n",
       "       (0.978, 0.022, b'Pennsylvania', 20),\n",
       "       (1.   , 0.   , b'Rhode Island',  4),\n",
       "       (0.   , 1.   , b'South Carolina',  9),\n",
       "       (0.001, 0.999, b'South Dakota',  3),\n",
       "       (0.001, 0.999, b'Tennessee', 11), (0.   , 1.   , b'Texas', 38),\n",
       "       (0.   , 1.   , b'Utah',  6), (1.   , 0.   , b'Vermont',  3),\n",
       "       (0.798, 0.202, b'Virginia', 13), (0.999, 0.001, b'Washington', 12),\n",
       "       (0.002, 0.998, b'West Virginia',  5),\n",
       "       (0.925, 0.075, b'Wisconsin', 10), (0.   , 1.   , b'Wyoming',  3)],\n",
       "      dtype=[('f0', '<f8'), ('f1', '<f8'), ('f2', 'S32'), ('f3', '<i8')])"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "numpy.ndarray"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(data2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What's this? Its a new datatype, a new kind of list, called a n-dimesnsional array, or ndarray. How many dimensions does it have? Whats so special about it?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data2.ndim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ok. Only one dimension. But we had 4 columns. So what is going on here?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype([('f0', '<f8'), ('f1', '<f8'), ('f2', 'S32'), ('f3', '<i8')])"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data2.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We ask for the datatype, and thus we notice some things. The datatype is a \"tuple\" of fields f0, f1, f2, f3, with sizes: 8byte floating point, 8 byte floating point, 32 byte string (each character takes a byte so we specified 32 chars to allow for long states like \"North Carolina\", and a 8 byte integer. \n",
    "\n",
    "We saw none of this with regular python integers, floats, characters, strings. Why?\n",
    "\n",
    "The reason is that numpy was created to let us do high performance array manipulations in python by making all these manipulations in C. One part of such high performance manipulations is to arrange for memory to be contiguously accessed. See this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(56,)"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data2.strides"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "56 is 8 + 8 + 32 + 8. Which means that you get a new row every 56 bytes in memory. There is no overhead from python objects, etc, just pure memory. From https://numpy.org/doc/stable/reference/internals.html :\n",
    "\n",
    ">NumPy arrays consist of two major components, the raw array data (from now on, referred to as the data buffer), and the information about the raw array data. The data buffer is typically what people think of as arrays in C or Fortran, a contiguous (and fixed) block of memory containing fixed sized data items. NumPy also contains a significant set of data that describes how to interpret the data in the data buffer. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((0., 1., b'Alabama', 9), (0., 1., b'Alaska', 3))"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data2[0], data2[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(9, 3)"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bytearray(data2.data)[48], bytearray(data2.data)[104]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This fast access can now be used for fast computations. You have given up something..python lists can have heterogeneous objects of differing sizes. But you have gained something: the ability to poke into the machine's memory and directly get data out to use in computations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](numpy_arrays.slides.dir/2.png)\n",
    "\n",
    "There are other ways to make numpy arrays:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 2, 3, 4])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_array = np.array([1, 2, 3, 4])\n",
    "my_array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.ones(10) # generates 10 floating point ones"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.ones(10, dtype='int') # generates 10 integer ones"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Numpy arrays are listy! Below we compute length, slice, and iterate. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4\n",
      "[3 4]\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "print(len(my_array))\n",
    "print(my_array[2:4])\n",
    "for ele in my_array:\n",
    "    print(ele)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**However, in general you should manipulate numpy arrays by using numpy module functions** (`np.mean`, for example). This is for efficiency purposes, see the Vanderplas book. But briefly:\n",
    "\n",
    "1. numpy arrays are typed, you keep ints together or floats together. They are not meant to combine objects of different types like python lists do.\n",
    "2. numpy arrays are defined in the C language. You do not have to convert between python floats and C floats for example. Python types are different from the corresponding C-types because Python is garbage collected.\n",
    "\n",
    "You can calculate the mean of the array elements either by calling the method `.mean` on a numpy array or by applying the function np.mean with the numpy array as an argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.5\n",
      "2.5\n"
     ]
    }
   ],
   "source": [
    "print(my_array.mean())\n",
    "print(np.mean(my_array))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To see this consider making two large numpy arrays, multiplying them, and adding the result..this is called a dot product:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {},
   "outputs": [],
   "source": [
    "one = np.random.randn(1000000)\n",
    "two = np.random.randn(1000000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "157 ms ± 376 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "accum = 0\n",
    "for i in range(1000000):\n",
    "    accum += one[i] * two[i]\n",
    "accum"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "489 µs ± 14.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "one @ two"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Numpy supports vector operations\n",
    "\n",
    "What does this mean? It means that instead of adding two arrays, element by element, you can just say: add the two arrays. Note that this behavior is very different from python lists.\n",
    "\n",
    "![](numpy_arrays.slides.dir/3.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2., 2., 2., 2., 2.])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "first = np.ones(5)\n",
    "second = np.ones(5)\n",
    "first + second"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "first_list = [1., 1., 1., 1., 1.]\n",
    "second_list = [1., 1., 1., 1., 1.]\n",
    "first_list + second_list #not what u want"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On some computer chips this addition actually happens in parallel, so speedups can be high. But even on regular chips, the advantage of greater readability is important."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Numpy supports a concept known as *broadcasting*, which dictates how arrays of different sizes are combined together. There are too many rules to list here, but importantly, multiplying an array by a number multiplies each element by the number. Adding a number adds the number to each element."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2., 2., 2., 2., 2.])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "first + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([5., 5., 5., 5., 5.])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "first*5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2D arrays\n",
    "\n",
    "Similarly, we can create two-dimensional arrays.\n",
    "\n",
    "![](numpy_arrays.slides.dir/6.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1,  2,  3,  4],\n",
       "       [ 5,  6,  7,  8],\n",
       "       [ 9, 10, 11, 12]])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_array2d = np.array([ [1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12] ])\n",
    "my_array2d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 1., 1., 1.],\n",
       "       [1., 1., 1., 1.],\n",
       "       [1., 1., 1., 1.]])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 3 x 4 array of ones\n",
    "ones_2d = np.ones([3, 4])\n",
    "ones_2d"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Like lists, numpy arrays are 0-indexed.  Thus we can access the $n$th row and the $m$th column of a two-dimensional array with the indices $[n - 1, m - 1]$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1  2  3  4]\n",
      " [ 5  6  7  8]\n",
      " [ 9 10 11 12]]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "12"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(my_array2d)\n",
    "my_array2d[2, 3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The 2D arrays are listy as well. They have set length (array dimensions), can be sliced, and can be iterated over with loop.  Below is a schematic illustrating slicing two-dimensional arrays.  \n",
    "\n",
    " <img src=\"images/2dindex_v2.png\" alt=\"Drawing\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In two dimensions, we need to provide the **shape** of the array, ie, the number of rows and columns of the array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 1., 1., 1.],\n",
       "       [1., 1., 1., 1.],\n",
       "       [1., 1., 1., 1.]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "onesarray = np.ones([3,4])\n",
    "onesarray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 4)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "onesarray.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Numpy functions will by default work on the entire array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "12.0"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sum(onesarray)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The axis 0 is the one going downwards (the $y$-axis, so to speak), whereas axis 1 is the one going across (the $x$-axis). You will often use functions such as `mean`, `sum`, with an axis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([3., 3., 3., 3.])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sum(onesarray, axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([15, 18, 21, 24])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sum(my_array2d, axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([4., 4., 4.])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sum(onesarray, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should notice that access is row-by-row and one dimensional iteration gives a row. This is because `numpy` lays out memory row-wise.\n",
    "\n",
    "![](images/2d-array-layout.png)\n",
    "\n",
    "(from https://aaronbloomfield.github.io)\n",
    "\n",
    "You know this...we saw the size 56 stride when we read in our file..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Idiomatic filling\n",
    "\n",
    "An often seen idiom allocates a two-dimensional array, and then fills in one-dimensional arrays from some function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 0., 0.],\n",
       "       [0., 0., 0.]])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "empty_array = np.empty((2,3))\n",
    "empty_array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0. 0. 0.]\n",
      "[0. 0. 0.]\n"
     ]
    }
   ],
   "source": [
    "for i in range(empty_array.shape[0]):\n",
    "    print(empty_array[i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.68625855, 0.09451053, 0.9559449 ],\n",
       "       [0.03386545, 0.08089574, 0.84962253]])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for i in range(empty_array.shape[0]):\n",
    "    empty_array[i] = np.random.rand(3)\n",
    "empty_array"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3: Read an array from a file and make computations on it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [],
   "source": [
    "data3 = np.genfromtxt(fname='data/inflammation-01.csv', \n",
    "                   delimiter=',',\n",
    "        )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 0., 1., ..., 3., 0., 0.],\n",
       "       [0., 1., 2., ..., 1., 0., 1.],\n",
       "       [0., 1., 1., ..., 2., 1., 1.],\n",
       "       ...,\n",
       "       [0., 1., 1., ..., 1., 1., 1.],\n",
       "       [0., 0., 0., ..., 0., 2., 0.],\n",
       "       [0., 0., 1., ..., 1., 1., 0.]])"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2, (60, 40), (320, 8))"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data3.ndim, data3.shape, data3.strides"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('float64')"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data3.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(40,)"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data3[0].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.14875"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data3.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.        ,  0.45      ,  1.11666667,  1.75      ,  2.43333333,\n",
       "        3.15      ,  3.8       ,  3.88333333,  5.23333333,  5.51666667,\n",
       "        5.95      ,  5.9       ,  8.35      ,  7.73333333,  8.36666667,\n",
       "        9.5       ,  9.58333333, 10.63333333, 11.56666667, 12.35      ,\n",
       "       13.25      , 11.96666667, 11.03333333, 10.16666667, 10.        ,\n",
       "        8.66666667,  9.15      ,  7.25      ,  7.33333333,  6.58333333,\n",
       "        6.06666667,  5.95      ,  5.11666667,  3.6       ,  3.3       ,\n",
       "        3.56666667,  2.48333333,  1.5       ,  1.13333333,  0.56666667])"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(data3, axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([5.45 , 5.425, 6.1  , 5.9  , 5.55 , 6.225, 5.975, 6.65 , 6.625,\n",
       "       6.525, 6.775, 5.8  , 6.225, 5.75 , 5.225, 6.3  , 6.55 , 5.7  ,\n",
       "       5.85 , 6.55 , 5.775, 5.825, 6.175, 6.1  , 5.8  , 6.425, 6.05 ,\n",
       "       6.025, 6.175, 6.55 , 6.175, 6.35 , 6.725, 6.125, 7.075, 5.725,\n",
       "       5.925, 6.15 , 6.075, 5.75 , 5.975, 5.725, 6.3  , 5.9  , 6.75 ,\n",
       "       5.925, 7.225, 6.15 , 5.95 , 6.275, 5.7  , 6.1  , 6.825, 5.975,\n",
       "       6.725, 5.7  , 6.25 , 6.4  , 7.05 , 5.9  ])"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(data3, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([18., 18., 19., 17., 17., 18., 17., 20., 17., 18., 18., 18., 17.,\n",
       "       16., 17., 18., 19., 19., 17., 19., 19., 16., 17., 15., 17., 17.,\n",
       "       18., 17., 20., 17., 16., 19., 15., 15., 19., 17., 16., 17., 19.,\n",
       "       16., 18., 19., 16., 19., 18., 16., 19., 15., 16., 18., 14., 20.,\n",
       "       17., 15., 17., 16., 17., 19., 18., 18.])"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data3.max(axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 0., 0., 0., 1., 1., 1., 1., 2., 2., 2., 2., 3., 3., 3., 3., 4.,\n",
       "       5., 5., 5., 5., 4., 4., 4., 4., 3., 3., 3., 3., 2., 2., 2., 2., 1.,\n",
       "       1., 1., 1., 0., 0., 0.])"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data3.min(axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}