IBAN_generation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "T4sh0ehhyAJz"
   },
   "source": [
    "# generate IBANs only to mutilate them later\n",
    "Pop Quiz, Invoice Suckers!\n",
    "\n",
    "Which of the following is a valid IBAN?\n",
    "```\n",
    "NL59 K1NAB 6291 4688 95   |   NL29 COBA 9059T 5649 88   |   NL47 ABNA 83Q91 2006 36\n",
    "NL59 KNAB 629?1 4688 95   |   N\\L29 COBA 9059 5649 88   |   NL47 ABNA_ 8391 2006 36\n",
    "NL59 KNAB 6291 468 95     |   NL29 COBA 90595649 88     |   NL47 ABNA 8391 2006 6\n",
    "NL59 \u000b",
    "KNAB 6291 4688 95    |   N\tL29 COBA 9059 5649 88|   NL47 \f",
    "ABNA 8391 2006 36\n",
    "NL59 KNAb 6291 4688 95    |   Nl29 COBA 9059 5649 88    |   NL47 ABnA 8391 2006 36\n",
    "NL59 KNAB 6291 5688 95    |   NL29 COBA 9059 5749 88    |   NL47 ABNA 8391 3006 36\n",
    "```\n",
    "The right answer is: None of them. The IBANs above were mutilated to simulate noise in our scanned invoices. I applied six different point-mutations to the strings. \n",
    "\n",
    "But is it really so difficult to combine six invalid IBANs into one valid IBAN? I believe a NN could learn to solve this. With the functions below it is possible to generate a training set for that NN. \n",
    "\n",
    "BTW, the correct IBANs are:\n",
    "```\n",
    "NL59 KNAB 6291 4688 95    |   NL29 COBA 9059 5649 88    |   NL47 ABNA 8391 2006 36\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FEJ37aWRLr-O"
   },
   "source": [
    "## notes on code\n",
    "```\n",
    "A formatted iban is called a fiban to distinguish a IBAN object from a string object.\n",
    "(I suggest the name ciban for a compact iban.)\n",
    "All functions (that handle lists of [cf]ibans) expect and return numpy arrays.\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "37P4BMat_dMr"
   },
   "source": [
    "## where next?\n",
    "* include cibans into the mix, Jannes' generator only uses the compact format\n",
    "* generate a large pandas dataframe filled with some number of different versions (X) of the same fiban. \n",
    "* Add a column with the correct fiban (y).\n",
    "* train a NN to take in different versions and predict the correct fiban.\n",
    "\n",
    "Problems I see with that:\n",
    "* How do we format the data for the NN's input? Onehot encoded characters, as Jannes' baseline?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "JA9mosAUyjHt"
   },
   "outputs": [],
   "source": [
    "!pip install -q schwifty"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "tIVSAyQ9ynd8"
   },
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'keras'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-20-21ad8d8ead18>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mre\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mpandas\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mkeras\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrandom\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mseed\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'keras'"
     ]
    }
   ],
   "source": [
    "from schwifty import IBAN \n",
    "import numpy as np\n",
    "import time as time\n",
    "import re\n",
    "import pandas as pd\n",
    "import keras\n",
    "\n",
    "np.random.seed(int(time.time()))\n",
    "  \n",
    "# NL bank codes taken from https://www.betaalvereniging.nl/aandachtsgebieden/giraal-betalingsverkeer/bic-sepa-transacties/\n",
    "dutch_bankcodes = \"ABNA,ADYB,AEGO,ANAA,ANDL,ARBN,ARSN,ASNB,ATBA,BCDM,BCIT,BICK,BINK,BKCH,BKMG,BLGW,BMEU,BNDA,BNGH,BNPA,BOFA,BOFS,BOTK,BUNQ,CHAS,CITC,CITI,COBA,DEUT,DHBN,DLBK,DNIB,EBUR,FBHL,FLOR,FRGH,FRNX,FTSB,FVLB,GILL,HAND,HHBA,HSBC,ICBK,INGB,ISBK,KABA,KASA,KNAB,KOEX,KRED,LOCY,LOYD,LPLN,MHCB,MOYO,NNBA,NWAB,PCBC,RABO,RBRB,SNSB,SOGE,TEBU,TRIO,UBSW,UGBI,VOWA,ZWLB\".split(\",\")\n",
    "\n",
    "# print(dutch_bankcodes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 70,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 490,
     "status": "ok",
     "timestamp": 1520950422214,
     "user": {
      "displayName": "rik",
      "photoUrl": "//lh6.googleusercontent.com/-ZGIi-vN4KMU/AAAAAAAAAAI/AAAAAAAAAFk/Ki-nNio6VVM/s50-c-k-no/photo.jpg",
      "userId": "113719912521762461555"
     },
     "user_tz": -60
    },
    "id": "RrQoxjuRzGIW",
    "outputId": "4d6e27df-d806-4329-ee4c-0f82f32e4327"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NL56 ASNB 1007 0273 82\n",
      "NL56ASNB1007027382\n"
     ]
    }
   ],
   "source": [
    "iban = IBAN.generate('NL', bank_code='ASNB', account_code='1007027382')\n",
    "print(iban.formatted)\n",
    "print(iban.compact)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "nqzLZ6tgGryq"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "26qVYhZlyk5d"
   },
   "outputs": [],
   "source": [
    "def generate_random_fibans(num_fibans, country_code='NL'):\n",
    "  fibans = []\n",
    "  for _ in range(num_fibans):\n",
    "    bank_code = dutch_bankcodes[ np.random.randint(len(dutch_bankcodes)) ]\n",
    "    account_code = str(np.random.randint(9999, high=9999999999))\n",
    "    iban = IBAN.generate(country_code, bank_code=bank_code, account_code=account_code)\n",
    "    fibans.append(iban.formatted)\n",
    "\n",
    "  return np.array(fibans)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "U36WbgnizuMO"
   },
   "outputs": [],
   "source": [
    "def validate_fibans(fibans, mode='all'):\n",
    "  validations = []\n",
    "  for fiban in fibans:\n",
    "    try:\n",
    "      IBAN(fiban)\n",
    "    except ValueError as e:\n",
    "      validations.append(False)\n",
    "    else:\n",
    "      validations.append(True)\n",
    "\n",
    "  if mode=='all':\n",
    "    judgment = np.all(validations)\n",
    "  elif mode=='any':\n",
    "    judgment = np.any(validations)\n",
    "\n",
    "  return judgment, np.array(validations)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "vrcYh79A303_"
   },
   "outputs": [],
   "source": [
    "# these functions take in a string, a position in the string and sometimes a new character\n",
    "# they return a new string\n",
    "\n",
    "def insert_char(instring, position, insert):\n",
    "  return instring[:position] + insert + instring[position:]\n",
    "\n",
    "def delete_char(instring, position):\n",
    "  return instring[:position] + instring[position+1:]\n",
    "\n",
    "def replace_char(instring, position, replacement):\n",
    "  return instring[:position] + replacement + instring[position+1:]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "h81MX7AQqvNN"
   },
   "outputs": [],
   "source": [
    "def mutilate_strings(string_list, mode='insval', num_mutils=1):\n",
    "  \"\"\"\n",
    "  mutilates all strings in the input list at random position(s) of the string\n",
    "  \n",
    "  Choose a mode fom this list;\n",
    "  insval:   insert valid character\n",
    "  insinv:   insert invalid character\n",
    "  del:      delete character\n",
    "  inswhite: insert whitespace\n",
    "  lower:    change character to lowercase\n",
    "  plus:     increase digit by 1, (9 to 0)\n",
    "  \"\"\"\n",
    "\n",
    "  \n",
    "  valid_chars = \"ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789\"\n",
    "  invalid_chars = \"!@#$%^&*()_+-=[]{}\\\\';:\\\"|?><,./`~\\\"'±§™\"\n",
    "  whitespaces = '\\t\\n\\x0b\\x0c\\r '\n",
    "  digits = \"0123456789\"\n",
    "  caps_re = re.compile(r'([A-Z])')\n",
    "  digits_re = re.compile(r'([0-9])')\n",
    "\n",
    "  mutilated_list = []\n",
    "  for string in string_list:\n",
    "    \n",
    "    if mode=='insval':\n",
    "      # insert one or more VALid characters anywhere in the string\n",
    "      for _ in range(num_mutils):\n",
    "        p = np.random.randint(len(string)+1)\n",
    "        c = valid_chars[ np.random.randint(len(valid_chars)) ]\n",
    "        string = insert_char(string, p, c)\n",
    "    elif mode=='insinv':\n",
    "      # insert one or more INValid characters anywhere in the string\n",
    "      for _ in range(num_mutils):\n",
    "        p = np.random.randint(len(string)+1)\n",
    "        c = invalid_chars[ np.random.randint(len(valid_chars)) ]\n",
    "        string = insert_char(string, p, c)\n",
    "    elif mode=='inswhite':\n",
    "      # insert one or more WHITEspace characters anywhere in the string\n",
    "      for _ in range(num_mutils):\n",
    "        p = np.random.randint(len(string)+1)\n",
    "        c = whitespaces[ np.random.randint(len(whitespaces)) ]\n",
    "        string = insert_char(string, p, c)\n",
    "    elif mode=='del':\n",
    "      # delete one or more characters anywhere in the string\n",
    "      for _ in range(num_mutils):\n",
    "        p = np.random.randint(len(string)) + 1\n",
    "        string = delete_char(string, p)\n",
    "    elif mode=='lower':\n",
    "      # change one or more characters to lowercase anywhere in the string\n",
    "      for _ in range(num_mutils):\n",
    "        caps_idx = [i.start() for i in caps_re.finditer(string) ]\n",
    "        if caps_idx:\n",
    "          p = caps_idx[np.random.randint(len(caps_idx))]\n",
    "          replacement = (string[p]).lower()\n",
    "          string = replace_char(string, p, replacement)\n",
    "    elif mode=='plus':\n",
    "      # change one or more characters to lowercase anywhere in the string\n",
    "      for _ in range(num_mutils):\n",
    "        digits_idx = [i.start() for i in digits_re.finditer(string) ]\n",
    "        if digits_idx:\n",
    "          p = digits_idx[np.random.randint(len(digits_idx))]\n",
    "          replacement = (int(string[p]) + 1) % 10\n",
    "          string = replace_char(string, p, str(replacement))\n",
    "    # endif\n",
    "\n",
    "    mutilated_list.append(string)\n",
    "    \n",
    "  return np.array(mutilated_list)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "WC4U1IbLSHml"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 153,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 493,
     "status": "ok",
     "timestamp": 1520952430520,
     "user": {
      "displayName": "rik",
      "photoUrl": "//lh6.googleusercontent.com/-ZGIi-vN4KMU/AAAAAAAAAAI/AAAAAAAAAFk/Ki-nNio6VVM/s50-c-k-no/photo.jpg",
      "userId": "113719912521762461555"
     },
     "user_tz": -60
    },
    "id": "Hx_NUTHo7ZAf",
    "outputId": "829b76df-607f-45f5-c6b3-45d215e8a7c1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NL85 LPLN 8569 5349 16   |   NL26 AEGO 2085 2103 43   |   NL31 DLBK 3955 9851 22\n",
      "NL85 LPLN 8569 534R9 16   |   NL26 AEGO 2085 2103 413   |   NL31 DLBKI 3955 9851 22\n",
      "NL85 LPLN 8569 53~49 16   |   NL2>6 AEGO 2085 2103 43   |   NL31' DLBK 3955 9851 22\n",
      "NL85 LPLN 8569 5349 6   |   NL6 AEGO 2085 2103 43   |   NL31 DLBK 3955 981 22\n",
      "NL85 LPLN\r",
      " 8569 5349 16   |   NL26 AEGO 2085 210\n",
      "3 43   |   NL\n",
      "31 DLBK 3955 9851 22\n",
      "Nl85 LPLN 8569 5349 16   |   NL26 AEGo 2085 2103 43   |   Nl31 DLBK 3955 9851 22\n",
      "NL85 LPLN 8579 5349 16   |   NL26 AEGO 2086 2103 43   |   NL31 DLBK 3965 9851 22\n"
     ]
    }
   ],
   "source": [
    "fibans = generate_random_fibans(3) # about 10s for 100k fibans in Colab\n",
    "print(\"   |   \".join(fibans))\n",
    "\n",
    "for m in ['insval', 'insinv', 'del', 'inswhite', 'lower', 'plus']:\n",
    "  fibans_mut = mutilate_strings(fibans, mode=m)\n",
    "  print(\"   |   \".join(fibans_mut))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 219,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 513,
     "status": "ok",
     "timestamp": 1520950791307,
     "user": {
      "displayName": "rik",
      "photoUrl": "//lh6.googleusercontent.com/-ZGIi-vN4KMU/AAAAAAAAAAI/AAAAAAAAAFk/Ki-nNio6VVM/s50-c-k-no/photo.jpg",
      "userId": "113719912521762461555"
     },
     "user_tz": -60
    },
    "id": "pmz3Zg8zHiUF",
    "outputId": "6aef1828-1e99-4626-8cc1-288a890c1458"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['NL85 LPLN 8569 5349 16' 'NL26 AEGO 2085 2103 43' 'NL31 DLBK 3955 9851 22']\n",
      "[]\n"
     ]
    }
   ],
   "source": [
    "(judgement, index) = validate_fibans(fibans)\n",
    "print(fibans[index])\n",
    "\n",
    "(judgement, index) = validate_fibans(fibans_mut)\n",
    "print(fibans_mut[index])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "W96WZbx-ES2p"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "3nq6K9DVKau5"
   },
   "outputs": [],
   "source": [
    "fibans_suspect = np.array(['NL57 ARBN 6235 3382 93', 'NL09 C0BA 9088 1794 75', 'NL5I FVLB 4249 7745 45', 'NL73 BCIT 1427 5181 56', 'NL92 BCIT 2066 7794 36', 'NL67 FTSB 3188 3991 40', 'NL82 GILL 5931 7640 18'])\n",
    "(judgement, index) = validate_fibans(fibans_suspect)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 83,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 503,
     "status": "ok",
     "timestamp": 1520951419205,
     "user": {
      "displayName": "rik",
      "photoUrl": "//lh6.googleusercontent.com/-ZGIi-vN4KMU/AAAAAAAAAAI/AAAAAAAAAFk/Ki-nNio6VVM/s50-c-k-no/photo.jpg",
      "userId": "113719912521762461555"
     },
     "user_tz": -60
    },
    "id": "-A8JN0jZl5Vc",
    "outputId": "da6af839-0952-416e-c9f0-ce56502556f1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['NL54 ABNA 0473 1693 71']\n",
      "(True, array([ True], dtype=bool))\n",
      "['NL54 ABNA 0473 2693 71']\n",
      "(False, array([False], dtype=bool))\n"
     ]
    }
   ],
   "source": [
    "# from https://tsociety.io\n",
    "cturban = 'NL54ABNA0473169371'\n",
    "turban = IBAN(cturban)\n",
    "fturbans = [turban.formatted]\n",
    "\n",
    "print(fturbans)\n",
    "print(validate_fibans(fturbans))\n",
    "\n",
    "fturbans_plus = mutilate_strings(fturbans, mode='plus')\n",
    "print(fturbans_plus)\n",
    "print(validate_fibans(fturbans_plus))\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "WOxG-FBlAe8e"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 54,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 208280,
     "status": "ok",
     "timestamp": 1520956296480,
     "user": {
      "displayName": "rik",
      "photoUrl": "//lh6.googleusercontent.com/-ZGIi-vN4KMU/AAAAAAAAAAI/AAAAAAAAAFk/Ki-nNio6VVM/s50-c-k-no/photo.jpg",
      "userId": "113719912521762461555"
     },
     "user_tz": -60
    },
    "id": "7sYnUhdSAfZ3",
    "outputId": "a7b9a622-90e4-40fb-d6d6-9ea48ab05f1c"
   },
   "outputs": [],
   "source": [
    "fibans = generate_random_fibans(100)  # 1M in 210 seconds, including the mutilations\n",
    "\n",
    "fibans_v1 = mutilate_strings(fibans, mode='insval', num_mutils=4)   # insert 4 extra chars\n",
    "fibans_v1 = mutilate_strings(fibans_v1, mode='lower')               # change 1 UPPER to lower case\n",
    "fibans_v1 = mutilate_strings(fibans_v1, mode='del', num_mutils=2)  # delete 2 chars\n",
    "\n",
    "fibans_v2 = mutilate_strings(fibans, mode='insval', num_mutils=2)   # insert 2 extra chars\n",
    "fibans_v2 = mutilate_strings(fibans_v2, mode='del', num_mutils=2)   # delete 2 chars\n",
    "fibans_v2 = mutilate_strings(fibans_v2, mode='inswhite', num_mutils=6)   # insert 6 whitespaces\n",
    "\n",
    "fibans_v3 = mutilate_strings(fibans, mode='del', num_mutils=4)      # delete 4 chars\n",
    "\n",
    "fibans_v4 = mutilate_strings(fibans, mode='insinv', num_mutils=4)      # insert 4 invalid chars\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "B9HHCWrnCK1R"
   },
   "outputs": [],
   "source": [
    "train = pd.DataFrame(columns=['noise1', 'noise2', 'noise3', 'noise4', 'iban'])\n",
    "\n",
    "train['noise1'] = fibans_v1\n",
    "train['noise2'] = fibans_v2\n",
    "train['noise3'] = fibans_v3\n",
    "train['noise4'] = fibans_v4\n",
    "train['iban']   = fibans\n",
    "\n",
    "train.to_csv('iban_noisy_100.csv')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 222,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 413,
     "status": "ok",
     "timestamp": 1520962145645,
     "user": {
      "displayName": "rik",
      "photoUrl": "//lh6.googleusercontent.com/-ZGIi-vN4KMU/AAAAAAAAAAI/AAAAAAAAAFk/Ki-nNio6VVM/s50-c-k-no/photo.jpg",
      "userId": "113719912521762461555"
     },
     "user_tz": -60
    },
    "id": "L_Zww8dZCHHy",
    "outputId": "b7227a0f-7ee1-49a9-9537-0a9960bf6c59"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>noise1</th>\n",
       "      <th>noise2</th>\n",
       "      <th>noise3</th>\n",
       "      <th>noise4</th>\n",
       "      <th>iban</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NL66 RbB 65J01 C137211N3</td>\n",
       "      <td>\\tN  LQZ 66\\r\\rRBRB 6501 131 13</td>\n",
       "      <td>NL66RBB651 1371 13</td>\n",
       "      <td>NL66 RBR`B 6;501 1+371% 13</td>\n",
       "      <td>NL66 RBRB 6501 1371 13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>oNL47 KNAB2266336 858 02</td>\n",
       "      <td>N47 K\\nNAFB  2\\t63\f",
       "6  8Q586 0\\r</td>\n",
       "      <td>NL47 KNAB 2636 860</td>\n",
       "      <td>NL47 KNAB \\2636 @\"8~586 02</td>\n",
       "      <td>NL47 KNAB 2636 8586 02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>nL73 NNBR 1346 2S53275 58</td>\n",
       "      <td>\\rN73NNB\\rA\\t 1\\r3462 32J\\t75 5\\r8</td>\n",
       "      <td>NL73 NNBA 1346327</td>\n",
       "      <td>'NL73 NNBA 1346 !;3275 5&lt;8</td>\n",
       "      <td>NL73 NNBA 1346 3275 58</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NL694 DHT 7609 2913 S25X</td>\n",
       "      <td>\\nNL\f",
       "6\u000b",
       "4\\r D\u000b",
       "HBN 7\u000b",
       "60 O2913 25</td>\n",
       "      <td>NL DHBN 7609 29325</td>\n",
       "      <td>NL64@ DHBN! 7609] 2913 2\"5</td>\n",
       "      <td>NL64 DHBN 7609 2913 25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>InJL64 BUYNPQ592 7256 26</td>\n",
       "      <td>N\f",
       "L4 \\tBUQ 592F1 Z725\u000b",
       "6\\t\u000b",
       " 26</td>\n",
       "      <td>NL64 UNQ 921 256 6</td>\n",
       "      <td>NL)64 BUNQ 5?92{1 72[56 26</td>\n",
       "      <td>NL64 BUNQ 5921 7256 26</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                      noise1                              noise2  \\\n",
       "0   NL66 RbB 65J01 C137211N3     \\tN  LQZ 66\\r\\rRBRB 6501 131 13   \n",
       "1   oNL47 KNAB2266336 858 02     N47 K\\nNAFB  2\\t63\n",
       "6  8Q586 0\\r   \n",
       "2  nL73 NNBR 1346 2S53275 58  \\rN73NNB\\rA\\t 1\\r3462 32J\\t75 5\\r8   \n",
       "3   NL694 DHT 7609 2913 S25X      \\nNL\n",
       "6\n",
       "4\\r D\n",
       "HBN 7\n",
       "60 O2913 25   \n",
       "4   InJL64 BUYNPQ592 7256 26      N\n",
       "L4 \\tBUQ 592F1 Z725\n",
       "6\\t\n",
       " 26\n",
       "   \n",
       "\n",
       "               noise3                      noise4                    iban  \n",
       "0  NL66RBB651 1371 13  NL66 RBR`B 6;501 1+371% 13  NL66 RBRB 6501 1371 13  \n",
       "1  NL47 KNAB 2636 860  NL47 KNAB \\2636 @\"8~586 02  NL47 KNAB 2636 8586 02  \n",
       "2  NL73 NNBA 1346327   'NL73 NNBA 1346 !;3275 5<8  NL73 NNBA 1346 3275 58  \n",
       "3  NL DHBN 7609 29325  NL64@ DHBN! 7609] 2913 2\"5  NL64 DHBN 7609 2913 25  \n",
       "4  NL64 UNQ 921 256 6  NL)64 BUNQ 5?92{1 72[56 26  NL64 BUNQ 5921 7256 26  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 87,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1709,
     "status": "ok",
     "timestamp": 1520962152606,
     "user": {
      "displayName": "rik",
      "photoUrl": "//lh6.googleusercontent.com/-ZGIi-vN4KMU/AAAAAAAAAAI/AAAAAAAAAFk/Ki-nNio6VVM/s50-c-k-no/photo.jpg",
      "userId": "113719912521762461555"
     },
     "user_tz": -60
    },
    "id": "Vk8QmRX1aItX",
    "outputId": "49aae42b-315a-430a-9af1-3330f227e179"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 200\r\n",
      "-rw-r--r--  1 rik  staff   1060 Mar 14 09:13 LICENSE\r\n",
      "-rw-r--r--@ 1 rik  staff  31159 Mar 14 09:23 Wk6_Ortec_IBAN_generation.ipynb\r\n",
      "-rw-r--r--@ 1 rik  staff  45008 Mar 14 09:46 Jannes_Baseline_Ortec.ipynb\r\n",
      "-rw-r--r--  1 rik  staff    190 Mar 14 10:00 README.md\r\n",
      "drwxr-xr-x  5 rik  staff    170 Mar 14 10:00 \u001b[34mtemplates\u001b[m\u001b[m\r\n",
      "-rw-r--r--  1 rik  staff  12871 Mar 14 10:02 iban_noisy_100.csv\r\n"
     ]
    }
   ],
   "source": [
    "! ls -ltr\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 54,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 8436,
     "status": "ok",
     "timestamp": 1520962352671,
     "user": {
      "displayName": "rik",
      "photoUrl": "//lh6.googleusercontent.com/-ZGIi-vN4KMU/AAAAAAAAAAI/AAAAAAAAAFk/Ki-nNio6VVM/s50-c-k-no/photo.jpg",
      "userId": "113719912521762461555"
     },
     "user_tz": -60
    },
    "id": "pK3pecQsa3-9",
    "outputId": "9ea0b982-fcf6-4b70-9a04-0947e58f63a9"
   },
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'keras'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-18-88d96843a926>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mkeras\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'keras'"
     ]
    }
   ],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "906FTddKa5St"
   },
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'keras' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-19-7d06999eec3d>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mmodel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mkeras\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmodels\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSequential\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlayers\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mDense\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput_size\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m4\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNameError\u001b[0m: name 'keras' is not defined"
     ]
    }
   ],
   "source": [
    "model = keras.models.Sequential()\n",
    "\n",
    "model.add(model.layers.Dense(input_size=4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "default_view": {},
   "name": "Wk6 Ortec IBAN generation.ipynb",
   "provenance": [],
   "version": "0.3.2",
   "views": {}
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}