Computation_landscape_metrics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<center> <font size=10> <b> Table of contents </b> </font> </center> \n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div id=\"toc\"></div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%javascript\n",
    "$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center> <font size=5> <h1>Define working environment</h1> </font> </center> \n",
    "\n",
    "The following cells are used to :\n",
    "- Set the environment variables for Python and GRASS GIS and R statistical computing \n",
    "- Define the [\"GRASSDATA\" folder](https://grass.osgeo.org/grass73/manuals/helptext.html), the name of \"location\" and \"mapset\" where you will to work."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Import libraries**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Import libraries needed for setting parameters of operating system \n",
    "import os\n",
    "import sys"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Set 'Python' and 'GRASS GIS' environment variables**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, we set [the environment variables allowing to use of GRASS GIS](https://grass.osgeo.org/grass64/manuals/variables.html) inside this Jupyter notebook. Please change the directory path according to your own system configuration. Here after are the environment variables defined on a Linux Mint system."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "### Define GRASS GIS environment variables for LINUX UBUNTU Mint 18.1 (Serena)\n",
    "# Check is environmental variables exists and create them (empty) if not exists.\n",
    "if not 'PYTHONPATH' in os.environ:\n",
    "    os.environ['PYTHONPATH']=''\n",
    "if not 'LD_LIBRARY_PATH' in os.environ:\n",
    "    os.environ['LD_LIBRARY_PATH']=''\n",
    "# Set environmental variables\n",
    "os.environ['GISBASE'] = '/home/tais/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-gnu'\n",
    "os.environ['PATH'] += os.pathsep + os.path.join(os.environ['GISBASE'],'bin')\n",
    "os.environ['PATH'] += os.pathsep + os.path.join(os.environ['GISBASE'],'script')\n",
    "os.environ['PATH'] += os.pathsep + os.path.join(os.environ['GISBASE'],'lib')\n",
    "#os.environ['PATH'] += os.pathsep + os.path.join(os.environ['GISBASE'],'etc','python')\n",
    "os.environ['PYTHONPATH'] += os.pathsep + os.path.join(os.environ['GISBASE'],'etc','python')\n",
    "os.environ['PYTHONPATH'] += os.pathsep + os.path.join(os.environ['GISBASE'],'etc','python','grass')\n",
    "os.environ['PYTHONPATH'] += os.pathsep + os.path.join(os.environ['GISBASE'],'etc','python','grass','script')\n",
    "os.environ['PYTHONLIB'] = '/usr/lib/python2.7'\n",
    "os.environ['LD_LIBRARY_PATH'] += os.pathsep + os.path.join(os.environ['GISBASE'],'lib')\n",
    "os.environ['GIS_LOCK'] = '$$'\n",
    "os.environ['GISRC'] = os.path.join(os.environ['HOME'],'.grass7','rc')\n",
    "os.environ['PATH'] += os.pathsep + os.path.join(os.environ['HOME'],'.grass7','addons')\n",
    "os.environ['PATH'] += os.pathsep + os.path.join(os.environ['HOME'],'.grass7','addons','bin')\n",
    "os.environ['PATH'] += os.pathsep + os.path.join(os.environ['HOME'],'.grass7','addons')\n",
    "os.environ['PATH'] += os.pathsep + os.path.join(os.environ['HOME'],'.grass7','addons','scripts')\n",
    "\n",
    "## Define GRASS-Python environment\n",
    "sys.path.append(os.path.join(os.environ['GISBASE'],'etc','python'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Import GRASS Python packages**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Import libraries needed to launch GRASS GIS in the jupyter notebook\n",
    "import grass.script.setup as gsetup\n",
    "\n",
    "## Import libraries needed to call GRASS using Python\n",
    "import grass.script as gscript\n",
    "from grass.script import core as grass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Display current environment variables of your computer**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MDMSESSION = mate \t\n",
      "MANDATORY_PATH = /usr/share/gconf/mate.mandatory.path \t\n",
      "MATE_DESKTOP_SESSION_ID = this-is-deprecated \t\n",
      "LESSOPEN = | /usr/bin/lesspipe %s \t\n",
      "MDM_LANG = fr_BE.UTF-8 \t\n",
      "LOGNAME = tais \t\n",
      "USER = tais \t\n",
      "HOME = /home/tais \t\n",
      "XDG_VTNR = 8 \t\n",
      "PATH = /usr/local/bin:/home/tais/BIN:/home/tais/bin:/home/tais/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/tais/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-gnu/bin:/home/tais/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-gnu/script:/home/tais/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-gnu/lib:/home/tais/.grass7/addons:/home/tais/.grass7/addons/bin:/home/tais/.grass7/addons:/home/tais/.grass7/addons/scripts \t\n",
      "CLICOLOR = 1 \t\n",
      "DISPLAY = :0.0 \t\n",
      "SSH_AGENT_PID = 1996 \t\n",
      "LANG = fr_BE.UTF-8 \t\n",
      "TERM = xterm-color \t\n",
      "SHELL = /bin/bash \t\n",
      "GIS_LOCK = $$ \t\n",
      "XAUTHORITY = /home/tais/.Xauthority \t\n",
      "SESSION_MANAGER = local/tais-HP-Z620-Workstation:@/tmp/.ICE-unix/1925,unix/tais-HP-Z620-Workstation:/tmp/.ICE-unix/1925 \t\n",
      "SHLVL = 1 \t\n",
      "QT_LINUX_ACCESSIBILITY_ALWAYS_ON = 1 \t\n",
      "INSIDE_CAJA_PYTHON =  \t\n",
      "QT_ACCESSIBILITY = 1 \t\n",
      "LD_LIBRARY_PATH = :/home/tais/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-gnu/lib \t\n",
      "COMPIZ_CONFIG_PROFILE = mate \t\n",
      "WINDOWPATH = 8 \t\n",
      "GTK_OVERLAY_SCROLLING = 0 \t\n",
      "PYTHONPATH = :/home/tais/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python:/home/tais/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass:/home/tais/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass/script \t\n",
      "GISBASE = /home/tais/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-gnu \t\n",
      "CLUTTER_BACKEND = x11 \t\n",
      "USERNAME = tais \t\n",
      "XDG_SESSION_DESKTOP = mate \t\n",
      "GDM_XSERVER_LOCATION = local \t\n",
      "XDG_RUNTIME_DIR = /run/user/1000 \t\n",
      "JPY_PARENT_PID = 3933 \t\n",
      "QT_STYLE_OVERRIDE = gtk \t\n",
      "SSH_AUTH_SOCK = /run/user/1000/keyring/ssh \t\n",
      "VTE_VERSION = 4205 \t\n",
      "GDMSESSION = mate \t\n",
      "GISRC = /home/tais/.grass7/rc \t\n",
      "GIT_PAGER = cat \t\n",
      "XDG_CONFIG_DIRS = /etc/xdg/xdg-mate:/etc/xdg \t\n",
      "XDG_CURRENT_DESKTOP = MATE \t\n",
      "XDG_SESSION_ID = c1 \t\n",
      "DBUS_SESSION_BUS_ADDRESS = unix:abstract=/tmp/dbus-F9KfIMoQNU,guid=c6e3bf59efb82a70f504530a5ae1a284 \t\n",
      "_ = /usr/local/bin/jupyter \t\n",
      "XDG_SESSION_COOKIE = 8441891e86e24d76b9616edf516d5734-1524736643.947702-420971996 \t\n",
      "DESKTOP_SESSION = mate \t\n",
      "WINDOWID = 56623110 \t\n",
      "LESSCLOSE = /usr/bin/lesspipe %s %s \t\n",
      "DEFAULTS_PATH = /usr/share/gconf/mate.default.path \t\n",
      "MPLBACKEND = module://ipykernel.pylab.backend_inline \t\n",
      "MDM_XSERVER_LOCATION = local \t\n",
      "GTK_MODULES = gail:atk-bridge \t\n",
      "XDG_DATA_DIRS = /usr/share/mate:/usr/local/share/:/usr/share/:/usr/share/mdm/ \t\n",
      "PWD = /media/tais/data/Dropbox/ULB/MAUPP/Traitements/Landscape_metrics/Landuse_mapping \t\n",
      "COLORTERM = mate-terminal \t\n",
      "PYTHONLIB = /usr/lib/python2.7 \t\n",
      "LS_COLORS = rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: \t\n",
      "PAGER = cat \t\n",
      "XDG_SEAT = seat0 \t\n"
     ]
    }
   ],
   "source": [
    "## Display the current defined environment variables\n",
    "for key in os.environ.keys():\n",
    "    print \"%s = %s \\t\" % (key,os.environ[key])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center> <font size=5> <h1>Define functions</h1> </font> </center> "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This section of the notebook is dedicated to defining functions which will then be called later in the script. If you want to create your own functions, define them here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Function for computing processing time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The \"print_processing_time\" is used to calculate and display the processing time for various stages of the processing chain. At the beginning of each major step, the current time is stored in a new variable, using [time.time() function](https://docs.python.org/2/library/time.html). At the end of the stage in question, the \"print_processing_time\" function is called and takes as argument the name of this new variable containing the recorded time at the beginning of the stage, and an output message."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Import library for managing time in python\n",
    "import time  \n",
    "\n",
    "## Function \"print_processing_time()\" compute processing time and printing it.\n",
    "# The argument \"begintime\" wait for a variable containing the begintime (result of time.time()) of the process for which to compute processing time.\n",
    "# The argument \"printmessage\" wait for a string format with information about the process. \n",
    "def print_processing_time(begintime, printmessage):    \n",
    "    endtime=time.time()           \n",
    "    processtime=endtime-begintime\n",
    "    remainingtime=processtime\n",
    "\n",
    "    days=int((remainingtime)/86400)\n",
    "    remainingtime-=(days*86400)\n",
    "    hours=int((remainingtime)/3600)\n",
    "    remainingtime-=(hours*3600)\n",
    "    minutes=int((remainingtime)/60)\n",
    "    remainingtime-=(minutes*60)\n",
    "    seconds=round((remainingtime)%60,1)\n",
    "\n",
    "    if processtime<60:\n",
    "        finalprintmessage=str(printmessage)+str(seconds)+\" seconds\"\n",
    "    elif processtime<3600:\n",
    "        finalprintmessage=str(printmessage)+str(minutes)+\" minutes and \"+str(seconds)+\" seconds\"\n",
    "    elif processtime<86400:\n",
    "        finalprintmessage=str(printmessage)+str(hours)+\" hours and \"+str(minutes)+\" minutes and \"+str(seconds)+\" seconds\"\n",
    "    elif processtime>=86400:\n",
    "        finalprintmessage=str(printmessage)+str(days)+\" days, \"+str(hours)+\" hours and \"+str(minutes)+\" minutes and \"+str(seconds)+\" seconds\"\n",
    "    \n",
    "    return finalprintmessage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Function for creation of configuration file for r.li (landscape units provided as polygons) (multiprocessed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "##### Function that create the r.li configuration file for a list of landcover raster.\n",
    "### It enable to create in one function as many configuration file as the number of raster provided in 'listoflandcoverraster'.\n",
    "### It could be use only in case study with a several landcover raster and only one landscape unit layer.\n",
    "### So, the landscape unit layer if fixed and there are the landcover raster which change. \n",
    "# 'listoflandcoverraster' wait for a list with the name (string) of landcover rasters.\n",
    "# 'landscape_polygons' wait for the name (string) of the vector layer containing the polygons to be used as landscape units.\n",
    "# 'masklayerhardcopy' wait for a boolean value (True/False) depending if the user want to create hard copy of the landscape units mask layers or not.\n",
    "# 'returnlistpath' wait for a boolean value (True/False) according to the fact that a list containing the path to the configuration files is desired.\n",
    "# 'ncores' wait for a integer corresponding to the number of desired cores to be used for parallelization.\n",
    "\n",
    "# Import libraries for multiprocessing \n",
    "import multiprocessing\n",
    "from multiprocessing import Pool\n",
    "from functools import partial \n",
    "\n",
    "# Function that copy the landscape unit raster masks on a new layer with name corresponding to the current 'landcover_raster'\n",
    "def copy_landscapeunitmasks(current_landcover_raster,base_landcover_raster,landscape_polygons,landscapeunit_bbox,cat):\n",
    "    ### Copy the landscape units mask for the current 'cat'\n",
    "    # Define the name of the current \"current_landscapeunit_rast\" layer\n",
    "    current_landscapeunit_rast=current_landcover_raster.split(\"@\")[0]+\"_\"+landscape_polygons.split(\"@\")[0]+\"_\"+str(cat)          \n",
    "    base_landscapeunit_rast=base_landcover_raster.split(\"@\")[0]+\"_\"+landscape_polygons.split(\"@\")[0]+\"_\"+str(cat)          \n",
    "    # Copy the the landscape unit created for the first landcover map in order to match the name of the current landcover map\n",
    "    gscript.run_command('g.copy', overwrite=True, quiet=True, raster=(base_landscapeunit_rast,current_landscapeunit_rast))\n",
    "    # Add the line to the text variable\n",
    "    text=\"MASKEDOVERLAYAREA \"+current_landscapeunit_rast+\"|\"+landscapeunit_bbox[cat]\n",
    "    return text\n",
    "\n",
    "# Function that create the r.li configuration file for the base landcover raster and then for all the binary rasters\n",
    "def create_rli_configfile(listoflandcoverraster,landscape_polygons,\n",
    "                          masklayerhardcopy=False,returnlistpath=True,ncores=2):\n",
    "    # Check if 'listoflandcoverraster' is not empty\n",
    "    if len(listoflandcoverraster)==0:\n",
    "        sys.exit(\"The list of landcover raster is empty and should contain at least one raster name\")\n",
    "    # Check if rasters provided in 'listoflandcoverraster' exists to avoid error in mutliprocessing \n",
    "    for cur_rast in listoflandcoverraster:\n",
    "        try:\n",
    "            mpset=cur_rast.split(\"@\")[1]\n",
    "        except:\n",
    "            mpset=\"\"\n",
    "        if cur_rast.split(\"@\")[0] not in [x[0] for x in gscript.list_pairs(type='raster',mapset=mpset)]:\n",
    "            sys.exit('Raster <%s> not found' %cur_rast)\n",
    "    # Check if rasters provided in 'listoflandcoverraster' have the same extend and spatial resolution \n",
    "    raster={}\n",
    "    for x, rast in enumerate(raster_list):\n",
    "        raster[x]=gscript.raster_info(rast)\n",
    "    key_list=raster.keys()\n",
    "    for x in key_list[1:]:\n",
    "        for info in ('north','south','east','west','ewres','nsres'):\n",
    "            if not raster[0][info]==raster[x][info]:\n",
    "                sys.exit(\"Some raster provided in the list have different spatial resolution or extend, please check\")    \n",
    "    # Get the version of GRASS GIS \n",
    "    version=grass.version()['version'].split('.')[0]\n",
    "    # Define the folder to save the r.li configuration files\n",
    "    if sys.platform==\"win32\":\n",
    "        rli_dir=os.path.join(os.environ['APPDATA'],\"GRASS\"+version,\"r.li\")\n",
    "    else: \n",
    "        rli_dir=os.path.join(os.environ['HOME'],\".grass\"+version,\"r.li\")\n",
    "    if not os.path.exists(rli_dir):\n",
    "        os.makedirs(rli_dir)\n",
    "    ## Create an ordered list with the 'cat' value of landscape units to be processed.\n",
    "    list_cat=[int(x) for x in gscript.parse_command('v.db.select', quiet=True, \n",
    "                                                        map=landscape_polygons, column='cat', flags='c')]\n",
    "    list_cat.sort()\n",
    "    # Declare a empty dictionnary which will contains the north, south, east, west values for each landscape unit\n",
    "    landscapeunit_bbox={}\n",
    "    # Declare a empty list which will contain the path of the configation files created\n",
    "    listpath=[]\n",
    "    # Declare a empty string variable which will contains the core part of the r.li configuration file\n",
    "    maskedoverlayarea_1=\"\"\n",
    "    # Duplicate 'listoflandcoverraster' in a new variable called 'tmp_list'\n",
    "    tmp_list=list(listoflandcoverraster)\n",
    "    # Set the current landcover raster as the first of the list\n",
    "    base_landcover_raster=tmp_list.pop(0) #The pop function return the first item of the list and delete it from the list at the same time\n",
    "    # Loop trough the landscape units\n",
    "    for cat in list_cat:\n",
    "        # Extract the current landscape unit polygon as temporary vector\n",
    "        tmp_vect=\"tmp_\"+base_landcover_raster.split(\"@\")[0]+\"_\"+landscape_polygons.split(\"@\")[0]+\"_\"+str(cat)\n",
    "        gscript.run_command('v.extract', overwrite=True, quiet=True, \n",
    "                            input=landscape_polygons, cats=cat, output=tmp_vect)\n",
    "        # Set region to match the extent of the current landscape polygon, with resolution and alignement matching the landcover raster\n",
    "        gscript.run_command('g.region', vector=tmp_vect, align=base_landcover_raster)\n",
    "        # Rasterize the landscape unit polygon\n",
    "        landscapeunit_rast=tmp_vect[4:]\n",
    "        gscript.run_command('v.to.rast', overwrite=True, quiet=True, input=tmp_vect, output=landscapeunit_rast, use='cat', memory='3000')\n",
    "        # Remove temporary vector\n",
    "        gscript.run_command('g.remove', quiet=True, flags=\"f\", type='vector', name=tmp_vect)\n",
    "        # Set the region to match the raster landscape unit extent and save the region info in a dictionary\n",
    "        region_info=gscript.parse_command('g.region', raster=landscapeunit_rast, flags='g')\n",
    "        n=str(round(float(region_info['n']),5)) #the config file need 5 decimal for north and south\n",
    "        s=str(round(float(region_info['s']),5))\n",
    "        e=str(round(float(region_info['e']),6)) #the config file need 6 decimal for east and west\n",
    "        w=str(round(float(region_info['w']),6))\n",
    "        # Save the coordinates of the bbox in the dictionary (n,s,e,w)\n",
    "        landscapeunit_bbox[cat]=n+\"|\"+s+\"|\"+e+\"|\"+w\n",
    "        # Add the line to the maskedoverlayarea_1 variable\n",
    "        maskedoverlayarea_1+=\"MASKEDOVERLAYAREA \"+landscapeunit_rast+\"|\"+landscapeunit_bbox[cat]+\"\\n\"\n",
    "\n",
    "    # Compile the content of the r.li configuration file\n",
    "    config_file_content=\"SAMPLINGFRAME 0|0|1|1\\n\"\n",
    "    config_file_content+=maskedoverlayarea_1\n",
    "    config_file_content+=\"RASTERMAP \"+base_landcover_raster+\"\\n\"\n",
    "    config_file_content+=\"VECTORMAP \"+landscape_polygons+\"\\n\"\n",
    "\n",
    "    # Create a new file and save the content\n",
    "    configfilename=base_landcover_raster.split(\"@\")[0]+\"_\"+landscape_polygons.split(\"@\")[0]\n",
    "    path=os.path.join(rli_dir,configfilename)\n",
    "    listpath.append(path)\n",
    "    f=open(path, 'w')\n",
    "    f.write(config_file_content)\n",
    "    f.close()\n",
    "    \n",
    "    # Continue creation of r.li configuration file and landscape unit raster the rest of the landcover raster provided\n",
    "    while len(tmp_list)>0:\n",
    "        # Initialize 'maskedoverlayarea_2' variable as an empty string\n",
    "        maskedoverlayarea_2=\"\"\n",
    "        # Set the current landcover raster as the first of the list\n",
    "        current_landcover_raster=tmp_list.pop(0) #The pop function return the first item of the list and delete it from the list at the same time\n",
    "        if masklayerhardcopy: # If the user asked for hard copy of the landscape units mask layers\n",
    "            # Copy all the landscape units masks for the current landcover raster\n",
    "            p=Pool(ncores) #Create a pool of processes and launch them using 'map' function\n",
    "            func=partial(copy_landscapeunitmasks,current_landcover_raster,base_landcover_raster,landscape_polygons,landscapeunit_bbox) # Set fixed argument of the function\n",
    "            maskedoverlayarea_2=p.map(func,list_cat) # Launch the processes for as many items in the list and get the ordered results using map function\n",
    "            p.close()\n",
    "            p.join()\n",
    "            # Compile the content of the r.li configuration file\n",
    "            config_file_content=\"SAMPLINGFRAME 0|0|1|1\\n\"\n",
    "            config_file_content+=\"\\n\".join(maskedoverlayarea_2)+\"\\n\"\n",
    "            config_file_content+=\"RASTERMAP \"+current_landcover_raster+\"\\n\"\n",
    "            config_file_content+=\"VECTORMAP \"+landscape_polygons+\"\\n\"\n",
    "        else: # If the user not asked for hard copy\n",
    "            # Compile the content of the r.li configuration file\n",
    "            config_file_content=\"SAMPLINGFRAME 0|0|1|1\\n\"\n",
    "            config_file_content+=maskedoverlayarea_1  # If user do not asked for hard copy, the mask layers are the same than for the first configuration file\n",
    "            config_file_content+=\"RASTERMAP \"+current_landcover_raster+\"\\n\"  # But the name of the RASTERMAP should be the one of the current landcover raster\n",
    "            config_file_content+=\"VECTORMAP \"+landscape_polygons+\"\\n\"\n",
    "        # Create a new file and save the content\n",
    "        configfilename=current_landcover_raster.split(\"@\")[0]+\"_\"+landscape_polygons.split(\"@\")[0]\n",
    "        path=os.path.join(rli_dir,configfilename)\n",
    "        listpath.append(path)\n",
    "        f=open(path, 'w')\n",
    "        f.write(config_file_content)\n",
    "        f.close()\n",
    "    \n",
    "    # Return a list of path of configuration files creates if option actived\n",
    "    if returnlistpath:\n",
    "        return list_cat,listpath\n",
    "    else:\n",
    "        return list_cat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Function for creation of binary raster from a categorical raster (multiprocessed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "###### Function creating a binary raster for each category of a base raster. \n",
    "### The function run within the current region. If a category do not exists in the current region, no binary map will be produce\n",
    "# 'categorical_raster' wait for the name of the base raster to be used. It is the one from which one binary raster will be produced for each category value\n",
    "# 'prefix' wait for a string corresponding to the prefix of the name of the binary raster which will be produced\n",
    "# 'setnull' wait for a boolean value (True, False) according to the fact that the output binary should be 1/0 or 1/null\n",
    "# 'returnlistraster' wait for a boolean value (True, False) regarding to the fact that a list containing the name of binary raster is desired as return of the function\n",
    "# 'category_list' wait for a list of interger corresponding to specific category of the base raster to be used \n",
    "# 'ncores' wait for a integer corresponding to the number of desired cores to be used for parallelization\n",
    "\n",
    "# Import libraries for multiprocessing \n",
    "import multiprocessing\n",
    "from multiprocessing import Pool\n",
    "from functools import partial   \n",
    "\n",
    "def create_binary_raster(categorical_raster,prefix=\"binary\",setnull=False,returnlistraster=True,category_list=None,ncores=2):\n",
    "    # Check if raster exists to avoid error in mutliprocessing \n",
    "    try:\n",
    "        mpset=categorical_raster.split(\"@\")[1]\n",
    "    except:\n",
    "        mpset=\"\"\n",
    "    if categorical_raster not in gscript.list_strings(type='raster',mapset=mpset):\n",
    "        sys.exit('Raster <%s> not found' %categorical_raster)\n",
    "    # Check for number of cores doesnt exceed available\n",
    "    nbcpu=multiprocessing.cpu_count()\n",
    "    if ncores>=nbcpu:\n",
    "        ncores=nbcpu-1\n",
    "    returnlist=[] #Declare empty list for return\n",
    "    #gscript.run_command('g.region', raster=categorical_raster, quiet=True) #Set the region\n",
    "    null='null()' if setnull else '0' #Set the value for r.mapcalc\n",
    "    minclass=1 if setnull else 2 #Set the value to check if the binary raster is empty\n",
    "    if category_list == None: #If no category_list provided\n",
    "        category_list=[cl for cl in gscript.parse_command('r.category',map=categorical_raster,quiet=True)]\n",
    "    for i,x in enumerate(category_list):  #Make sure the format is UTF8 and not Unicode\n",
    "        category_list[i]=x.encode('UTF8')\n",
    "    category_list.sort(key=float) #Sort the raster categories in ascending.\n",
    "    p=Pool(ncores) #Create a pool of processes and launch them using 'map' function\n",
    "    func=partial(get_binary,categorical_raster,prefix,null,minclass) # Set the two fixed argument of the function\n",
    "    returnlist=p.map(func,category_list) # Launch the processes for as many items in the 'functions_name' list and get the ordered results using map function\n",
    "    p.close()\n",
    "    p.join()\n",
    "    if returnlistraster:\n",
    "        return returnlist\n",
    "\n",
    "#### Function that extract binary raster for a specified class (called in 'create_binary_raster' function)\n",
    "def get_binary(categorical_raster,prefix,null,minclass,cl):\n",
    "    binary_class=prefix+\"_\"+cl\n",
    "    gscript.run_command('r.mapcalc', expression=binary_class+'=if('+categorical_raster+'=='+str(cl)+',1,'+null+')',overwrite=True, quiet=True)\n",
    "    if len(gscript.parse_command('r.category',map=binary_class,quiet=True))>=minclass:  #Check if created binary is not empty\n",
    "        return binary_class\n",
    "    else:\n",
    "        gscript.run_command('g.remove', quiet=True, flags=\"f\", type='raster', name=binary_class)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Function for computation of spatial metrics at landscape level (multiprocessed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "##### Function that compute different landscape metrics (spatial metrics) at landscape level. \n",
    "### The metric computed are \"dominance\",\"pielou\",\"renyi\",\"richness\",\"shannon\",\"simpson\".\n",
    "### It is important to set the computation region before runing this script so that it match the extent of the 'raster' layer.\n",
    "# 'configfile' wait for the path (string) to the configuration file corresponding to the 'raster' layer.\n",
    "# 'raster' wait for the name (string) of the landcover map on which landscape metrics will be computed.\n",
    "# 'returnlistresult' wait for a boolean value (True/False) according to the fact that a list containing the path to the result files is desired.\n",
    "# 'ncores' wait for a integer corresponding to the number of desired cores to be used for parallelization.\n",
    "\n",
    "# Import libraries for multiprocessing \n",
    "import multiprocessing\n",
    "from multiprocessing import Pool\n",
    "from functools import partial   \n",
    "\n",
    "def compute_landscapelevel_metrics(configfile, raster, spatial_metric):\n",
    "    filename=raster.split(\"@\")[0]+\"_%s\" %spatial_metric\n",
    "    outputfile=os.path.join(os.path.split(configfile)[0],\"output\",filename)\n",
    "    if spatial_metric=='renyi': # The alpha parameter was set to 2 as in https://en.wikipedia.org/wiki/R%C3%A9nyi_entropy\n",
    "        gscript.run_command('r.li.%s' %spatial_metric, overwrite=True,\n",
    "                            input=raster,config=configfile,alpha='2', output=filename)\n",
    "    else:\n",
    "        gscript.run_command('r.li.%s' %spatial_metric, overwrite=True,\n",
    "                    input=raster,config=configfile, output=filename)\n",
    "    return outputfile\n",
    "    \n",
    "def get_landscapelevel_metrics(configfile, raster, returnlistresult=True, ncores=2):\n",
    "    # Check if raster exists to avoid error in mutliprocessing \n",
    "    try:\n",
    "        mpset=raster.split(\"@\")[1]\n",
    "    except:\n",
    "        mpset=\"\"\n",
    "    if raster not in gscript.list_strings(type='raster',mapset=mpset):\n",
    "        sys.exit('Raster <%s> not found' %raster)\n",
    "    # Check if configfile exists to avoid error in mutliprocessing \n",
    "    if not os.path.exists(configfile):\n",
    "        sys.exit('Configuration file <%s> not found' %configfile)\n",
    "    # List of metrics to be computed\n",
    "    spatial_metric_list=[\"dominance\",\"pielou\",\"renyi\",\"richness\",\"shannon\",\"simpson\"]\n",
    "    # Check for number of cores doesnt exceed available\n",
    "    nbcpu=multiprocessing.cpu_count()\n",
    "    if ncores>=nbcpu:\n",
    "        ncores=nbcpu-1\n",
    "        if ncores>len(spatial_metric_list):\n",
    "            ncores=len(spatial_metric_list)  #Adapt number of cores to number of metrics to compute\n",
    "    #Declare empty list for return\n",
    "    returnlist=[] \n",
    "    # Create a new pool\n",
    "    p=Pool(ncores)\n",
    "    # Set the two fixed argument of the 'compute_landscapelevel_metrics' function\n",
    "    func=partial(compute_landscapelevel_metrics,configfile, raster)\n",
    "    # Launch the processes for as many items in the 'functions_name' list and get the ordered results using map function\n",
    "    returnlist=p.map(func,spatial_metric_list)\n",
    "    p.close()\n",
    "    p.join()\n",
    "    # Return list of paths to result files\n",
    "    if returnlistresult:\n",
    "        return returnlist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Function for computation of spatial metrics at class level (multiprocessed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "##### Function that compute different landscape metrics (spatial metrics) at class level. \n",
    "### The metric computed are \"patch number (patchnum)\",\"patch density (patchdensity)\",\"mean patch size(mps)\",\n",
    "### \"coefficient of variation of patch area (padcv)\",\"range of patch area size (padrange)\",\n",
    "### \"standard deviation of patch area (padsd)\", \"shape index (shape)\", \"edge density (edgedensity)\".\n",
    "### It is important to set the computation region before runing this script so that it match the extent of the 'raster' layer.\n",
    "# 'configfile' wait for the path (string) to the configuration file corresponding to the 'raster' layer.\n",
    "# 'raster' wait for the name (string) of the landcover map on which landscape metrics will be computed.\n",
    "# 'returnlistresult' wait for a boolean value (True/False) according to the fact that a list containing the path to the result files is desired.\n",
    "# 'ncores' wait for a integer corresponding to the number of desired cores to be used for parallelization.\n",
    "\n",
    "# Import libraries for multiprocessing \n",
    "import multiprocessing\n",
    "from multiprocessing import Pool\n",
    "from functools import partial   \n",
    "\n",
    "def compute_classlevel_metrics(configfile, raster, spatial_metric):\n",
    "    filename=raster.split(\"@\")[0]+\"_%s\" %spatial_metric\n",
    "    gscript.run_command('r.li.%s' %spatial_metric, overwrite=True,\n",
    "                        input=raster,config=configfile,output=filename)\n",
    "    outputfile=os.path.join(os.path.split(configfile)[0],\"output\",filename)\n",
    "    return outputfile\n",
    "    \n",
    "def get_classlevel_metrics(configfile, raster, returnlistresult=True, ncores=2):\n",
    "    # Check if raster exists to avoid error in mutliprocessing \n",
    "    try:\n",
    "        mpset=raster.split(\"@\")[1]\n",
    "    except:\n",
    "        mpset=\"\"\n",
    "    if raster not in [x.split(\"@\")[0] for x in gscript.list_strings(type='raster',mapset=mpset)]:\n",
    "        sys.exit('Raster <%s> not found' %raster)\n",
    "    # Check if configfile exists to avoid error in mutliprocessing \n",
    "    if not os.path.exists(configfile):\n",
    "        sys.exit('Configuration file <%s> not found' %configfile)\n",
    "    # List of metrics to be computed\n",
    "    spatial_metric_list=[\"patchnum\",\"patchdensity\",\"mps\",\"padcv\",\"padrange\",\"padsd\",\"shape\",\"edgedensity\"]\n",
    "    # Check for number of cores doesnt exceed available\n",
    "    nbcpu=multiprocessing.cpu_count()\n",
    "    if ncores>=nbcpu:\n",
    "        ncores=nbcpu-1\n",
    "        if ncores>len(spatial_metric_list):\n",
    "            ncores=len(spatial_metric_list)  #Adapt number of cores to number of metrics to compute\n",
    "    # Declare empty list for return\n",
    "    returnlist=[] \n",
    "    # Create a new pool\n",
    "    p=Pool(ncores)\n",
    "    # Set the two fixed argument of the 'compute_classlevel_metrics' function\n",
    "    func=partial(compute_classlevel_metrics,configfile, raster)\n",
    "    # Launch the processes for as many items in the 'functions_name' list and get the ordered results using map function\n",
    "    returnlist=p.map(func,spatial_metric_list)\n",
    "    p.close()\n",
    "    p.join()\n",
    "    # Return list of paths to result files\n",
    "    if returnlistresult:\n",
    "        return returnlist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Function for computation of each land cover class proportion in landscape units"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "### Function that compute the proportion of each class of landcover\n",
    "import itertools\n",
    "import multiprocessing\n",
    "from functools import partial\n",
    "import sys, csv\n",
    "import grass.script as gscript\n",
    "\n",
    "def random_string(N):\n",
    "    import random, string\n",
    "    prefix=random.choice(string.ascii_uppercase + string.ascii_lowercase)\n",
    "    suffix=''.join(random.choice(string.ascii_uppercase + string.ascii_lowercase + string.digits) for _ in range(N))\n",
    "    return prefix+suffix\n",
    "\n",
    "def compute_prop_landcover(outputfolder,landscape_units_raster,binary_class_raster):\n",
    "    # Save the current class in a new variable \n",
    "    current_class=binary_class_raster.split(\"@\")[0].split(\"_\")[-1]\n",
    "    # Initialize a temp files\n",
    "    temp_file=os.path.join(outputfolder,random_string(10))\n",
    "    tmp_copy='%s_%s'%(random_string(4),current_class)\n",
    "    # Make copy of the raster and fill the null values with zero\n",
    "    gscript.run_command('g.copy', overwrite=True, raster='%s,%s'%(binary_class_raster,tmp_copy))\n",
    "    gscript.run_command('r.null', map=tmp_copy, null='0')\n",
    "    # Compute area of landscape unit and total pixels of the current binary class raster\n",
    "    gscript.run_command('i.segment.stats', flags=\"r\", quiet=True, overwrite=True, \n",
    "                        map=landscape_units_raster, area_measures='area', \n",
    "                        rasters=tmp_copy, raster_statistics='sum', \n",
    "                        csvfile=temp_file, separator='pipe')\n",
    "    gscript.run_command('g.remove', flags='f', type='raster', name=tmp_copy)\n",
    "    # Create outputfile and compute proportion of class\n",
    "    outfile=os.path.join(outputfolder, \"prop_%s\"%current_class)\n",
    "    writer=csv.writer(open(outfile,'w'), delimiter=\"|\")\n",
    "    reader=csv.reader(open(temp_file,'r'), delimiter=\"|\")\n",
    "    header=reader.next()\n",
    "    crash_report=[]\n",
    "    content=[]\n",
    "    content.append([header[0],\"prop_%s\"%current_class])  #Create new header with first original column and current class related name for proportion\n",
    "    for row in reader:\n",
    "        try:\n",
    "            prop=100*float(row[2])/float(row[1])\n",
    "            content.append([row[0],\"{0:.5f}\".format(prop)])\n",
    "        except ZeroDivisionError:\n",
    "            crash_report.append(row[0])\n",
    "            continue\n",
    "    writer.writerows(content)\n",
    "    os.remove(temp_file)\n",
    "    # Print notification of ZeroDivisionError if it happened\n",
    "    if len(crash_report)>0:\n",
    "        print \"An 'ZeroDivisionError' has been registered for the following <%s>\"%header[0]+\"\\n\".join(crash_report)\n",
    "    # Return the result file\n",
    "    return outfile\n",
    "    \n",
    "def get_classproportions(outputfolder,landscape_units_raster,binary_class_raster_list,\n",
    "                         returnlistresult=True,ncores=2):\n",
    "    # Check if raster exists to avoid error in mutliprocessing \n",
    "    try:\n",
    "        mpset=landscape_units_raster.split(\"@\")[1]\n",
    "    except:\n",
    "        mpset=\"\"\n",
    "    if landscape_units_raster not in [x.split(\"@\")[0] for x in gscript.list_strings(type='raster',mapset=mpset)]:\n",
    "        sys.exit('Raster <%s> not found' %landscape_units_raster)\n",
    "    # Check for number of cores doesnt exceed available\n",
    "    nbcpu=multiprocessing.cpu_count()\n",
    "    if ncores>=nbcpu:\n",
    "        ncores=nbcpu-1\n",
    "        if ncores>len(binary_class_raster_list):\n",
    "            ncores=len(binary_class_raster_list)  #Adapt number of cores to number of metrics to compute\n",
    "    # Declare empty list for return\n",
    "    returnlist=[] \n",
    "    # Create a new pool\n",
    "    p=Pool(ncores)\n",
    "    # Set the two fixed argument of the 'compute_prop_landcover' function\n",
    "    func=partial(compute_prop_landcover,outputfolder,landscape_units_raster)\n",
    "    # Launch the processes for as many items in the 'functions_name' list and get the ordered results using map function\n",
    "    returnlist=p.map(func,binary_class_raster_list)\n",
    "    p.close()\n",
    "    p.join()\n",
    "    # Return list of paths to result files\n",
    "    if returnlistresult:\n",
    "        return returnlist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Function that enable for join of multiple different .csv files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def atoi(text):\n",
    "    '''\n",
    "    Function that return integer if text is digit - Used in 'natural_keys' function\n",
    "    '''\n",
    "    return int(text) if text.isdigit() else text\n",
    "\n",
    "def natural_keys(text):   #     Trick was found here: https://stackoverflow.com/questions/5967500/how-to-correctly-sort-a-string-with-a-number-inside\n",
    "    '''\n",
    "    Return key to be used for sorting string containing numerical values - Used in 'leftjoin_2csv' function\n",
    "    '''\n",
    "    import re  #Import needed library\n",
    "    return [ atoi(c) for c in re.split('(\\d+)', text) ]  #Split the string\n",
    "\n",
    "def join_2csv(file1,file2,separator=\";\",join='inner',fillempty='NULL'):\n",
    "    '''\n",
    "    Function that join two csv files according to the first column (primary key).\n",
    "    'file1' and 'file2' wait for complete path (strings) to the corresponding files. Please not that 'file1' is assume to be the left-one in the join\n",
    "\t'separator' wait for the character to be considered as .csv delimiter (string)\n",
    "\t'join' parameter wait either for 'left' or 'inner' according to type of join\n",
    "\t'fillempty' wait for the string to be use to fill the blank when no occurance is found for the join operation\n",
    "    '''\n",
    "    import tempfile,csv,os\n",
    "    header_list=[]\n",
    "    file1_values_dict={}\n",
    "    file2_values_dict={}\n",
    "    reader1=csv.reader(open(file1), delimiter=separator) #Csv reader for file 1\n",
    "    reader2=csv.reader(open(file2), delimiter=separator) #Csv reader for file 2\n",
    "    # Make a list of headers\n",
    "    header_list1=[ x for x in reader1.next()]\n",
    "    header_list2=[ x for x in reader2.next()[1:]]\n",
    "    # Make a list of unique IDs from the first and second table according to type of join\n",
    "    if join=='inner':\n",
    "        id_list=[ row[0] for row in reader1]\n",
    "        [id_list.append(row[0]) for row in reader2]\n",
    "        id_list=list(set(id_list))\n",
    "        id_list.sort(key=natural_keys)\n",
    "    if join=='left':\n",
    "        id_list=[ row[0] for row in reader1]\n",
    "        id_list=list(set(id_list))\n",
    "        id_list.sort(key=natural_keys)\n",
    "    # Build dictionnary for values of file 1\n",
    "    reader1=csv.reader(open(file1), delimiter=separator)\n",
    "    reader1.next()\n",
    "    values_dict1={rows[0]:rows[1:] for rows in reader1}\n",
    "    # Build dictionnary for values of file 2\n",
    "    reader2=csv.reader(open(file2), delimiter=separator)\n",
    "    reader2.next()\n",
    "    values_dict2={rows[0]:rows[1:] for rows in reader2}\n",
    "    # Built new content\n",
    "    new_content=[]\n",
    "    new_header=header_list1+header_list2\n",
    "    new_content.append(new_header)\n",
    "    for key in id_list:\n",
    "        new_row=[key]\n",
    "        try:\n",
    "            [new_row.append(value) for value in values_dict1[key]]\n",
    "        except:\n",
    "            [new_row.append('%s'%fillempty) for x in header_list1[1:]]\n",
    "        try:\n",
    "            [new_row.append(value) for value in values_dict2[key]]\n",
    "        except:\n",
    "            [new_row.append('%s'%fillempty) for x in header_list2]\n",
    "        new_content.append(new_row)\n",
    "    #Return the result\n",
    "    outfile=os.path.join(tempfile.gettempdir(),\"temp\")\n",
    "    writer=csv.writer(open(outfile,\"w\"), delimiter=separator)\n",
    "    writer.writerows(new_content) #Write multiples rows in the file\n",
    "    return outfile\n",
    "\n",
    "def join_multiplecsv(fileList,outfile,separator=\";\",join='inner', fillempty='NULL', overwrite=False):\n",
    "    '''\n",
    "    Function that apply join on multiple csv files\n",
    "    '''\n",
    "    import os, sys, shutil\n",
    "    # Stop execution if outputfile exitst and can not be overwriten\n",
    "    if os.path.isfile(outfile) and overwrite==False:\n",
    "        print \"File '%s' aleady exists and overwrite option is not enabled.\"%outfile\n",
    "    else:\n",
    "        if os.path.isfile(outfile) and overwrite==True:  # If outputfile exitst and can be overwriten\n",
    "            #os.remove(outfile)\n",
    "            print \"File '%s' will be overwrited.\"%outfile\n",
    "        nbfile=len(fileList)\n",
    "        if nbfile<=1: #Check if there are at least 2 files in the list\n",
    "            sys.exit(\"This function require at least two .csv files to be jointed together.\")\n",
    "        # Copy the list of file in a queue list\n",
    "        queue_list=list(fileList)\n",
    "        # Left join on the two first files\n",
    "        file1=queue_list.pop(0)\n",
    "        file2=queue_list.pop(0)\n",
    "        tmp_file=join_2csv(file1,file2,separator=separator,join=join, fillempty=fillempty)\n",
    "        # Left join on the rest of the files in the list\n",
    "        while len(queue_list)>0:\n",
    "            file2=queue_list.pop(0)\n",
    "            tmp_file=join_2csv(tmp_file,file2,separator=separator,join=join, fillempty=fillempty)\n",
    "        #Copy the temporary file to the desired output path\n",
    "        shutil.copy2(tmp_file,outfile)\n",
    "        # Print what happend\n",
    "        print \"%s individual .csv files were joint together.\"%nbfile\n",
    "\n",
    "def create_csvt(csv_file,separator=\";\",first_col_type=\"Integer\",rest_type=\"Real\"):\n",
    "    '''\n",
    "    Function that create a .csvt file with the same type of all columns except first one\n",
    "    '''\n",
    "    import csv\n",
    "    writer=csv.writer(open(csv_file+\"t\",\"w\"),delimiter=separator)\n",
    "    reader=csv.reader(open(csv_file,\"r\"),delimiter=separator)\n",
    "    header=reader.next()\n",
    "    typecolumn=[]\n",
    "    typecolumn.append(first_col_type)\n",
    "    for columns in header[1:]:\n",
    "        typecolumn.append(rest_type)\n",
    "    writer.writerow(typecolumn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Function to concatenate .csv files together"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Function which concatenate individual .csv files stored in a folder. \n",
    "# 'indir' parameter wait for a string with the path to the repertory to look for individual .csv files\n",
    "# 'pattern' parameter wait for a string with the pattern of filename to look for (example: \"TMP_*.csv\")\n",
    "# 'sep' parameter wait for a string with the delimiter of the .csv file\n",
    "# 'replacedict' parameter wait for a dictionary containing the unwanted values as keys and the replace string as corresponding value\n",
    "# 'outputfilename' parameter wait for the name of the outputfile (including extansion but without the complete path)\n",
    "\n",
    "import os\n",
    "import glob\n",
    "import csv\n",
    "\n",
    "def concat_findreplace(indir,pattern,sep,replacedict,outputfilename):\n",
    "    # Initialise some variables\n",
    "    messagetoprint=None\n",
    "    returnmessage=None\n",
    "    countdict={}\n",
    "    for k in replacedict:\n",
    "        countdict[k]=0\n",
    "    # Change the working directory\n",
    "    os.chdir(indir) \n",
    "    # Get a list with file in the current directory corresponding to a specific pattern\n",
    "    fileList=None\n",
    "    fileList=glob.glob(pattern)\n",
    "    # Print\n",
    "    messagetoprint=\"Going to concatenate \"+str(len(fileList))+\" .csv files together and replace unwanted values.\"\n",
    "    #print (messagetoprint)\n",
    "    returnmessage=messagetoprint\n",
    "    # Create a new file \n",
    "    outfile=os.path.join(indir, outputfilename)\n",
    "    writercsvSubset = open(outfile, 'wb')\n",
    "    writercsv=csv.writer(writercsvSubset,delimiter=sep)\n",
    "    # Concatenate individuals files and replace unwanted values\n",
    "    for indivfile in fileList:\n",
    "        with open(indivfile) as readercsvSubset:\n",
    "            readercsv=csv.reader(readercsvSubset, delimiter=sep)\n",
    "            if indivfile!=fileList[0]:\n",
    "                readercsv.next()\n",
    "            count=0\n",
    "            for row in readercsv:\n",
    "                newline=[]\n",
    "                for i, x in enumerate(row):\n",
    "                    if x in replacedict:\n",
    "                        newline.append(replacedict[x])\n",
    "                        countdict[x]+=1\n",
    "                    else:\n",
    "                        newline.append(row[i])\n",
    "                writercsv.writerow(newline)\n",
    "        # Close the current input file\n",
    "        readercsvSubset.close()\n",
    "    # Close the outputfile\n",
    "    writercsvSubset.close()\n",
    "    # Count number of changes\n",
    "    countchange=0\n",
    "    for k in countdict:\n",
    "        countchange+=countdict[k]\n",
    "    # Print\n",
    "    if countchange>0:\n",
    "        messagetoprint=\"\\n\"\n",
    "        returnmessage+=messagetoprint\n",
    "        messagetoprint=\"Values have been changed:\"+\"\\n\"\n",
    "        #print (messagetoprint)\n",
    "        returnmessage+=messagetoprint\n",
    "        for k in replacedict:\n",
    "            if countdict[k]>0:\n",
    "                messagetoprint=str(countdict[k])+\" '\"+k+\"' value(s) replaced by '\"+replacedict[k]+\"'\\n\"\n",
    "                #print (messagetoprint)\n",
    "                returnmessage+=messagetoprint\n",
    "    else:\n",
    "        messagetoprint=\"Nothing changed. No unwanted values found !\"\n",
    "        #print (messagetoprint)\n",
    "        returnmessage+=messagetoprint\n",
    "    # Return\n",
    "    return returnmessage[:-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Function to zip files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "'''\n",
    "Define function for creation of zip archive\n",
    "Inspired from https://thispointer.com/python-how-to-create-a-zip-archive-from-multiple-files-or-directory/\n",
    "and from https://github.com/tgrippa/Semi_automated_OBIA_processing_with_local_USPO/blob/3275c61534aec6a33d77b39935727b9ff74178f1/Postclassification/Postclassification_Ouaga_FullAOI.ipynb\n",
    "'''\n",
    "import zipfile\n",
    "from zipfile import ZipFile\n",
    "\n",
    "# Function that zip a list of files in one zip folder\n",
    "def zipListFiles(list_files, zipFileName): \n",
    "    try:\n",
    "        # Create a ZipFile Object\n",
    "        with ZipFile(zipFileName, 'w', zipfile.ZIP_DEFLATED) as zipObj:\n",
    "            # Add multiple files to the zip\n",
    "            for f in list_files:\n",
    "                zipObj.write(f,os.path.basename(f))\n",
    "        # close the Zip File\n",
    "        zipObj.close()\n",
    "    except zipfile.LargeZipFile:\n",
    "        # Create a Large ZipFile Object\n",
    "        with ZipFile(zipFileName, 'w', zipfile.ZIP_DEFLATED, allowZip64=True) as zipObj:\n",
    "            # Add multiple files to the zip\n",
    "            for f in list_files:\n",
    "                zipObj.write(f,os.path.basename(f))\n",
    "        # close the Zip File\n",
    "        zipObj.close()\n",
    "\n",
    "# Function that zip the files from given directory that matches the filter\n",
    "def zipFilesInDir(dirName, zipFileName, filter=\"\"):\n",
    "    # create a ZipFile object\n",
    "    with ZipFile(zipFileName, 'w') as zipObj:\n",
    "        # Iterate over all the files in directory\n",
    "        for folderName, subfolders, filenames in os.walk(dirName):\n",
    "            for filename in filenames:\n",
    "                if not filter:\n",
    "                    # create complete filepath of file in directory\n",
    "                    filePath = os.path.join(folderName, filename)\n",
    "                    locationInZip = os.path.join(os.path.relpath(folderName,os.path.split(dirName)[0]), filename)\n",
    "                    # Add file to zip\n",
    "                    zipObj.write(filePath, locationInZip)\n",
    "                elif filter(filename):\n",
    "                    # create complete filepath of file in directory\n",
    "                    filePath = os.path.join(folderName, filename)\n",
    "                    locationInZip = os.path.join(os.path.relpath(folderName,os.path.split(dirName)[0]), filename)\n",
    "                    # Add file to zip\n",
    "                    zipObj.write(filePath, locationInZip)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center> <font size=5> <h1>User inputs</h1> </font> </center> "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Define a empty dictionnary for saving user inputs\n",
    "user={}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Enter the path to GRASSDATA folder\n",
    "user[\"gisdb\"] = \"/media/tais/My_Book_1/MAUPP/Traitement/Ouagadougou/Segmentation_fullAOI_localapproach/GRASSDATA\"\n",
    "\n",
    "## Enter the name of the location (existing or for a new one)\n",
    "user[\"location\"] = \"Ouaga_32630\"\n",
    "\n",
    "## Enter the EPSG code for this location \n",
    "user[\"locationepsg\"] = \"32630\"\n",
    "\n",
    "## Enter the name of the mapset to use for segmentation\n",
    "user[\"mapsetname\"] = \"Landuse_classif\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Compute landscape metrics (spatial metrics) from the land cover map, using r.li"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Launch GRASS GIS working session**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You are now working in mapset 'Landuse_classif'\n"
     ]
    }
   ],
   "source": [
    "## Set the name of the mapset in which to work\n",
    "mapsetname=user[\"mapsetname\"]\n",
    "\n",
    "## Launch GRASS GIS working session in the mapset\n",
    "if os.path.exists(os.path.join(user[\"gisdb\"],user[\"location\"],mapsetname)):\n",
    "    gsetup.init(os.environ['GISBASE'], user[\"gisdb\"], user[\"location\"], mapsetname)\n",
    "    print \"You are now working in mapset '\"+mapsetname+\"'\" \n",
    "else: \n",
    "    print \"'\"+mapsetname+\"' mapset doesn't exists in \"+user[\"gisdb\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Set the path to the r.li folder for configuration file and for results**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GRASS GIS add-on's r.li configuration files will be saved under </home/tais/.grass7/r.li>.\n",
      "GRASS GIS add-on's r.li outputs will be saved under </home/tais/.grass7/r.li/output>.\n"
     ]
    }
   ],
   "source": [
    "# Define path of the outputfile (in r.li folder)\n",
    "version=grass.version()['version'].split('.')[0] # Get the version of GRASS GIS \n",
    "if sys.platform==\"win32\":\n",
    "    rli_config_dir=os.path.join(os.environ['APPDATA'],\"GRASS\"+version,\"r.li\")\n",
    "    rli_output_dir=os.path.join(os.environ['APPDATA'],\"GRASS\"+version,\"r.li\",\"output\")\n",
    "else: \n",
    "    rli_config_dir=os.path.join(os.environ['HOME'],\".grass\"+version,\"r.li\")\n",
    "    rli_output_dir=os.path.join(os.environ['HOME'],\".grass\"+version,\"r.li\",\"output\")\n",
    "if not os.path.exists(rli_config_dir):\n",
    "    os.makedirs(rli_config_dir)\n",
    "if not os.path.exists(rli_output_dir):\n",
    "    os.makedirs(rli_output_dir)\n",
    "# Print\n",
    "print \"GRASS GIS add-on's r.li configuration files will be saved under <%s>.\"%(rli_config_dir,)\n",
    "print \"GRASS GIS add-on's r.li outputs will be saved under <%s>.\"%(rli_output_dir,)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Define the name of the base landcover map and landscape units polygons"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Set the name of the 'base' landcover map\n",
    "baselandcoverraster=\"postclassif_map_6_shadowback@postclassification\"\n",
    "# Set the name of the vector polygon layer containing the landscape units\n",
    "landscape_polygons=\"streetblocks\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import shapefile containing street blocks polygons"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Set the path to the shapefile containing streetblocks polygons\n",
    "pathtoshp=\"/media/tais/data/Dropbox/ULB/MAUPP/Landuse_mapping/City_block_extraction/Ouagadougou/20170822/Street_blocks_snap7_manualfix_2smallartifact.shp\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Import shapefile\n",
    "gscript.run_command('v.in.ogr', quiet=True, overwrite=True, input=pathtoshp, output=landscape_polygons)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Create binary rasters from the base landcover map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Extraction of binary rasters achieved in 4 minutes and 59.6 seconds'"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save time for computing processin time\n",
    "begintime=time.time()\n",
    "\n",
    "# Create as many binary raster layer as categorical values existing in the base landcover map\n",
    "gscript.run_command('g.region', raster=baselandcoverraster, quiet=True) #Set the region\n",
    "pref=baselandcoverraster.split(\"@\")[0]+\"_cl\"  #Set the prefix\n",
    "\n",
    "raster_list=[]  # Initialize a empty list for results\n",
    "raster_list=create_binary_raster(baselandcoverraster,\n",
    "                                 prefix=pref,setnull=True,returnlistraster=True,\n",
    "                                 category_list=None,ncores=15)  #Extract binary raster \n",
    "\n",
    "# Compute and print processing time\n",
    "print_processing_time(begintime,\"Extraction of binary rasters achieved in \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'raster_list' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-19-cf44cb1b4268>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Insert the name of the base landcover map at first position in the list\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mraster_list\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minsert\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mbaselandcoverraster\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0;31m# Display the raster to be used for landscape analysis\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mraster_list\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNameError\u001b[0m: name 'raster_list' is not defined"
     ]
    }
   ],
   "source": [
    "# Insert the name of the base landcover map at first position in the list\n",
    "raster_list.insert(0,baselandcoverraster)\n",
    "# Display the raster to be used for landscape analysis\n",
    "raster_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "raster_list=['postclassif_map_6_shadowback@postclassification',\n",
    " 'postclassif_map_6_shadowback_cl_13',\n",
    " 'postclassif_map_6_shadowback_cl_14',\n",
    " 'postclassif_map_6_shadowback_cl_20',\n",
    " 'postclassif_map_6_shadowback_cl_30',\n",
    " 'postclassif_map_6_shadowback_cl_31',\n",
    " 'postclassif_map_6_shadowback_cl_41',\n",
    " 'postclassif_map_6_shadowback_cl_51',\n",
    " 'postclassif_map_6_shadowback_cl_111',\n",
    " 'postclassif_map_6_shadowback_cl_112']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Create r.li configuration file for a list of landcover rasters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Creation of r.li configuration files achieved in 12 hours and 59 minutes and 37.6 seconds'"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save time for computing processin time\n",
    "begintime=time.time()\n",
    "# Run creation of r.li configuration file and associated raster layers\n",
    "list_cats,list_configfile=create_rli_configfile(raster_list,landscape_polygons,masklayerhardcopy=False,returnlistpath=True,ncores=20)\n",
    "# Compute and print processing time\n",
    "print_processing_time(begintime,\"Creation of r.li configuration files achieved in \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Display the list of 'cat' values\n",
    "print \"There are %s 'cat' values in the list\"%len(list_cats)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create a archive .py script that store the list of cats\n",
    "f=open(os.path.join(rli_config_dir,'list_cats.py'),'w')\n",
    "content = \"list_cats = [%s]\"%','.join([str(a) for a in list_cats])\n",
    "f.write(content)\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "%run /home/tais/.grass7/r.li/list_cats.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_streetblocks',\n",
       " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_13_streetblocks',\n",
       " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_14_streetblocks',\n",
       " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_20_streetblocks',\n",
       " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_30_streetblocks',\n",
       " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_31_streetblocks',\n",
       " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_41_streetblocks',\n",
       " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_51_streetblocks',\n",
       " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_111_streetblocks',\n",
       " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_112_streetblocks']"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Display the path to the configuration files created\n",
    "list_configfile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Display the path to the configuration files created\n",
    "list_configfile=[u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_streetblocks',\n",
    " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_13_streetblocks',\n",
    " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_14_streetblocks',\n",
    " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_20_streetblocks',\n",
    " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_30_streetblocks',\n",
    " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_31_streetblocks',\n",
    " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_41_streetblocks',\n",
    " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_51_streetblocks',\n",
    " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_111_streetblocks',\n",
    " u'/home/tais/.grass7/r.li/postclassif_map_6_shadowback_cl_112_streetblocks']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Compute spatial metrics at landscape level"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Initialize an empty list which will contains the resultfiles \n",
    "resultfiles=[]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Computation of spatial metric achieved in 3 hours and 55 minutes and 12.1 seconds'"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save time for computing processin time\n",
    "begintime=time.time()\n",
    "# Get the path to the configuration file for the base landcover raster\n",
    "currentconfigfile=list_configfile[0]\n",
    "# Get the name of the base landcover raster\n",
    "currentraster=raster_list[0]\n",
    "# Set the region to match the extent of the base raster\n",
    "gscript.run_command('g.region', raster=currentraster, quiet=True)\n",
    "# Launch the processes for as many items in the 'functions_name' list and get the ordered results using map function\n",
    "resultfiles.append(get_landscapelevel_metrics(currentconfigfile, currentraster, returnlistresult=True, ncores=15))\n",
    "# Compute and print processing time\n",
    "print_processing_time(begintime,\"Computation of spatial metric achieved in \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_dominance',\n",
       "  u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_pielou',\n",
       "  u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_renyi',\n",
       "  u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_richness',\n",
       "  u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_shannon',\n",
       "  u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_simpson']]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "resultfiles"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Compute spatial metrics at class level"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Computation of spatial metric achieved in 1 days, 15 hours and 46 minutes and 16.8 seconds'"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save time for computing processin time\n",
    "begintime=time.time()\n",
    "# Get a list with paths to the configuration file for class level metrics\n",
    "classlevelconfigfiles=list_configfile[1:]\n",
    "# Get a list with name of binary landcover raster for class level metrics\n",
    "classlevelrasters=raster_list[1:]\n",
    "\n",
    "for x,currentraster in enumerate(classlevelrasters[:]):\n",
    "    # Get the path to the configuration file for the base landcover raster\n",
    "    currentconfigfile=classlevelconfigfiles[x]\n",
    "    # Set the region to match the extent of the base raster\n",
    "    gscript.run_command('g.region', raster=currentraster, quiet=True)\n",
    "    # Launch the processes for as many items in the 'functions_name' list and get the ordered results using map function\n",
    "    resultfiles.append(get_classlevel_metrics(currentconfigfile, currentraster, returnlistresult=True, ncores=10))\n",
    "\n",
    "# Compute and print processing time\n",
    "print_processing_time(begintime,\"Computation of spatial metric achieved in \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'resultfiles' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-22-fa437988379b>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mresultfiles\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m: name 'resultfiles' is not defined"
     ]
    }
   ],
   "source": [
    "resultfiles"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Change the results files from r.li to get the correct 'cat' value for each landscape unit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Flat the 'resultfiles' list which contains several lists\n",
    "resultfiles=[item for sublist in resultfiles for item in sublist]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Zip output of r.li for backup \n",
    "zipListFiles(resultfiles,os.path.join(rli_output_dir,\"rli_output.zip\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import csv, shutil\n",
    "from itertools import izip\n",
    "\n",
    "for f in resultfiles:\n",
    "    f_in=open(f)\n",
    "    f_tmp=open(f+'_tmp',\"w\")\n",
    "    f_in_reader=csv.reader(f_in,delimiter='|')\n",
    "    f_tmp_writer=csv.writer(f_tmp,delimiter='|')\n",
    "    f_tmp_writer.writerow(['cat',\"_\".join(os.path.split(f)[-1].split(\"_\")[1:])])\n",
    "    for i,row in enumerate(f_in_reader):\n",
    "        newrow=[]\n",
    "        try:\n",
    "            newrow.append(list_cats[i])\n",
    "            newrow.append(row[1])\n",
    "            f_tmp_writer.writerow(newrow)\n",
    "        except:\n",
    "            continue\n",
    "    f_in.close()\n",
    "    f_tmp.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for f in resultfiles:\n",
    "    os.remove(f)\n",
    "    shutil.copy2(f+'_tmp',f)\n",
    "    os.remove(f+'_tmp')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Zip output of r.li for backup \n",
    "zipListFiles(resultfiles,os.path.join(rli_output_dir,\"spatial_metrics_backup.zip\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Compute some special metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Creating a raster layer of landscape units"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Creating a raster layer of landscape units with resolution corresponding to the one of land cover raster"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a raster corresponding to the landscape units (for computing statistics using i.segment.stats)\n",
    "gscript.run_command('g.region', raster=baselandcoverraster, quiet=True) #Set the region\n",
    "raster_landscapeunits=\"temp_%s\"%landscape_polygons.split(\"@\")[0]\n",
    "gscript.run_command('v.to.rast', overwrite=True, input=landscape_polygons, output=raster_landscapeunits, use='cat') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Creating a raster layer with \"tiles\" to be used for computing statistics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Set the name of the layer to be used as tiling for computing landscape units' statistics (using i.segment.stats)\n",
    "tile_layer=\"tile\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a raster corresponding to the landscape units (for computing statistics using i.segment.stats)\n",
    "gscript.run_command('g.region', raster=baselandcoverraster, quiet=True) #Set the region\n",
    "raster_landscapeunits=\"temp_%s\"%landscape_polygons.split(\"@\")[0]\n",
    "gscript.run_command('v.to.rast', overwrite=True, input=landscape_polygons, \n",
    "                    output=tile_layer, use='attr', attribute_column='morpho_typ')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Save the list of polygons to be processed (save the 'cat' value)\n",
    "listofregion=list(grass.parse_command('v.db.select', map=landscape_polygons, \n",
    "                                      columns='morpho_typ', flags='c'))[:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 cat:24\n",
      "1 cat:11\n",
      "2 cat:13\n",
      "3 cat:12\n",
      "4 cat:14\n",
      "5 cat:22\n",
      "6 cat:23\n",
      "7 cat:32\n",
      "8 cat:31\n",
      "9 cat:34\n",
      "10 cat:99\n",
      "11 cat:0\n",
      "12 cat:21\n"
     ]
    }
   ],
   "source": [
    "for count, cat in enumerate(listofregion):\n",
    "    print str(count)+\" cat:\"+str(cat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Shape statistics for the landscape units"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Set the path to the file for i.segment.stats results for landscape units shape metrics\n",
    "landscape_units_shape_metrics=os.path.join(rli_output_dir,\"landscape_units_shape_metrics\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Metrics computed in 1 minutes and 13.4 seconds'"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save time for computing processin time\n",
    "begintime=time.time()\n",
    "\n",
    "# Run i.segment.stats\n",
    "gscript.run_command('i.segment.stats', overwrite=True, map=raster_landscapeunits,\n",
    "                    area_measures='area,perimeter,compact_circle,compact_square,fd,xcoords,ycoords',\n",
    "                    csvfile=landscape_units_shape_metrics,\n",
    "                    processes='1')\n",
    "\n",
    "# Compute and print processing time\n",
    "print_processing_time(begintime,\"Metrics computed in \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "resultfiles.append(landscape_units_shape_metrics)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_dominance',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_pielou',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_renyi',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_richness',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_shannon',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_simpson',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/landscape_units_shape_metrics']"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "resultfiles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "resultfiles=[u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_dominance',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_pielou',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_renyi',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_richness',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_shannon',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_simpson',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/landscape_units_shape_metrics']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Mean and standard deviation of NDVI, NDWI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Set the name of the NDVI layer\n",
    "ndvi=\"ndvi@PERMANENT\"\n",
    "# Set the name of the NDWI layer\n",
    "ndwi=\"ndwi@PERMANENT\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Layer to be used :\n",
      "\n",
      "ndvi@PERMANENT\n",
      "ndwi@PERMANENT\n"
     ]
    }
   ],
   "source": [
    "# Set up a list with name of raster layer to be used\n",
    "ancillarylayers=[]\n",
    "ancillarylayers.append(ndvi)\n",
    "ancillarylayers.append(ndwi)\n",
    "print \"Layer to be used :\\n\\n\"+'\\n'.join(ancillarylayers)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Metrics computed in 51 minutes and 33.2 seconds'"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save time for computing processin time\n",
    "begintime=time.time()\n",
    "###### Compute shape metrics as well as mean and stddev of ancillary layers for each landscape unit\n",
    "## Set number of cores to be used\n",
    "ncores=len(ancillarylayers) \n",
    "nbcpu=multiprocessing.cpu_count()\n",
    "if ncores>=nbcpu:\n",
    "    ncores=nbcpu-1\n",
    "    if ncores>len(ancillarylayers):\n",
    "        ncores=len(ancillarylayers)  #Adapt number of cores to number of metrics to compute\n",
    "\n",
    "listtempcsv=[]\n",
    "## Loop on tiles        \n",
    "for cat in listofregion[:]:\n",
    "    ## Save current time at loop' start. \n",
    "    begintime_current_id=time.time()\n",
    "    \n",
    "    ## Create a computional region for the current polygon\n",
    "    condition=\"morpho_typ=\"+cat\n",
    "    outputname=\"tmp_\"+cat\n",
    "    grass.run_command('v.extract', overwrite=True, quiet=True, \n",
    "                      input=landscape_polygons, type='area', where=condition, output=outputname)\n",
    "    grass.run_command('g.region', overwrite=True, vector=outputname, align=raster_landscapeunits)\n",
    "    grass.run_command('r.mask', overwrite=True, raster=tile_layer, maskcats=cat)\n",
    "    grass.run_command('g.remove', quiet=True, type=\"vector\", name=outputname, flags=\"f\")\n",
    "    \n",
    "    ## Define the csv output file name, according to the optimization function selected\n",
    "    outputcsv=os.path.join(rli_output_dir,outputname+\".csv\")\n",
    "    listtempcsv.append(outputcsv)\n",
    "    # Run i.segment.stats\n",
    "    gscript.run_command('i.segment.stats', overwrite=True, map=raster_landscapeunits,\n",
    "                        raster_statistics='stddev,median',\n",
    "                        rasters=','.join(ancillarylayers),\n",
    "                        csvfile=outputcsv,\n",
    "                        processes=ncores,\n",
    "                        flags='s')\n",
    "    \n",
    "    ## Remove current mask\n",
    "    grass.run_command('r.mask', flags='r')\n",
    "\n",
    "# Compute and print processing time\n",
    "print_processing_time(begintime,\"Metrics computed in \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Merge intermediate csv in one single csv**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Set the path to the file for i.segment.stats results for metrics_ndvi_ndwi_sar\n",
    "metrics_ndvi_ndwi=os.path.join(rli_output_dir,\"metrics_ndvi_ndwi\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create a dictionary with 'key' to be replaced by 'values' \n",
    "findreplacedict={}\n",
    "findreplacedict['nan']=\"0\"\n",
    "findreplacedict['null']=\"0\"\n",
    "findreplacedict['inf']=\"0\"\n",
    "\n",
    "# Define pattern of file to concatenate\n",
    "pat=\"tmp_*.csv\"\n",
    "sep=\"|\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Going to concatenate 13 .csv files together and replace unwanted values.Nothing changed. No unwanted values found '"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Concatenate csv files\n",
    "concat_findreplace(rli_output_dir,pat,sep,findreplacedict,metrics_ndvi_ndwi)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Remove temporary files**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for tmp_file in listtempcsv:\n",
    "    os.remove(tmp_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Append the file with metrics to the list of result files**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "resultfiles.append(metrics_ndvi_ndwi)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_dominance',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_pielou',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_renyi',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_richness',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_shannon',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_simpson',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/landscape_units_shape_metrics',\n",
       " u'/home/tais/.grass7/r.li/output/metrics_ndvi_ndwi']"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "resultfiles"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Mean and standard deviation of SAR textures"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Set the prefix of SAR textures layer\n",
    "SAR_prefix=\"SAR_\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Layer to be used :\n",
      "\n",
      "SAR_11_Cont@Compute_seg_stats\n",
      "SAR_11_Dis@Compute_seg_stats\n",
      "SAR_11_Entr@Compute_seg_stats\n",
      "SAR_11_Homo@Compute_seg_stats\n",
      "SAR_11_Mean@Compute_seg_stats\n",
      "SAR_11_SM@Compute_seg_stats\n",
      "SAR_11_Var@Compute_seg_stats\n",
      "SAR_7_Cont@Compute_seg_stats\n",
      "SAR_7_Dis@Compute_seg_stats\n",
      "SAR_7_Entr@Compute_seg_stats\n",
      "SAR_7_Homo@Compute_seg_stats\n",
      "SAR_7_Mean@Compute_seg_stats\n",
      "SAR_7_SM@Compute_seg_stats\n",
      "SAR_7_Var@Compute_seg_stats\n"
     ]
    }
   ],
   "source": [
    "# Set up a list with name of raster layer to be used\n",
    "ancillarylayers=[]\n",
    "[ancillarylayers.append(x) for x in gscript.list_strings(\"rast\", pattern=SAR_prefix, flag='r', mapset=\"Compute_seg_stats\")] #Append SAR textures\n",
    "print \"Layer to be used :\\n\\n\"+'\\n'.join(ancillarylayers)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Metrics computed in 47 minutes and 4.7 seconds'"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save time for computing processin time\n",
    "begintime=time.time()\n",
    "###### Compute shape metrics as well as mean and stddev of ancillary layers for each landscape unit\n",
    "## Set number of cores to be used\n",
    "ncores=len(ancillarylayers) \n",
    "nbcpu=multiprocessing.cpu_count()\n",
    "if ncores>=nbcpu:\n",
    "    ncores=nbcpu-1\n",
    "    if ncores>len(ancillarylayers):\n",
    "        ncores=len(ancillarylayers)  #Adapt number of cores to number of metrics to compute\n",
    "\n",
    "listtempcsv=[]\n",
    "## Loop on tiles        \n",
    "for cat in listofregion[:]:\n",
    "    ## Save current time at loop' start. \n",
    "    begintime_current_id=time.time()\n",
    "    \n",
    "    ## Create a computional region for the current polygon\n",
    "    condition=\"morpho_typ=\"+cat\n",
    "    outputname=\"tmp_\"+cat\n",
    "    grass.run_command('v.extract', overwrite=True, quiet=True, \n",
    "                      input=landscape_polygons, type='area', where=condition, output=outputname)\n",
    "    grass.run_command('g.region', overwrite=True, vector=outputname, align=raster_landscapeunits)\n",
    "    grass.run_command('r.mask', overwrite=True, raster=tile_layer, maskcats=cat)\n",
    "    grass.run_command('g.remove', quiet=True, type=\"vector\", name=outputname, flags=\"f\")\n",
    "    \n",
    "    ## Define the csv output file name, according to the optimization function selected\n",
    "    outputcsv=os.path.join(rli_output_dir,outputname+\".csv\")\n",
    "    listtempcsv.append(outputcsv)\n",
    "    # Run i.segment.stats\n",
    "    gscript.run_command('i.segment.stats', overwrite=True, map=raster_landscapeunits,\n",
    "                        raster_statistics='stddev,median',\n",
    "                        rasters=','.join(ancillarylayers),\n",
    "                        csvfile=outputcsv,\n",
    "                        processes=ncores,\n",
    "                        flags='s')\n",
    "    \n",
    "    ## Remove current mask\n",
    "    grass.run_command('r.mask', flags='r')\n",
    "\n",
    "# Compute and print processing time\n",
    "print_processing_time(begintime,\"Metrics computed in \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Merge intermediate csv in one single csv**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Set the path to the file for i.segment.stats results for metrics_ndvi_ndwi_sar\n",
    "metrics_sar=os.path.join(rli_output_dir,\"metrics_sar\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create a dictionary with 'key' to be replaced by 'values' \n",
    "findreplacedict={}\n",
    "findreplacedict['nan']=\"0\"\n",
    "findreplacedict['null']=\"0\"\n",
    "findreplacedict['inf']=\"0\"\n",
    "\n",
    "# Define pattern of file to concatenate\n",
    "pat=\"tmp_*.csv\"\n",
    "sep=\"|\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Going to concatenate 13 .csv files together and replace unwanted values.Nothing changed. No unwanted values found '"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Concatenate csv files\n",
    "concat_findreplace(rli_output_dir,pat,sep,findreplacedict,metrics_sar)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Remove temporary files**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for tmp_file in listtempcsv:\n",
    "    os.remove(tmp_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Append the file with metrics to the list of result files**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "resultfiles.append(metrics_sar)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_dominance',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_pielou',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_renyi',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_richness',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_shannon',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_simpson',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/landscape_units_shape_metrics',\n",
       " u'/home/tais/.grass7/r.li/output/metrics_ndvi_ndwi',\n",
       " u'/home/tais/.grass7/r.li/output/metrics_sar']"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "resultfiles"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Mean and standard deviation of built pixels' height"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Create raster with nDSM value of 'buildings' pixels**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Set the name of the nDSM layer\n",
    "ndsm=\"ndsm@PERMANENT\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Set pixel value of 'buildings' on the 'baselandcoverraster'\n",
    "buildpixel_A=111\n",
    "buildpixel_B=112\n",
    "# Set the name of the new layer containing height of buildings\n",
    "buildings_height='buildings_height'\n",
    "# Compute number of built pixels using i.segment.stats\n",
    "binary_builtup_raster=\"%s_builtmask\"%baselandcoverraster.split(\"@\")[0]\n",
    "# Set the path to the file for i.segment.stats results for metrics_ndvi_ndwi_sar\n",
    "metrics_buildings_height=os.path.join(rli_output_dir,\"metrics_buildings_height\")\n",
    "# Create temp fil which will contain intermediate results\n",
    "TMP_sumheights=grass.tempfile()+'_sumheights.csv'\n",
    "TMP_nbrbuildpixels=grass.tempfile()+'_nbrbuildpixels.csv'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Creation of layer in 8 minutes and 3.7 seconds'"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save time for computing processin time\n",
    "begintime=time.time()\n",
    "# Create a raster layer with height of pixels classified as 'buildings'\n",
    "gscript.run_command('g.region', raster=baselandcoverraster, quiet=True) #Set the region\n",
    "formula=\"%s=if(%s==%s || %s==%s , %s, 0)\"%(buildings_height,baselandcoverraster,buildpixel_A,\n",
    "                                           baselandcoverraster,buildpixel_B,ndsm)\n",
    "gscript.mapcalc(formula, overwrite=True)\n",
    "# Compute and print processing time\n",
    "print_processing_time(begintime,\"Creation of layer in \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Creation of layer in 7 minutes and 0.7 seconds'"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save time for computing processin time\n",
    "begintime=time.time()\n",
    "# Create a binary mask layer for pixels classified as 'buildings'\n",
    "gscript.run_command('g.region', raster=baselandcoverraster, quiet=True) #Set the region\n",
    "formula=\"%s=if(%s==%s || %s==%s , 1, 0)\"%(binary_builtup_raster,baselandcoverraster,buildpixel_A,\n",
    "                                           baselandcoverraster,buildpixel_B)\n",
    "gscript.mapcalc(formula, overwrite=True)\n",
    "gscript.run_command('r.null', map=binary_builtup_raster, null=0)\n",
    "# Compute and print processing time\n",
    "print_processing_time(begintime,\"Creation of layer in \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Compute the metric**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'i.segment.stats run in 7 minutes and 20.1 seconds'"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save time for computing processin time\n",
    "begintime=time.time()\n",
    "# Compute sum of build pixels's height using i.segment.stats \n",
    "gscript.run_command('i.segment.stats', overwrite=True, map=raster_landscapeunits,\n",
    "                    raster_statistics='sum', flags='s', rasters=buildings_height,\n",
    "                    csvfile=TMP_sumheights,processes=ncores)\n",
    "\n",
    "# Compute total number of build pixels using i.segment.stats \n",
    "gscript.run_command('i.segment.stats', overwrite=True, map=raster_landscapeunits,\n",
    "                    raster_statistics='sum', flags='s', rasters=binary_builtup_raster,\n",
    "                    csvfile=TMP_nbrbuildpixels,processes=ncores)\n",
    "# Compute and print processing time\n",
    "print_processing_time(begintime,\"i.segment.stats run in \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Mean build pixels's height computed in 0.2 seconds\""
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Save time for computing processin time\n",
    "begintime=time.time()\n",
    "# Improt library to be able to iterate on two files in the same loop\n",
    "from itertools import izip\n",
    "# Declare empty dictionnary\n",
    "sumheight_dic={}\n",
    "nbpixel_dic={}\n",
    "tmp_dic={}\n",
    "# Loop through files and fill dictionnaries\n",
    "for i, (line_from_file_1, line_from_file_2) in enumerate(izip(open(TMP_sumheights), open(TMP_nbrbuildpixels))):\n",
    "    if i==0:\n",
    "        continue\n",
    "    f1_items=line_from_file_1.split(\"\\n\")[0].split(\"|\")\n",
    "    f2_items=line_from_file_2.split(\"\\n\")[0].split(\"|\")\n",
    "    sumheight_dic[f1_items[0]]=f1_items[1]\n",
    "    nbpixel_dic[f2_items[0]]=f2_items[1]\n",
    "\n",
    "# Loop through dictionnaries keys\n",
    "keys_list=[]\n",
    "[keys_list.append(k) for k in sumheight_dic.keys()]\n",
    "[keys_list.append(k) for k in nbpixel_dic.keys()]\n",
    "keys_list=set(keys_list) # Get unique values\n",
    "for key in keys_list:\n",
    "    try:\n",
    "        mean_height=\"{0:.2f}\".format(float(sumheight_dic[key])/float(nbpixel_dic[key]))\n",
    "    except ZeroDivisionError:\n",
    "        mean_height=0\n",
    "    tmp_dic[key]=mean_height\n",
    "# Get the name of the first colum\n",
    "with open(TMP_sumheights) as f:\n",
    "    column_a=f.next().split(\"\\n\")[0].split(\"|\")[0]\n",
    "# Built the content of the file\n",
    "content=[]\n",
    "content.append((column_a,'mean_build_height','count_buildpixels'))\n",
    "for key in tmp_dic.keys():\n",
    "    content.append((key,tmp_dic[key],nbpixel_dic[key]))\n",
    "# Create a new file\n",
    "fout=open(metrics_buildings_height,\"w\")\n",
    "writer=csv.writer(fout, delimiter='|')\n",
    "writer.writerows(content)\n",
    "fout.close()\n",
    "# Compute and print processing time\n",
    "print_processing_time(begintime,\"Mean build pixels's height computed in \")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Remove temporary layers\n",
    "gscript.run_command('g.remove', flags='ef', type='raster', name=binary_builtup_raster)\n",
    "# Remove temporary files\n",
    "os.remove(TMP_sumheights)\n",
    "os.remove(TMP_nbrbuildpixels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "resultfiles.append(metrics_buildings_height)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_dominance',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_pielou',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_renyi',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_richness',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_shannon',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_simpson',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/landscape_units_shape_metrics',\n",
       " u'/home/tais/.grass7/r.li/output/metrics_ndvi_ndwi',\n",
       " u'/home/tais/.grass7/r.li/output/metrics_sar',\n",
       " u'/home/tais/.grass7/r.li/output/metrics_buildings_height']"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "resultfiles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "resultfiles=[u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_dominance',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_pielou',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_renyi',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_richness',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_shannon',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_simpson',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_patchnum',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_patchdensity',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_mps',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padcv',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padrange',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padsd',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_shape',\n",
    " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_edgedensity',\n",
    " u'/home/tais/.grass7/r.li/output/landscape_units_shape_metrics',\n",
    " u'/home/tais/.grass7/r.li/output/metrics_ndvi_ndwi',\n",
    " u'/home/tais/.grass7/r.li/output/metrics_sar',\n",
    " u'/home/tais/.grass7/r.li/output/metrics_buildings_height']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Proportion of each landcover class in the landscape units"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Set a list with name of land cover class binary raster to be processed \n",
    "listofbinary=list(raster_list[1:])\n",
    "# Compute proportion of each land cover class in landscape units\n",
    "proportion_results=get_classproportions(rli_output_dir,raster_landscapeunits,listofbinary,returnlistresult=True,ncores=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[None, None, None, None, None, None, None, None, None]"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[resultfiles.append(x) for x in proportion_results]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_dominance',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_pielou',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_renyi',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_richness',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_shannon',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_simpson',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_13_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_14_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_20_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_30_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_31_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_41_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_51_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_111_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_patchnum',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_patchdensity',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_mps',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padcv',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padrange',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_padsd',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_shape',\n",
       " u'/home/tais/.grass7/r.li/output/postclassif_map_6_shadowback_cl_112_edgedensity',\n",
       " u'/home/tais/.grass7/r.li/output/landscape_units_shape_metrics',\n",
       " u'/home/tais/.grass7/r.li/output/metrics_ndvi_ndwi',\n",
       " u'/home/tais/.grass7/r.li/output/metrics_sar',\n",
       " u'/home/tais/.grass7/r.li/output/metrics_buildings_height',\n",
       " u'/home/tais/.grass7/r.li/output/prop_13',\n",
       " u'/home/tais/.grass7/r.li/output/prop_14',\n",
       " u'/home/tais/.grass7/r.li/output/prop_20',\n",
       " u'/home/tais/.grass7/r.li/output/prop_30',\n",
       " u'/home/tais/.grass7/r.li/output/prop_31',\n",
       " u'/home/tais/.grass7/r.li/output/prop_41',\n",
       " u'/home/tais/.grass7/r.li/output/prop_51',\n",
       " u'/home/tais/.grass7/r.li/output/prop_111',\n",
       " u'/home/tais/.grass7/r.li/output/prop_112']"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "resultfiles"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# File management (join, cleaning, etc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Combine all .csv files together"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Define the path for the .csv with final results\n",
    "outfile=os.path.join(rli_output_dir,\"land_use_metrics.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "91 individual .csv files were joint together.\n"
     ]
    }
   ],
   "source": [
    "# Join (inner join) all result files together in a new .csv file\n",
    "join_multiplecsv(resultfiles,outfile,separator=\"|\",join='inner',fillempty='NULL',overwrite=True)\n",
    "# Create .csvt file\n",
    "create_csvt(outfile,separator=\"|\",first_col_type=\"Integer\",rest_type=\"Real\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Make a copy of the .csv file with results, where 'null' values are empty cells "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**For .csvt**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import shutil\n",
    "# Define path to the .csv files\n",
    "afile=outfile+'t'\n",
    "pathtofile,extension=os.path.splitext(afile)\n",
    "bfile=pathtofile+\"_blanknull\"+extension\n",
    "# Make copy of the file\n",
    "shutil.copy2(afile,bfile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**For .csv**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Define path to the .csv files\n",
    "afile=outfile\n",
    "pathtofile,extension=os.path.splitext(afile)\n",
    "outfile_withoutnull=pathtofile+\"_blanknull\"+extension"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import csv\n",
    "# Create a copy by removing the 'NULL' values\n",
    "reader=csv.reader(open(afile,'r'),delimiter=\"|\")\n",
    "writer=csv.writer(open(outfile_withoutnull,'w'),delimiter=\"|\")\n",
    "for row in reader:\n",
    "    newline=[]\n",
    "    [newline.append(x) if x != \"NULL\" else newline.append(\"\") for x in row]\n",
    "    writer.writerow(newline)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Check if all the lines have the same number of items"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#### Function that check if rows of a .csv file have the same number of items than number of column in the first line (header)\n",
    "### If rows's lenght are identical, the function return 0. If not, it return a list containing the index number of row(s) that failed\n",
    "### The index is starting with value 1 for the first line of data (the header row is indexed 0)\n",
    "# 'csvfile' wait for the complete path (string) to the csvfile to be checked\n",
    "# 'separator' wait for the character used as separator in the .csv file (string)\n",
    "# 'allowemptycell' wait for a boolean value (True, False) depending if a empty cell should not be considered as a problem in the file\n",
    "\n",
    "def check_lenght_row(csvfile,separator,allowemptycell=True):\n",
    "    listofunequalrow=[]\n",
    "    afile=open(csvfile,'r')\n",
    "    header=afile.next()  # Save header row and go to the next one\n",
    "    nb_item_control=len(header.split(separator))\n",
    "    for x,row in enumerate(afile,1):    # Start counting at \"1\" because header is already skipped\n",
    "        row_items=row.split(separator)\n",
    "        if len(row_items) != nb_item_control: # Check it number of items is identical than the number of items in the header\n",
    "            listofunequalrow.append(x)\n",
    "        if not allowemptycell:\n",
    "            if \"\" in row_items:\n",
    "                listofunequalrow.append(x)\n",
    "    listofunequalrow=list(set(listofunequalrow))   # Recreate a list with uniques values from the original list\n",
    "    if len(listofunequalrow)>0:\n",
    "        return listofunequalrow  # Return a list of indexes for line whose length is not equal to the header\n",
    "    else:\n",
    "        return 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The csv looks good.\n"
     ]
    }
   ],
   "source": [
    "# Check number of row in the .csv file\n",
    "fout=check_lenght_row(outfile_withoutnull,\";\",allowemptycell=True)\n",
    "if fout>0:\n",
    "    print \"Rows in csv do not have the same lenght. Please check the following row(s) index(es).\"\n",
    "    print \"/n\".join(fout)\n",
    "else:\n",
    "    print \"The csv looks good.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Replace current delimiter by ';'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import csv\n",
    "reader=csv.reader(open(outfile,'r'),delimiter=\"|\")\n",
    "newfile=[]\n",
    "for row in reader:\n",
    "    newline=[]\n",
    "    [newline.append(x) for x in row]\n",
    "    newfile.append(newline)\n",
    "writer=csv.writer(open(outfile,'w'),delimiter=\";\")\n",
    "writer.writerows(newfile)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import csv\n",
    "reader=csv.reader(open(outfile+'t','r'),delimiter=\"|\")\n",
    "newfile=[]\n",
    "for row in reader:\n",
    "    newline=[]\n",
    "    [newline.append(x) for x in row]\n",
    "    newfile.append(newline)\n",
    "writer=csv.writer(open(outfile+'t','w'),delimiter=\";\")\n",
    "writer.writerows(newfile)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import csv\n",
    "reader=csv.reader(open(outfile_withoutnull,'r'),delimiter=\"|\")\n",
    "newfile=[]\n",
    "for row in reader:\n",
    "    newline=[]\n",
    "    [newline.append(x) for x in row]\n",
    "    newfile.append(newline)\n",
    "writer=csv.writer(open(outfile_withoutnull,'w'),delimiter=\";\")\n",
    "writer.writerows(newfile)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import csv\n",
    "reader=csv.reader(open(outfile_withoutnull+'t','r'),delimiter=\"|\")\n",
    "newfile=[]\n",
    "for row in reader:\n",
    "    newline=[]\n",
    "    [newline.append(x) for x in row]\n",
    "    newfile.append(newline)\n",
    "writer=csv.writer(open(outfile_withoutnull+'t','w'),delimiter=\";\")\n",
    "writer.writerows(newfile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visual check of the table (display the .csv using pandas)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cat</th>\n",
       "      <th>map_6_shadowback_dominance</th>\n",
       "      <th>map_6_shadowback_pielou</th>\n",
       "      <th>map_6_shadowback_renyi</th>\n",
       "      <th>map_6_shadowback_richness</th>\n",
       "      <th>map_6_shadowback_shannon</th>\n",
       "      <th>map_6_shadowback_simpson</th>\n",
       "      <th>map_6_shadowback_cl_13_patchnum</th>\n",
       "      <th>map_6_shadowback_cl_13_patchdensity</th>\n",
       "      <th>map_6_shadowback_cl_13_mps</th>\n",
       "      <th>...</th>\n",
       "      <th>count_buildpixels</th>\n",
       "      <th>prop_13</th>\n",
       "      <th>prop_14</th>\n",
       "      <th>prop_20</th>\n",
       "      <th>prop_30</th>\n",
       "      <th>prop_31</th>\n",
       "      <th>prop_41</th>\n",
       "      <th>prop_51</th>\n",
       "      <th>prop_111</th>\n",
       "      <th>prop_112</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0.774693</td>\n",
       "      <td>0.441177</td>\n",
       "      <td>0.409695</td>\n",
       "      <td>4</td>\n",
       "      <td>0.611602</td>\n",
       "      <td>0.336147</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>17.01099</td>\n",
       "      <td>79.63006</td>\n",
       "      <td>2.81252</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.54643</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1.170281</td>\n",
       "      <td>0.155821</td>\n",
       "      <td>0.088073</td>\n",
       "      <td>4</td>\n",
       "      <td>0.216013</td>\n",
       "      <td>0.084306</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.87729</td>\n",
       "      <td>95.64191</td>\n",
       "      <td>1.05089</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.42991</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1.114241</td>\n",
       "      <td>0.307683</td>\n",
       "      <td>0.279488</td>\n",
       "      <td>5</td>\n",
       "      <td>0.495197</td>\n",
       "      <td>0.243830</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>21</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.15909</td>\n",
       "      <td>86.35501</td>\n",
       "      <td>9.44135</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>3.91605</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.12850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>0.529658</td>\n",
       "      <td>0.670905</td>\n",
       "      <td>0.862815</td>\n",
       "      <td>5</td>\n",
       "      <td>1.079780</td>\n",
       "      <td>0.578028</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>198</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>26.06844</td>\n",
       "      <td>58.57774</td>\n",
       "      <td>7.41317</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>7.31149</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.62915</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0.550163</td>\n",
       "      <td>0.692948</td>\n",
       "      <td>1.020990</td>\n",
       "      <td>6</td>\n",
       "      <td>1.241597</td>\n",
       "      <td>0.639762</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1458</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>25.00636</td>\n",
       "      <td>52.02209</td>\n",
       "      <td>1.76592</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>15.90785</td>\n",
       "      <td>2.12202</td>\n",
       "      <td>3.17576</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>0.151481</td>\n",
       "      <td>0.915457</td>\n",
       "      <td>1.537313</td>\n",
       "      <td>6</td>\n",
       "      <td>1.640279</td>\n",
       "      <td>0.785042</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>6193</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>30.25345</td>\n",
       "      <td>21.38390</td>\n",
       "      <td>23.63766</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>8.91804</td>\n",
       "      <td>5.27323</td>\n",
       "      <td>10.53370</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>0.495817</td>\n",
       "      <td>0.723279</td>\n",
       "      <td>1.005671</td>\n",
       "      <td>6</td>\n",
       "      <td>1.295942</td>\n",
       "      <td>0.634201</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1400</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>20.64206</td>\n",
       "      <td>55.25953</td>\n",
       "      <td>6.48065</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>9.21692</td>\n",
       "      <td>1.36814</td>\n",
       "      <td>7.03270</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>0.402974</td>\n",
       "      <td>0.775096</td>\n",
       "      <td>1.065929</td>\n",
       "      <td>6</td>\n",
       "      <td>1.388786</td>\n",
       "      <td>0.655592</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>2017</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>11.42346</td>\n",
       "      <td>54.51495</td>\n",
       "      <td>4.73492</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>14.28678</td>\n",
       "      <td>6.50213</td>\n",
       "      <td>8.53777</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>0.565619</td>\n",
       "      <td>0.684322</td>\n",
       "      <td>1.000676</td>\n",
       "      <td>6</td>\n",
       "      <td>1.226141</td>\n",
       "      <td>0.632369</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1637</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>48.41663</td>\n",
       "      <td>35.48171</td>\n",
       "      <td>3.65050</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>6.35954</td>\n",
       "      <td>3.70260</td>\n",
       "      <td>2.38901</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>0.615504</td>\n",
       "      <td>0.439744</td>\n",
       "      <td>0.358073</td>\n",
       "      <td>3</td>\n",
       "      <td>0.483109</td>\n",
       "      <td>0.300978</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>18.34918</td>\n",
       "      <td>81.56910</td>\n",
       "      <td>0.08172</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>11</td>\n",
       "      <td>0.617945</td>\n",
       "      <td>0.554247</td>\n",
       "      <td>0.672758</td>\n",
       "      <td>4</td>\n",
       "      <td>0.768349</td>\n",
       "      <td>0.489701</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>35.26711</td>\n",
       "      <td>62.07568</td>\n",
       "      <td>2.40215</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.25505</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>12</td>\n",
       "      <td>0.783739</td>\n",
       "      <td>0.562587</td>\n",
       "      <td>0.664487</td>\n",
       "      <td>6</td>\n",
       "      <td>1.008020</td>\n",
       "      <td>0.485463</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>347</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>11.86831</td>\n",
       "      <td>8.73912</td>\n",
       "      <td>69.79167</td>\n",
       "      <td>0.76138</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>7.38788</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>1.45164</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>13</td>\n",
       "      <td>0.992274</td>\n",
       "      <td>0.446201</td>\n",
       "      <td>0.452768</td>\n",
       "      <td>6</td>\n",
       "      <td>0.799486</td>\n",
       "      <td>0.364134</td>\n",
       "      <td>1</td>\n",
       "      <td>105263.157895</td>\n",
       "      <td>0.000950</td>\n",
       "      <td>...</td>\n",
       "      <td>1156</td>\n",
       "      <td>0.08838</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>6.01684</td>\n",
       "      <td>78.99107</td>\n",
       "      <td>5.39120</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>6.82389</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.68862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>14</td>\n",
       "      <td>0.397194</td>\n",
       "      <td>0.638459</td>\n",
       "      <td>0.695489</td>\n",
       "      <td>3</td>\n",
       "      <td>0.701419</td>\n",
       "      <td>0.501170</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>50.00690</td>\n",
       "      <td>49.87584</td>\n",
       "      <td>0.11726</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>15</td>\n",
       "      <td>1.113281</td>\n",
       "      <td>0.308279</td>\n",
       "      <td>0.249488</td>\n",
       "      <td>5</td>\n",
       "      <td>0.496157</td>\n",
       "      <td>0.220800</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>743</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.63222</td>\n",
       "      <td>87.92716</td>\n",
       "      <td>6.93959</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>3.28095</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>1.22009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>16</td>\n",
       "      <td>0.384469</td>\n",
       "      <td>0.650042</td>\n",
       "      <td>0.700119</td>\n",
       "      <td>3</td>\n",
       "      <td>0.714144</td>\n",
       "      <td>0.503474</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>50.29112</td>\n",
       "      <td>49.35524</td>\n",
       "      <td>0.35363</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>17</td>\n",
       "      <td>0.420727</td>\n",
       "      <td>0.696510</td>\n",
       "      <td>0.823578</td>\n",
       "      <td>4</td>\n",
       "      <td>0.965568</td>\n",
       "      <td>0.561141</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1311</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>35.20563</td>\n",
       "      <td>55.66754</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.48683</td>\n",
       "      <td>6.63999</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>18</td>\n",
       "      <td>1.166417</td>\n",
       "      <td>0.158608</td>\n",
       "      <td>0.089096</td>\n",
       "      <td>4</td>\n",
       "      <td>0.219877</td>\n",
       "      <td>0.085242</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.49371</td>\n",
       "      <td>95.59516</td>\n",
       "      <td>2.74628</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>1.16485</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>19</td>\n",
       "      <td>0.849419</td>\n",
       "      <td>0.387273</td>\n",
       "      <td>0.279701</td>\n",
       "      <td>4</td>\n",
       "      <td>0.536875</td>\n",
       "      <td>0.243991</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.85655</td>\n",
       "      <td>86.57941</td>\n",
       "      <td>5.01714</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>5.54690</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>20</td>\n",
       "      <td>0.788068</td>\n",
       "      <td>0.510346</td>\n",
       "      <td>0.527234</td>\n",
       "      <td>5</td>\n",
       "      <td>0.821370</td>\n",
       "      <td>0.409765</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1162</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>15.97967</td>\n",
       "      <td>74.94286</td>\n",
       "      <td>1.73429</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>4.21876</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>3.12441</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>21</td>\n",
       "      <td>0.904822</td>\n",
       "      <td>0.495009</td>\n",
       "      <td>0.560870</td>\n",
       "      <td>6</td>\n",
       "      <td>0.886937</td>\n",
       "      <td>0.429287</td>\n",
       "      <td>2</td>\n",
       "      <td>126984.126984</td>\n",
       "      <td>0.000788</td>\n",
       "      <td>...</td>\n",
       "      <td>6279</td>\n",
       "      <td>0.03782</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>13.75793</td>\n",
       "      <td>73.91414</td>\n",
       "      <td>2.84020</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>5.68039</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>3.76952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>22</td>\n",
       "      <td>1.173795</td>\n",
       "      <td>0.270680</td>\n",
       "      <td>0.205305</td>\n",
       "      <td>5</td>\n",
       "      <td>0.435643</td>\n",
       "      <td>0.185601</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>397</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.16425</td>\n",
       "      <td>90.05287</td>\n",
       "      <td>4.26012</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>3.48509</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.03767</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>23</td>\n",
       "      <td>0.768708</td>\n",
       "      <td>0.522375</td>\n",
       "      <td>0.741694</td>\n",
       "      <td>5</td>\n",
       "      <td>0.840730</td>\n",
       "      <td>0.523694</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>133</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>40.44650</td>\n",
       "      <td>55.85087</td>\n",
       "      <td>2.66423</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.82495</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.21346</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>24</td>\n",
       "      <td>0.770413</td>\n",
       "      <td>0.570024</td>\n",
       "      <td>0.723157</td>\n",
       "      <td>6</td>\n",
       "      <td>1.021346</td>\n",
       "      <td>0.514782</td>\n",
       "      <td>3</td>\n",
       "      <td>110091.743119</td>\n",
       "      <td>0.000908</td>\n",
       "      <td>...</td>\n",
       "      <td>11177</td>\n",
       "      <td>0.05721</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>18.99993</td>\n",
       "      <td>66.42628</td>\n",
       "      <td>2.46737</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>6.18285</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>5.86636</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>25</td>\n",
       "      <td>1.047587</td>\n",
       "      <td>0.349098</td>\n",
       "      <td>0.303876</td>\n",
       "      <td>5</td>\n",
       "      <td>0.561851</td>\n",
       "      <td>0.262047</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>206</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.26164</td>\n",
       "      <td>85.39093</td>\n",
       "      <td>7.07028</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>6.02373</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>1.25342</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>26</td>\n",
       "      <td>1.254656</td>\n",
       "      <td>0.220438</td>\n",
       "      <td>0.169420</td>\n",
       "      <td>5</td>\n",
       "      <td>0.354782</td>\n",
       "      <td>0.155846</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>36</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>5.43920</td>\n",
       "      <td>91.68610</td>\n",
       "      <td>2.30147</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.56470</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00853</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>27</td>\n",
       "      <td>0.850166</td>\n",
       "      <td>0.386735</td>\n",
       "      <td>0.357589</td>\n",
       "      <td>4</td>\n",
       "      <td>0.536128</td>\n",
       "      <td>0.300640</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>16.17332</td>\n",
       "      <td>82.03377</td>\n",
       "      <td>1.56004</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.23286</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>28</td>\n",
       "      <td>0.866753</td>\n",
       "      <td>0.374770</td>\n",
       "      <td>0.301444</td>\n",
       "      <td>4</td>\n",
       "      <td>0.519542</td>\n",
       "      <td>0.260251</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>69</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>85.35832</td>\n",
       "      <td>8.97414</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>5.55880</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.10875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>29</td>\n",
       "      <td>0.633206</td>\n",
       "      <td>0.543239</td>\n",
       "      <td>0.713622</td>\n",
       "      <td>4</td>\n",
       "      <td>0.753089</td>\n",
       "      <td>0.510133</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>51.43901</td>\n",
       "      <td>47.45612</td>\n",
       "      <td>0.51680</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.58808</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>30</td>\n",
       "      <td>0.741641</td>\n",
       "      <td>0.465019</td>\n",
       "      <td>0.494383</td>\n",
       "      <td>4</td>\n",
       "      <td>0.644653</td>\n",
       "      <td>0.390053</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>23.94607</td>\n",
       "      <td>74.32720</td>\n",
       "      <td>0.99559</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.73114</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31904</th>\n",
       "      <td>31905</td>\n",
       "      <td>0.586662</td>\n",
       "      <td>0.672578</td>\n",
       "      <td>0.983242</td>\n",
       "      <td>6</td>\n",
       "      <td>1.205098</td>\n",
       "      <td>0.625904</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>909</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>20.17679</td>\n",
       "      <td>54.73998</td>\n",
       "      <td>17.38262</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>5.61359</td>\n",
       "      <td>0.16301</td>\n",
       "      <td>1.92400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31905</th>\n",
       "      <td>31906</td>\n",
       "      <td>0.884348</td>\n",
       "      <td>0.506436</td>\n",
       "      <td>0.625676</td>\n",
       "      <td>6</td>\n",
       "      <td>0.907412</td>\n",
       "      <td>0.465100</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1811</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>20.20195</td>\n",
       "      <td>70.05947</td>\n",
       "      <td>3.89465</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>3.48736</td>\n",
       "      <td>0.07287</td>\n",
       "      <td>2.28370</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31906</th>\n",
       "      <td>31907</td>\n",
       "      <td>0.544956</td>\n",
       "      <td>0.606897</td>\n",
       "      <td>0.737414</td>\n",
       "      <td>4</td>\n",
       "      <td>0.841338</td>\n",
       "      <td>0.521651</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>38.07967</td>\n",
       "      <td>57.62506</td>\n",
       "      <td>3.48183</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.81344</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31907</th>\n",
       "      <td>31908</td>\n",
       "      <td>1.095905</td>\n",
       "      <td>0.209472</td>\n",
       "      <td>0.147315</td>\n",
       "      <td>4</td>\n",
       "      <td>0.290389</td>\n",
       "      <td>0.136978</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.03342</td>\n",
       "      <td>92.67097</td>\n",
       "      <td>6.44910</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.84651</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31908</th>\n",
       "      <td>31909</td>\n",
       "      <td>0.678298</td>\n",
       "      <td>0.510712</td>\n",
       "      <td>0.440721</td>\n",
       "      <td>4</td>\n",
       "      <td>0.707997</td>\n",
       "      <td>0.356428</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>12.32871</td>\n",
       "      <td>78.98450</td>\n",
       "      <td>6.27243</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.41436</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31909</th>\n",
       "      <td>31910</td>\n",
       "      <td>0.544666</td>\n",
       "      <td>0.661580</td>\n",
       "      <td>0.786918</td>\n",
       "      <td>5</td>\n",
       "      <td>1.064772</td>\n",
       "      <td>0.544754</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>547</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>15.51585</td>\n",
       "      <td>63.96050</td>\n",
       "      <td>13.83402</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>5.21770</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>1.47193</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31910</th>\n",
       "      <td>31911</td>\n",
       "      <td>1.095834</td>\n",
       "      <td>0.319120</td>\n",
       "      <td>0.268876</td>\n",
       "      <td>5</td>\n",
       "      <td>0.513604</td>\n",
       "      <td>0.235762</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>417</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.08198</td>\n",
       "      <td>86.98554</td>\n",
       "      <td>7.98135</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.79980</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.15133</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31911</th>\n",
       "      <td>31912</td>\n",
       "      <td>0.921634</td>\n",
       "      <td>0.427357</td>\n",
       "      <td>0.382812</td>\n",
       "      <td>5</td>\n",
       "      <td>0.687804</td>\n",
       "      <td>0.318059</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>892</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>9.78549</td>\n",
       "      <td>81.33859</td>\n",
       "      <td>3.73936</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.26888</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.25372</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31912</th>\n",
       "      <td>31913</td>\n",
       "      <td>0.943234</td>\n",
       "      <td>0.413936</td>\n",
       "      <td>0.382958</td>\n",
       "      <td>5</td>\n",
       "      <td>0.666204</td>\n",
       "      <td>0.318158</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>211</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>7.06218</td>\n",
       "      <td>81.80815</td>\n",
       "      <td>8.31494</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.60494</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.20979</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31913</th>\n",
       "      <td>31914</td>\n",
       "      <td>0.789784</td>\n",
       "      <td>0.430291</td>\n",
       "      <td>0.342754</td>\n",
       "      <td>4</td>\n",
       "      <td>0.596510</td>\n",
       "      <td>0.290187</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>4.48893</td>\n",
       "      <td>83.49616</td>\n",
       "      <td>10.14227</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>1.87264</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31914</th>\n",
       "      <td>31915</td>\n",
       "      <td>0.299604</td>\n",
       "      <td>0.783881</td>\n",
       "      <td>0.969324</td>\n",
       "      <td>4</td>\n",
       "      <td>1.086690</td>\n",
       "      <td>0.620661</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>20.83009</td>\n",
       "      <td>52.19338</td>\n",
       "      <td>25.13913</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>1.83740</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31915</th>\n",
       "      <td>31916</td>\n",
       "      <td>0.827453</td>\n",
       "      <td>0.485875</td>\n",
       "      <td>0.436869</td>\n",
       "      <td>5</td>\n",
       "      <td>0.781985</td>\n",
       "      <td>0.353944</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>3014</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>4.68044</td>\n",
       "      <td>79.71051</td>\n",
       "      <td>6.68654</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>4.89428</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>4.02823</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31916</th>\n",
       "      <td>31917</td>\n",
       "      <td>0.926510</td>\n",
       "      <td>0.424327</td>\n",
       "      <td>0.384052</td>\n",
       "      <td>5</td>\n",
       "      <td>0.682928</td>\n",
       "      <td>0.318904</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>820</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>5.97878</td>\n",
       "      <td>78.16263</td>\n",
       "      <td>8.16645</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.32110</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.91947</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31917</th>\n",
       "      <td>31918</td>\n",
       "      <td>0.794901</td>\n",
       "      <td>0.556357</td>\n",
       "      <td>0.741552</td>\n",
       "      <td>6</td>\n",
       "      <td>0.996858</td>\n",
       "      <td>0.523626</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>137</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>18.59962</td>\n",
       "      <td>65.27960</td>\n",
       "      <td>11.97098</td>\n",
       "      <td>0.44512</td>\n",
       "      <td>3.58463</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.12004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31918</th>\n",
       "      <td>31919</td>\n",
       "      <td>1.041905</td>\n",
       "      <td>0.418502</td>\n",
       "      <td>0.435553</td>\n",
       "      <td>6</td>\n",
       "      <td>0.749854</td>\n",
       "      <td>0.353093</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>26</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>8.97562</td>\n",
       "      <td>78.44909</td>\n",
       "      <td>7.28642</td>\n",
       "      <td>1.60008</td>\n",
       "      <td>2.31407</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.02663</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31919</th>\n",
       "      <td>31920</td>\n",
       "      <td>0.786770</td>\n",
       "      <td>0.560896</td>\n",
       "      <td>0.690804</td>\n",
       "      <td>6</td>\n",
       "      <td>1.004990</td>\n",
       "      <td>0.498827</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1792</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>8.45431</td>\n",
       "      <td>68.22031</td>\n",
       "      <td>16.10474</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>4.36804</td>\n",
       "      <td>0.06049</td>\n",
       "      <td>2.79210</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31920</th>\n",
       "      <td>31921</td>\n",
       "      <td>0.569683</td>\n",
       "      <td>0.646036</td>\n",
       "      <td>0.718487</td>\n",
       "      <td>5</td>\n",
       "      <td>1.039755</td>\n",
       "      <td>0.512511</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>516</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>16.30763</td>\n",
       "      <td>67.20783</td>\n",
       "      <td>4.70392</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>6.41066</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>5.36997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31921</th>\n",
       "      <td>31922</td>\n",
       "      <td>0.706554</td>\n",
       "      <td>0.490329</td>\n",
       "      <td>0.397666</td>\n",
       "      <td>4</td>\n",
       "      <td>0.679740</td>\n",
       "      <td>0.328114</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>7.06095</td>\n",
       "      <td>81.16648</td>\n",
       "      <td>8.30603</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>3.46653</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31922</th>\n",
       "      <td>31923</td>\n",
       "      <td>0.671756</td>\n",
       "      <td>0.582614</td>\n",
       "      <td>0.628961</td>\n",
       "      <td>5</td>\n",
       "      <td>0.937682</td>\n",
       "      <td>0.466854</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>636</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>14.41738</td>\n",
       "      <td>70.87162</td>\n",
       "      <td>9.09412</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>3.88777</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>1.72911</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31923</th>\n",
       "      <td>31924</td>\n",
       "      <td>0.847219</td>\n",
       "      <td>0.473593</td>\n",
       "      <td>0.474367</td>\n",
       "      <td>5</td>\n",
       "      <td>0.762219</td>\n",
       "      <td>0.377721</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>49</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>6.68112</td>\n",
       "      <td>77.14309</td>\n",
       "      <td>11.13499</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>4.33027</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.03043</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31924</th>\n",
       "      <td>31925</td>\n",
       "      <td>0.450412</td>\n",
       "      <td>0.720143</td>\n",
       "      <td>0.984763</td>\n",
       "      <td>5</td>\n",
       "      <td>1.159026</td>\n",
       "      <td>0.626472</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1744</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>35.68032</td>\n",
       "      <td>47.79205</td>\n",
       "      <td>9.21554</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>3.91952</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.42485</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31925</th>\n",
       "      <td>31926</td>\n",
       "      <td>0.400964</td>\n",
       "      <td>0.776218</td>\n",
       "      <td>1.208737</td>\n",
       "      <td>6</td>\n",
       "      <td>1.390795</td>\n",
       "      <td>0.701426</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>2749</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>31.18065</td>\n",
       "      <td>39.45612</td>\n",
       "      <td>6.50613</td>\n",
       "      <td>14.13866</td>\n",
       "      <td>2.98855</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.31946</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31926</th>\n",
       "      <td>31927</td>\n",
       "      <td>0.862406</td>\n",
       "      <td>0.377906</td>\n",
       "      <td>0.310638</td>\n",
       "      <td>4</td>\n",
       "      <td>0.523889</td>\n",
       "      <td>0.267021</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.35940</td>\n",
       "      <td>82.48316</td>\n",
       "      <td>11.14427</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>3.30908</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31927</th>\n",
       "      <td>31928</td>\n",
       "      <td>0.916352</td>\n",
       "      <td>0.338992</td>\n",
       "      <td>0.240125</td>\n",
       "      <td>4</td>\n",
       "      <td>0.469943</td>\n",
       "      <td>0.213471</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.58326</td>\n",
       "      <td>88.34590</td>\n",
       "      <td>7.03366</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>2.03718</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31928</th>\n",
       "      <td>31929</td>\n",
       "      <td>0.768018</td>\n",
       "      <td>0.522804</td>\n",
       "      <td>0.647813</td>\n",
       "      <td>5</td>\n",
       "      <td>0.841420</td>\n",
       "      <td>0.476811</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>72</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.75567</td>\n",
       "      <td>64.33146</td>\n",
       "      <td>22.46334</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>7.11859</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.05496</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31929</th>\n",
       "      <td>31930</td>\n",
       "      <td>0.875352</td>\n",
       "      <td>0.511457</td>\n",
       "      <td>0.552846</td>\n",
       "      <td>6</td>\n",
       "      <td>0.916407</td>\n",
       "      <td>0.424690</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>5041</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>6.93743</td>\n",
       "      <td>74.76517</td>\n",
       "      <td>5.82594</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>7.75514</td>\n",
       "      <td>0.12911</td>\n",
       "      <td>4.58722</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31930</th>\n",
       "      <td>31931</td>\n",
       "      <td>0.887180</td>\n",
       "      <td>0.448764</td>\n",
       "      <td>0.444995</td>\n",
       "      <td>5</td>\n",
       "      <td>0.722258</td>\n",
       "      <td>0.359172</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>26</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>4.40903</td>\n",
       "      <td>75.92320</td>\n",
       "      <td>11.80054</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>4.12951</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.01536</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31931</th>\n",
       "      <td>31932</td>\n",
       "      <td>1.040861</td>\n",
       "      <td>0.353276</td>\n",
       "      <td>0.316568</td>\n",
       "      <td>5</td>\n",
       "      <td>0.568577</td>\n",
       "      <td>0.271355</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>173</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>1.61736</td>\n",
       "      <td>79.88560</td>\n",
       "      <td>9.11581</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>3.56329</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.12277</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31932</th>\n",
       "      <td>31933</td>\n",
       "      <td>0.417404</td>\n",
       "      <td>0.810031</td>\n",
       "      <td>1.697095</td>\n",
       "      <td>9</td>\n",
       "      <td>1.779820</td>\n",
       "      <td>0.816785</td>\n",
       "      <td>2</td>\n",
       "      <td>117647.058824</td>\n",
       "      <td>0.000850</td>\n",
       "      <td>...</td>\n",
       "      <td>31485</td>\n",
       "      <td>0.08366</td>\n",
       "      <td>19.63312</td>\n",
       "      <td>22.41606</td>\n",
       "      <td>3.37348</td>\n",
       "      <td>2.05091</td>\n",
       "      <td>0.36417</td>\n",
       "      <td>13.32046</td>\n",
       "      <td>19.35385</td>\n",
       "      <td>19.38214</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31933</th>\n",
       "      <td>31934</td>\n",
       "      <td>0.237904</td>\n",
       "      <td>0.877741</td>\n",
       "      <td>1.486975</td>\n",
       "      <td>7</td>\n",
       "      <td>1.708006</td>\n",
       "      <td>0.773945</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>16533</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>12.82307</td>\n",
       "      <td>11.12766</td>\n",
       "      <td>39.07789</td>\n",
       "      <td>16.29195</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>6.59996</td>\n",
       "      <td>8.66423</td>\n",
       "      <td>4.50779</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>31934 rows × 127 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         cat  map_6_shadowback_dominance  map_6_shadowback_pielou  \\\n",
       "0          1                    0.774693                 0.441177   \n",
       "1          2                    1.170281                 0.155821   \n",
       "2          3                    1.114241                 0.307683   \n",
       "3          4                    0.529658                 0.670905   \n",
       "4          5                    0.550163                 0.692948   \n",
       "5          6                    0.151481                 0.915457   \n",
       "6          7                    0.495817                 0.723279   \n",
       "7          8                    0.402974                 0.775096   \n",
       "8          9                    0.565619                 0.684322   \n",
       "9         10                    0.615504                 0.439744   \n",
       "10        11                    0.617945                 0.554247   \n",
       "11        12                    0.783739                 0.562587   \n",
       "12        13                    0.992274                 0.446201   \n",
       "13        14                    0.397194                 0.638459   \n",
       "14        15                    1.113281                 0.308279   \n",
       "15        16                    0.384469                 0.650042   \n",
       "16        17                    0.420727                 0.696510   \n",
       "17        18                    1.166417                 0.158608   \n",
       "18        19                    0.849419                 0.387273   \n",
       "19        20                    0.788068                 0.510346   \n",
       "20        21                    0.904822                 0.495009   \n",
       "21        22                    1.173795                 0.270680   \n",
       "22        23                    0.768708                 0.522375   \n",
       "23        24                    0.770413                 0.570024   \n",
       "24        25                    1.047587                 0.349098   \n",
       "25        26                    1.254656                 0.220438   \n",
       "26        27                    0.850166                 0.386735   \n",
       "27        28                    0.866753                 0.374770   \n",
       "28        29                    0.633206                 0.543239   \n",
       "29        30                    0.741641                 0.465019   \n",
       "...      ...                         ...                      ...   \n",
       "31904  31905                    0.586662                 0.672578   \n",
       "31905  31906                    0.884348                 0.506436   \n",
       "31906  31907                    0.544956                 0.606897   \n",
       "31907  31908                    1.095905                 0.209472   \n",
       "31908  31909                    0.678298                 0.510712   \n",
       "31909  31910                    0.544666                 0.661580   \n",
       "31910  31911                    1.095834                 0.319120   \n",
       "31911  31912                    0.921634                 0.427357   \n",
       "31912  31913                    0.943234                 0.413936   \n",
       "31913  31914                    0.789784                 0.430291   \n",
       "31914  31915                    0.299604                 0.783881   \n",
       "31915  31916                    0.827453                 0.485875   \n",
       "31916  31917                    0.926510                 0.424327   \n",
       "31917  31918                    0.794901                 0.556357   \n",
       "31918  31919                    1.041905                 0.418502   \n",
       "31919  31920                    0.786770                 0.560896   \n",
       "31920  31921                    0.569683                 0.646036   \n",
       "31921  31922                    0.706554                 0.490329   \n",
       "31922  31923                    0.671756                 0.582614   \n",
       "31923  31924                    0.847219                 0.473593   \n",
       "31924  31925                    0.450412                 0.720143   \n",
       "31925  31926                    0.400964                 0.776218   \n",
       "31926  31927                    0.862406                 0.377906   \n",
       "31927  31928                    0.916352                 0.338992   \n",
       "31928  31929                    0.768018                 0.522804   \n",
       "31929  31930                    0.875352                 0.511457   \n",
       "31930  31931                    0.887180                 0.448764   \n",
       "31931  31932                    1.040861                 0.353276   \n",
       "31932  31933                    0.417404                 0.810031   \n",
       "31933  31934                    0.237904                 0.877741   \n",
       "\n",
       "       map_6_shadowback_renyi  map_6_shadowback_richness  \\\n",
       "0                    0.409695                          4   \n",
       "1                    0.088073                          4   \n",
       "2                    0.279488                          5   \n",
       "3                    0.862815                          5   \n",
       "4                    1.020990                          6   \n",
       "5                    1.537313                          6   \n",
       "6                    1.005671                          6   \n",
       "7                    1.065929                          6   \n",
       "8                    1.000676                          6   \n",
       "9                    0.358073                          3   \n",
       "10                   0.672758                          4   \n",
       "11                   0.664487                          6   \n",
       "12                   0.452768                          6   \n",
       "13                   0.695489                          3   \n",
       "14                   0.249488                          5   \n",
       "15                   0.700119                          3   \n",
       "16                   0.823578                          4   \n",
       "17                   0.089096                          4   \n",
       "18                   0.279701                          4   \n",
       "19                   0.527234                          5   \n",
       "20                   0.560870                          6   \n",
       "21                   0.205305                          5   \n",
       "22                   0.741694                          5   \n",
       "23                   0.723157                          6   \n",
       "24                   0.303876                          5   \n",
       "25                   0.169420                          5   \n",
       "26                   0.357589                          4   \n",
       "27                   0.301444                          4   \n",
       "28                   0.713622                          4   \n",
       "29                   0.494383                          4   \n",
       "...                       ...                        ...   \n",
       "31904                0.983242                          6   \n",
       "31905                0.625676                          6   \n",
       "31906                0.737414                          4   \n",
       "31907                0.147315                          4   \n",
       "31908                0.440721                          4   \n",
       "31909                0.786918                          5   \n",
       "31910                0.268876                          5   \n",
       "31911                0.382812                          5   \n",
       "31912                0.382958                          5   \n",
       "31913                0.342754                          4   \n",
       "31914                0.969324                          4   \n",
       "31915                0.436869                          5   \n",
       "31916                0.384052                          5   \n",
       "31917                0.741552                          6   \n",
       "31918                0.435553                          6   \n",
       "31919                0.690804                          6   \n",
       "31920                0.718487                          5   \n",
       "31921                0.397666                          4   \n",
       "31922                0.628961                          5   \n",
       "31923                0.474367                          5   \n",
       "31924                0.984763                          5   \n",
       "31925                1.208737                          6   \n",
       "31926                0.310638                          4   \n",
       "31927                0.240125                          4   \n",
       "31928                0.647813                          5   \n",
       "31929                0.552846                          6   \n",
       "31930                0.444995                          5   \n",
       "31931                0.316568                          5   \n",
       "31932                1.697095                          9   \n",
       "31933                1.486975                          7   \n",
       "\n",
       "       map_6_shadowback_shannon  map_6_shadowback_simpson  \\\n",
       "0                      0.611602                  0.336147   \n",
       "1                      0.216013                  0.084306   \n",
       "2                      0.495197                  0.243830   \n",
       "3                      1.079780                  0.578028   \n",
       "4                      1.241597                  0.639762   \n",
       "5                      1.640279                  0.785042   \n",
       "6                      1.295942                  0.634201   \n",
       "7                      1.388786                  0.655592   \n",
       "8                      1.226141                  0.632369   \n",
       "9                      0.483109                  0.300978   \n",
       "10                     0.768349                  0.489701   \n",
       "11                     1.008020                  0.485463   \n",
       "12                     0.799486                  0.364134   \n",
       "13                     0.701419                  0.501170   \n",
       "14                     0.496157                  0.220800   \n",
       "15                     0.714144                  0.503474   \n",
       "16                     0.965568                  0.561141   \n",
       "17                     0.219877                  0.085242   \n",
       "18                     0.536875                  0.243991   \n",
       "19                     0.821370                  0.409765   \n",
       "20                     0.886937                  0.429287   \n",
       "21                     0.435643                  0.185601   \n",
       "22                     0.840730                  0.523694   \n",
       "23                     1.021346                  0.514782   \n",
       "24                     0.561851                  0.262047   \n",
       "25                     0.354782                  0.155846   \n",
       "26                     0.536128                  0.300640   \n",
       "27                     0.519542                  0.260251   \n",
       "28                     0.753089                  0.510133   \n",
       "29                     0.644653                  0.390053   \n",
       "...                         ...                       ...   \n",
       "31904                  1.205098                  0.625904   \n",
       "31905                  0.907412                  0.465100   \n",
       "31906                  0.841338                  0.521651   \n",
       "31907                  0.290389                  0.136978   \n",
       "31908                  0.707997                  0.356428   \n",
       "31909                  1.064772                  0.544754   \n",
       "31910                  0.513604                  0.235762   \n",
       "31911                  0.687804                  0.318059   \n",
       "31912                  0.666204                  0.318158   \n",
       "31913                  0.596510                  0.290187   \n",
       "31914                  1.086690                  0.620661   \n",
       "31915                  0.781985                  0.353944   \n",
       "31916                  0.682928                  0.318904   \n",
       "31917                  0.996858                  0.523626   \n",
       "31918                  0.749854                  0.353093   \n",
       "31919                  1.004990                  0.498827   \n",
       "31920                  1.039755                  0.512511   \n",
       "31921                  0.679740                  0.328114   \n",
       "31922                  0.937682                  0.466854   \n",
       "31923                  0.762219                  0.377721   \n",
       "31924                  1.159026                  0.626472   \n",
       "31925                  1.390795                  0.701426   \n",
       "31926                  0.523889                  0.267021   \n",
       "31927                  0.469943                  0.213471   \n",
       "31928                  0.841420                  0.476811   \n",
       "31929                  0.916407                  0.424690   \n",
       "31930                  0.722258                  0.359172   \n",
       "31931                  0.568577                  0.271355   \n",
       "31932                  1.779820                  0.816785   \n",
       "31933                  1.708006                  0.773945   \n",
       "\n",
       "       map_6_shadowback_cl_13_patchnum  map_6_shadowback_cl_13_patchdensity  \\\n",
       "0                                    0                                  NaN   \n",
       "1                                    0                                  NaN   \n",
       "2                                    0                                  NaN   \n",
       "3                                    0                                  NaN   \n",
       "4                                    0                                  NaN   \n",
       "5                                    0                                  NaN   \n",
       "6                                    0                                  NaN   \n",
       "7                                    0                                  NaN   \n",
       "8                                    0                                  NaN   \n",
       "9                                    0                                  NaN   \n",
       "10                                   0                                  NaN   \n",
       "11                                   0                                  NaN   \n",
       "12                                   1                        105263.157895   \n",
       "13                                   0                                  NaN   \n",
       "14                                   0                                  NaN   \n",
       "15                                   0                                  NaN   \n",
       "16                                   0                                  NaN   \n",
       "17                                   0                                  NaN   \n",
       "18                                   0                                  NaN   \n",
       "19                                   0                                  NaN   \n",
       "20                                   2                        126984.126984   \n",
       "21                                   0                                  NaN   \n",
       "22                                   0                                  NaN   \n",
       "23                                   3                        110091.743119   \n",
       "24                                   0                                  NaN   \n",
       "25                                   0                                  NaN   \n",
       "26                                   0                                  NaN   \n",
       "27                                   0                                  NaN   \n",
       "28                                   0                                  NaN   \n",
       "29                                   0                                  NaN   \n",
       "...                                ...                                  ...   \n",
       "31904                                0                                  NaN   \n",
       "31905                                0                                  NaN   \n",
       "31906                                0                                  NaN   \n",
       "31907                                0                                  NaN   \n",
       "31908                                0                                  NaN   \n",
       "31909                                0                                  NaN   \n",
       "31910                                0                                  NaN   \n",
       "31911                                0                                  NaN   \n",
       "31912                                0                                  NaN   \n",
       "31913                                0                                  NaN   \n",
       "31914                                0                                  NaN   \n",
       "31915                                0                                  NaN   \n",
       "31916                                0                                  NaN   \n",
       "31917                                0                                  NaN   \n",
       "31918                                0                                  NaN   \n",
       "31919                                0                                  NaN   \n",
       "31920                                0                                  NaN   \n",
       "31921                                0                                  NaN   \n",
       "31922                                0                                  NaN   \n",
       "31923                                0                                  NaN   \n",
       "31924                                0                                  NaN   \n",
       "31925                                0                                  NaN   \n",
       "31926                                0                                  NaN   \n",
       "31927                                0                                  NaN   \n",
       "31928                                0                                  NaN   \n",
       "31929                                0                                  NaN   \n",
       "31930                                0                                  NaN   \n",
       "31931                                0                                  NaN   \n",
       "31932                                2                        117647.058824   \n",
       "31933                                0                                  NaN   \n",
       "\n",
       "       map_6_shadowback_cl_13_mps    ...     count_buildpixels  prop_13  \\\n",
       "0                        0.000000    ...                     0  0.00000   \n",
       "1                        0.000000    ...                     0  0.00000   \n",
       "2                        0.000000    ...                    21  0.00000   \n",
       "3                        0.000000    ...                   198  0.00000   \n",
       "4                        0.000000    ...                  1458  0.00000   \n",
       "5                        0.000000    ...                  6193  0.00000   \n",
       "6                        0.000000    ...                  1400  0.00000   \n",
       "7                        0.000000    ...                  2017  0.00000   \n",
       "8                        0.000000    ...                  1637  0.00000   \n",
       "9                        0.000000    ...                     0  0.00000   \n",
       "10                       0.000000    ...                     0  0.00000   \n",
       "11                       0.000000    ...                   347  0.00000   \n",
       "12                       0.000950    ...                  1156  0.08838   \n",
       "13                       0.000000    ...                     0  0.00000   \n",
       "14                       0.000000    ...                   743  0.00000   \n",
       "15                       0.000000    ...                     0  0.00000   \n",
       "16                       0.000000    ...                  1311  0.00000   \n",
       "17                       0.000000    ...                     0  0.00000   \n",
       "18                       0.000000    ...                     0  0.00000   \n",
       "19                       0.000000    ...                  1162  0.00000   \n",
       "20                       0.000788    ...                  6279  0.03782   \n",
       "21                       0.000000    ...                   397  0.00000   \n",
       "22                       0.000000    ...                   133  0.00000   \n",
       "23                       0.000908    ...                 11177  0.05721   \n",
       "24                       0.000000    ...                   206  0.00000   \n",
       "25                       0.000000    ...                    36  0.00000   \n",
       "26                       0.000000    ...                     0  0.00000   \n",
       "27                       0.000000    ...                    69  0.00000   \n",
       "28                       0.000000    ...                     0  0.00000   \n",
       "29                       0.000000    ...                     0  0.00000   \n",
       "...                           ...    ...                   ...      ...   \n",
       "31904                    0.000000    ...                   909  0.00000   \n",
       "31905                    0.000000    ...                  1811  0.00000   \n",
       "31906                    0.000000    ...                     0  0.00000   \n",
       "31907                    0.000000    ...                     0  0.00000   \n",
       "31908                    0.000000    ...                     0  0.00000   \n",
       "31909                    0.000000    ...                   547  0.00000   \n",
       "31910                    0.000000    ...                   417  0.00000   \n",
       "31911                    0.000000    ...                   892  0.00000   \n",
       "31912                    0.000000    ...                   211  0.00000   \n",
       "31913                    0.000000    ...                     0  0.00000   \n",
       "31914                    0.000000    ...                     0  0.00000   \n",
       "31915                    0.000000    ...                  3014  0.00000   \n",
       "31916                    0.000000    ...                   820  0.00000   \n",
       "31917                    0.000000    ...                   137  0.00000   \n",
       "31918                    0.000000    ...                    26  0.00000   \n",
       "31919                    0.000000    ...                  1792  0.00000   \n",
       "31920                    0.000000    ...                   516  0.00000   \n",
       "31921                    0.000000    ...                     0  0.00000   \n",
       "31922                    0.000000    ...                   636  0.00000   \n",
       "31923                    0.000000    ...                    49  0.00000   \n",
       "31924                    0.000000    ...                  1744  0.00000   \n",
       "31925                    0.000000    ...                  2749  0.00000   \n",
       "31926                    0.000000    ...                     0  0.00000   \n",
       "31927                    0.000000    ...                     0  0.00000   \n",
       "31928                    0.000000    ...                    72  0.00000   \n",
       "31929                    0.000000    ...                  5041  0.00000   \n",
       "31930                    0.000000    ...                    26  0.00000   \n",
       "31931                    0.000000    ...                   173  0.00000   \n",
       "31932                    0.000850    ...                 31485  0.08366   \n",
       "31933                    0.000000    ...                 16533  0.00000   \n",
       "\n",
       "        prop_14   prop_20   prop_30   prop_31   prop_41   prop_51  prop_111  \\\n",
       "0       0.00000  17.01099  79.63006   2.81252   0.00000   0.54643   0.00000   \n",
       "1       0.00000   2.87729  95.64191   1.05089   0.00000   0.42991   0.00000   \n",
       "2       0.00000   0.15909  86.35501   9.44135   0.00000   3.91605   0.00000   \n",
       "3       0.00000  26.06844  58.57774   7.41317   0.00000   7.31149   0.00000   \n",
       "4       0.00000  25.00636  52.02209   1.76592   0.00000  15.90785   2.12202   \n",
       "5       0.00000  30.25345  21.38390  23.63766   0.00000   8.91804   5.27323   \n",
       "6       0.00000  20.64206  55.25953   6.48065   0.00000   9.21692   1.36814   \n",
       "7       0.00000  11.42346  54.51495   4.73492   0.00000  14.28678   6.50213   \n",
       "8       0.00000  48.41663  35.48171   3.65050   0.00000   6.35954   3.70260   \n",
       "9       0.00000  18.34918  81.56910   0.08172   0.00000   0.00000   0.00000   \n",
       "10      0.00000  35.26711  62.07568   2.40215   0.00000   0.25505   0.00000   \n",
       "11     11.86831   8.73912  69.79167   0.76138   0.00000   7.38788   0.00000   \n",
       "12      0.00000   6.01684  78.99107   5.39120   0.00000   6.82389   0.00000   \n",
       "13      0.00000  50.00690  49.87584   0.11726   0.00000   0.00000   0.00000   \n",
       "14      0.00000   0.63222  87.92716   6.93959   0.00000   3.28095   0.00000   \n",
       "15      0.00000  50.29112  49.35524   0.35363   0.00000   0.00000   0.00000   \n",
       "16      0.00000  35.20563  55.66754   0.00000   0.00000   2.48683   6.63999   \n",
       "17      0.00000   0.49371  95.59516   2.74628   0.00000   1.16485   0.00000   \n",
       "18      0.00000   2.85655  86.57941   5.01714   0.00000   5.54690   0.00000   \n",
       "19      0.00000  15.97967  74.94286   1.73429   0.00000   4.21876   0.00000   \n",
       "20      0.00000  13.75793  73.91414   2.84020   0.00000   5.68039   0.00000   \n",
       "21      0.00000   0.16425  90.05287   4.26012   0.00000   3.48509   0.00000   \n",
       "22      0.00000  40.44650  55.85087   2.66423   0.00000   0.82495   0.00000   \n",
       "23      0.00000  18.99993  66.42628   2.46737   0.00000   6.18285   0.00000   \n",
       "24      0.00000   0.26164  85.39093   7.07028   0.00000   6.02373   0.00000   \n",
       "25      0.00000   5.43920  91.68610   2.30147   0.00000   0.56470   0.00000   \n",
       "26      0.00000  16.17332  82.03377   1.56004   0.00000   0.23286   0.00000   \n",
       "27      0.00000   0.00000  85.35832   8.97414   0.00000   5.55880   0.00000   \n",
       "28      0.00000  51.43901  47.45612   0.51680   0.00000   0.58808   0.00000   \n",
       "29      0.00000  23.94607  74.32720   0.99559   0.00000   0.73114   0.00000   \n",
       "...         ...       ...       ...       ...       ...       ...       ...   \n",
       "31904   0.00000  20.17679  54.73998  17.38262   0.00000   5.61359   0.16301   \n",
       "31905   0.00000  20.20195  70.05947   3.89465   0.00000   3.48736   0.07287   \n",
       "31906   0.00000  38.07967  57.62506   3.48183   0.00000   0.81344   0.00000   \n",
       "31907   0.00000   0.03342  92.67097   6.44910   0.00000   0.84651   0.00000   \n",
       "31908   0.00000  12.32871  78.98450   6.27243   0.00000   2.41436   0.00000   \n",
       "31909   0.00000  15.51585  63.96050  13.83402   0.00000   5.21770   0.00000   \n",
       "31910   0.00000   2.08198  86.98554   7.98135   0.00000   2.79980   0.00000   \n",
       "31911   0.00000   9.78549  81.33859   3.73936   0.00000   2.26888   0.00000   \n",
       "31912   0.00000   7.06218  81.80815   8.31494   0.00000   2.60494   0.00000   \n",
       "31913   0.00000   4.48893  83.49616  10.14227   0.00000   1.87264   0.00000   \n",
       "31914   0.00000  20.83009  52.19338  25.13913   0.00000   1.83740   0.00000   \n",
       "31915   0.00000   4.68044  79.71051   6.68654   0.00000   4.89428   0.00000   \n",
       "31916   0.00000   5.97878  78.16263   8.16645   0.00000   2.32110   0.00000   \n",
       "31917   0.00000  18.59962  65.27960  11.97098   0.44512   3.58463   0.00000   \n",
       "31918   0.00000   8.97562  78.44909   7.28642   1.60008   2.31407   0.00000   \n",
       "31919   0.00000   8.45431  68.22031  16.10474   0.00000   4.36804   0.06049   \n",
       "31920   0.00000  16.30763  67.20783   4.70392   0.00000   6.41066   0.00000   \n",
       "31921   0.00000   7.06095  81.16648   8.30603   0.00000   3.46653   0.00000   \n",
       "31922   0.00000  14.41738  70.87162   9.09412   0.00000   3.88777   0.00000   \n",
       "31923   0.00000   6.68112  77.14309  11.13499   0.00000   4.33027   0.00000   \n",
       "31924   0.00000  35.68032  47.79205   9.21554   0.00000   3.91952   0.00000   \n",
       "31925   0.00000  31.18065  39.45612   6.50613  14.13866   2.98855   0.00000   \n",
       "31926   0.00000   0.35940  82.48316  11.14427   0.00000   3.30908   0.00000   \n",
       "31927   0.00000   2.58326  88.34590   7.03366   0.00000   2.03718   0.00000   \n",
       "31928   0.00000   0.75567  64.33146  22.46334   0.00000   7.11859   0.00000   \n",
       "31929   0.00000   6.93743  74.76517   5.82594   0.00000   7.75514   0.12911   \n",
       "31930   0.00000   4.40903  75.92320  11.80054   0.00000   4.12951   0.00000   \n",
       "31931   0.00000   1.61736  79.88560   9.11581   0.00000   3.56329   0.00000   \n",
       "31932  19.63312  22.41606   3.37348   2.05091   0.36417  13.32046  19.35385   \n",
       "31933  12.82307  11.12766  39.07789  16.29195   0.00000   6.59996   8.66423   \n",
       "\n",
       "       prop_112  \n",
       "0       0.00000  \n",
       "1       0.00000  \n",
       "2       0.12850  \n",
       "3       0.62915  \n",
       "4       3.17576  \n",
       "5      10.53370  \n",
       "6       7.03270  \n",
       "7       8.53777  \n",
       "8       2.38901  \n",
       "9       0.00000  \n",
       "10      0.00000  \n",
       "11      1.45164  \n",
       "12      2.68862  \n",
       "13      0.00000  \n",
       "14      1.22009  \n",
       "15      0.00000  \n",
       "16      0.00000  \n",
       "17      0.00000  \n",
       "18      0.00000  \n",
       "19      3.12441  \n",
       "20      3.76952  \n",
       "21      2.03767  \n",
       "22      0.21346  \n",
       "23      5.86636  \n",
       "24      1.25342  \n",
       "25      0.00853  \n",
       "26      0.00000  \n",
       "27      0.10875  \n",
       "28      0.00000  \n",
       "29      0.00000  \n",
       "...         ...  \n",
       "31904   1.92400  \n",
       "31905   2.28370  \n",
       "31906   0.00000  \n",
       "31907   0.00000  \n",
       "31908   0.00000  \n",
       "31909   1.47193  \n",
       "31910   0.15133  \n",
       "31911   2.25372  \n",
       "31912   0.20979  \n",
       "31913   0.00000  \n",
       "31914   0.00000  \n",
       "31915   4.02823  \n",
       "31916   0.91947  \n",
       "31917   0.12004  \n",
       "31918   0.02663  \n",
       "31919   2.79210  \n",
       "31920   5.36997  \n",
       "31921   0.00000  \n",
       "31922   1.72911  \n",
       "31923   0.03043  \n",
       "31924   2.42485  \n",
       "31925   2.31946  \n",
       "31926   0.00000  \n",
       "31927   0.00000  \n",
       "31928   0.05496  \n",
       "31929   4.58722  \n",
       "31930   0.01536  \n",
       "31931   0.12277  \n",
       "31932  19.38214  \n",
       "31933   4.50779  \n",
       "\n",
       "[31934 rows x 127 columns]"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "# Load the .csv file in a pandas dataframe\n",
    "df=pd.read_csv(outfile_withoutnull, sep=';',header=0)\n",
    "# Display the dataframe\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Move files to dedicated folder"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Configuration files**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Set the folder where to move the configuration files\n",
    "finalfolder='/media/tais/data/Dropbox/ULB/MAUPP/Landuse_mapping/Ouagadougou/rli_config'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Folder '/media/tais/data/Dropbox/ULB/MAUPP/Landuse_mapping/Ouagadougou/rli_config' created\n"
     ]
    }
   ],
   "source": [
    "## Create the folder if does not exists\n",
    "if not os.path.exists(finalfolder):\n",
    "    os.makedirs(finalfolder)\n",
    "    print \"Folder '\"+finalfolder+\"' created\"\n",
    "## Copy the files to the final folder and remove them from the original folder\n",
    "for configfile in list_configfile:\n",
    "    shutil.copy2(configfile,finalfolder)\n",
    "    os.remove(configfile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "**Result files**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Set the folder where to move the configuration files\n",
    "finalfolder='/media/tais/data/Dropbox/ULB/MAUPP/Landuse_mapping/Ouagadougou/rli_results'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Folder '/media/tais/data/Dropbox/ULB/MAUPP/Landuse_mapping/Ouagadougou/rli_results' created\n"
     ]
    }
   ],
   "source": [
    "## Create the folder if does not exists\n",
    "if not os.path.exists(finalfolder):\n",
    "    os.makedirs(finalfolder)\n",
    "    print \"Folder '\"+finalfolder+\"' created\"\n",
    "## Copy the files to the final folder and remove them from the original folder\n",
    "for res_file in resultfiles:\n",
    "    shutil.copy2(res_file,finalfolder)\n",
    "    os.remove(res_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Copy the final csv file with all the results\n",
    "shutil.copy2(outfile,finalfolder)\n",
    "os.remove(outfile)\n",
    "shutil.copy2(outfile+'t',finalfolder)\n",
    "os.remove(outfile+'t')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Reasign the new path to the variable\n",
    "outfile=os.path.join(finalfolder,os.path.split(outfile)[-1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Copy the final csv file with all the results\n",
    "shutil.copy2(outfile_withoutnull,finalfolder)\n",
    "os.remove(outfile_withoutnull)\n",
    "shutil.copy2(outfile_withoutnull+'t',finalfolder)\n",
    "os.remove(outfile_withoutnull+'t')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Reasign the new path to the variable\n",
    "outfile_withoutnull=os.path.join(finalfolder,os.path.split(outfile_withoutnull)[-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Export the landscape polygons (with 'cat' column) as shapefile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Define the name of the shapefile with landscape units and the 'cat' column\n",
    "outputshp='/media/tais/data/Dropbox/ULB/MAUPP/Landuse_mapping/Ouagadougou/shapefile/Ouaga_subset_streetblocks.shp'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Folder '/media/tais/My_Book_1/MAUPP/Traitement/Ouagadougou/Segmentation_fullAOI_localapproach/Results/LANDUSE_MAPPING/shapefile' created\n"
     ]
    }
   ],
   "source": [
    "## Create the folder if does not exists\n",
    "dirname=os.path.dirname(outputshp)\n",
    "if not os.path.exists(dirname):\n",
    "    os.makedirs(dirname)\n",
    "    print \"Folder '\"+dirname+\"' created\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Export vector layer as shapefile\n",
    "gscript.run_command('v.out.ogr', flags='cem', overwrite=True, \n",
    "                    input=landscape_polygons, output=outputshp, format='ESRI_Shapefile')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Join the .csv file to the landscape unit polygon layer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "ename": "CalledModuleError",
     "evalue": "Module None : ['db.in.ogr', '--o', '--q', 'input=/media/tais/data/Dropbox/ULB/MAUPP/Landuse_mapping/Ouagadougou/rli_results/land_use_metrics.csv', 'output=spatial_metrics_table'] s'est terminée avec une erreur\nLe processus s'est terminé avec un code de retour non null 1. Voir les erreus dans la sortie (d'erreurs).",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mCalledModuleError\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-102-b3185f07c40c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Import .csv as new table in GRASS\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0mcsvfile\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0moutfile\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mgscript\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_command\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'db.in.ogr'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0moverwrite\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mquiet\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mTrue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minput\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcsvfile\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0moutput\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'spatial_metrics_table'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/home/tais/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass/script/core.pyc\u001b[0m in \u001b[0;36mrun_command\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    419\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    420\u001b[0m         \u001b[0mreturncode\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mps\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwait\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 421\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mhandle_errors\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mreturncode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mreturncode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    422\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    423\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/home/tais/SRC/GRASS/grass_trunk/dist.x86_64-pc-linux-gnu/etc/python/grass/script/core.pyc\u001b[0m in \u001b[0;36mhandle_errors\u001b[0;34m(returncode, result, args, kwargs)\u001b[0m\n\u001b[1;32m    330\u001b[0m         \u001b[0margs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmake_command\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    331\u001b[0m         raise CalledModuleError(module=None, code=repr(args),\n\u001b[0;32m--> 332\u001b[0;31m                                 returncode=returncode)\n\u001b[0m\u001b[1;32m    333\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    334\u001b[0m def start_command(prog, flags=b\"\", overwrite=False, quiet=False,\n",
      "\u001b[0;31mCalledModuleError\u001b[0m: Module None : ['db.in.ogr', '--o', '--q', 'input=/media/tais/data/Dropbox/ULB/MAUPP/Landuse_mapping/Ouagadougou/rli_results/land_use_metrics.csv', 'output=spatial_metrics_table'] s'est terminée avec une erreur\nLe processus s'est terminé avec un code de retour non null 1. Voir les erreus dans la sortie (d'erreurs)."
     ]
    }
   ],
   "source": [
    "# Import .csv as new table in GRASS\n",
    "csvfile=outfile\n",
    "gscript.run_command('db.in.ogr', overwrite=True, quiet=True, input=csvfile, output='spatial_metrics_table')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Join the vector layer with the new table\n",
    "gscript.run_command('v.db.join', quiet=True, map=landscape_polygons, column='cat', \n",
    "                    other_table='spatial_metrics_table', other_column='cat_')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Remove temporary raster layer with landscape units"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gscript.run_command('g.remove',flags='f',type='raster', name=raster_landscapeunits)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-**"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}