examples/Intro_Python_Geo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Python for Geoscientists\n",
    "\n",
    "This is a \"Jupyter Notebook\". Formerly an IPython Notebook. It is a good way to document workflows and analysis pipelines. You can have Python, R, Fortran, Julia, etc notebooks. This is a \"Markdown\" cell, you can write notes, equations in latex style, $E=\\kappa A ^m (\\nabla z)^n$, or embed figures. \n",
    "\n",
    "Today we are using a Python kernel to do some cool stuff with Geoscience data. The following cell is your first Python code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "2+4*10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#This is a comment. This is for a human to read (so you remember what your code does!)\n",
    "#Python ignores anything behind the '#'.\n",
    "\n",
    "#The next line is an example of a 'variable'. Assign values using a single '=' sign.\n",
    "time=145"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: A variable (e.g. time) must start with a letter or underscore, and can include a number"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Now you can use that variable in different ways.... firstly print it out to the screen\n",
    "print(\"The age of the sample is\", time, \" Million years\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Make a new variable called 'endtime' and add a constant to our 'time' variable\n",
    "endtime=time+56"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Nothing printed out above? Good. Jupyter Notebooks won't always do that so let's tell Python to print it.\n",
    "print(endtime)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Make a new 'string' variable\n",
    "geological_age='Jurassic'\n",
    "\n",
    "#Print out some useful information that includes our different variables\n",
    "print(\"My sample is \", endtime, \" Million years old from the\", geological_age, \" age.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Make a Python List object, similar to an array.\n",
    "times=[1,4.5,5+3.2,geological_age,\"Another string\",True]\n",
    "\n",
    "print(times)\n",
    "\n",
    "#There are many different types of data types and objects: \n",
    "#int, long, float, complex, NaN, logical, String, Lists, Tuple, Dictionary, functions, classes, etc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#indexing\n",
    "print(times[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(times[4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(times[4][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Functions\n",
    "These are bits of code you want to perhaps use many times, or keep self contained, or refer to at different points. They can take values as input and give values back (or not). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Declare the name of the function\n",
    "def add_numbers(x,y):\n",
    "    '''adds two numbers\n",
    "    usage: myaddition=addnumbers(x,y)\n",
    "    returns: z\n",
    "    inputs: x,y\n",
    "    x and y are two integers\n",
    "    z is the summation of x and y\n",
    "    '''\n",
    "    \n",
    "    z=x+y\n",
    "    \n",
    "    return(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note the indentation - Python forces your code to be nicely readable by using 'whitespace'/indentation to signify what chunks of code are related. You will see this more later.\n",
    "    \n",
    "Many functions have a header - formatted as a multiline comment with three '''. This hopefully will tell you about the function\n",
    "\n",
    "Anyway, let's run our function, now that we have initialised it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "add_numbers(1,2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loops, operators, conditions\n",
    "Python is great for doing something a million times. It can be useful if you have many samples/datapoints and you want to operate or manipulate those points."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Loop through our list 'times' that we defined above\n",
    "for mything in times:\n",
    "    print(mything)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes you need to loop through a list, but simultaneously keep track of which index you're up to."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for myindex, mything in enumerate(times):\n",
    "    print(\"index:\",myindex,\" The thing in my 'times' list:\",mything)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You don't always need a pre-defined list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "age=140\n",
    "# What is the value of \"time\" ?\n",
    "# age < time is a \"logical\" data-type. It's either True or False\n",
    "while age < time:\n",
    "    print(\"time:\", time, \" age:\", age, \" difference:\",time-age)\n",
    "    age=age+1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Control statements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Control statements: if, for, while, try, \n",
    "if time < 200:\n",
    "    print(time)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if time <= 200:\n",
    "    print(geological_age)\n",
    "elif time > 200:\n",
    "    print(\"Triassic age\")\n",
    "else:\n",
    "    pass #This option is not necessarily needed, but can be useful in some scenarios"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Another function\n",
    "def timescale(t):\n",
    "    print(t)\n",
    "    if (t <= 4500) & (t > 2500):\n",
    "        return(\"Archean\")\n",
    "\n",
    "    elif (t <=2500) & (t > 541):\n",
    "        return(\"Proterozoic\")\n",
    "\n",
    "    elif (t <= 541) & (t > 252):\n",
    "        return(\"Palaeozoic\")\n",
    "\n",
    "    elif (t <=252) & (t > 65):\n",
    "        return(\"Palaeozoic\")\n",
    "\n",
    "    elif (t <=65) & (t >= 0):\n",
    "        return(\"Cenozoic\")\n",
    "\n",
    "    else:\n",
    "        print (\"Expect number between 0 and 4500, got:\",t)\n",
    "        return(float('nan'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "timescale(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That is the basics. Now we are going to load in some data and manipulate it.\n",
    "\n",
    "## Loading data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#First we have to load some modules to do the work for us.\n",
    "#Modules are packages people have written so we do not have to re-invent everything!\n",
    "\n",
    "#The first is NUMerical PYthon. A very popular matrix, math, array and data manipulation library.\n",
    "import numpy\n",
    "\n",
    "#Pandas is a module that is great for dealing with tables of data\n",
    "import pandas\n",
    "\n",
    "#This is a library for making figures (orignally based off Matlab plotting routines)\n",
    "#We use the alias 'plt' because we don't want to type out the whole name everytime we reference it!\n",
    "import matplotlib.pyplot as plt "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Set the variable name for the file we are loading in. \n",
    "#It is in the 'data' directory, and the file is called EarthChemCU.txt. \n",
    "#We are currently working in /examples.\n",
    "filename = '../data/EarthChemCU.txt'\n",
    "\n",
    "#Now read in the data\n",
    "# loadtxt() is a function that we can now use because we loaded the library called numpy\n",
    "chemdata=numpy.loadtxt(filename, delimiter=',')\n",
    "#chemdata <- the name of a variable we are making that will hold the table of data\n",
    "#filename <- this is the name of the variable we declared above\n",
    "#delimiter <- this is a csv file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Want more details about a command/function we use?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#Try this help command\n",
    "help(numpy.loadtxt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploring your data\n",
    "#### It is often a good idea to look at the data to have some idea of what you are working with"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#What does the data look like. Print it out\n",
    "print(chemdata)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#This is in the style: Latitude, Longitude(-180:180), Age(Ma), pp\n",
    "#Print the dimensions of the data\n",
    "print(chemdata.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "207431 rows! A good example for why we use Python and not something like Excel.\n",
    "\n",
    "### Accessing data from an array\n",
    "chemdata is a table of data: an array with two dimensions. So to access/look at/change parts of it, we need to specify both row and column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Print the number in the first row and third column. \n",
    "#IMPORTANT: Python counts from 0\n",
    "print(chemdata[0,2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Print the first row\n",
    "print(chemdata[0,:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Print the third column\n",
    "print(chemdata[:,2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Print the first two columns for row id 2, 5 and 6. \n",
    "print(chemdata[[2,5,6],0:2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting data\n",
    "Now to make our first plot!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Plot the lats and lons, i.e. the first column vs the second column\n",
    "plt.plot(chemdata[:,1],chemdata[:,0],'k.')\n",
    "plt.title('Copper Deposit Data')\n",
    "plt.ylabel('Latitude')\n",
    "plt.xlabel('Longitude')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This does not look right... It is a messy dataset! This is not uncommon. \n",
    "Maybe the Lats/Lons are stored as Norhtings/Eastings for some samples. \n",
    "Maybe they are missing a decimal place.\n",
    "\n",
    "Anyway, Python is a great tool to clean things up! Let's investigate further."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Plot the Latitudes\n",
    "plt.plot(chemdata[:,0])\n",
    "plt.ylabel('Latitude')\n",
    "plt.xlabel('Number')\n",
    "plt.show()\n",
    "\n",
    "#Plot the Longitudes\n",
    "plt.plot(chemdata[:,1],'r')\n",
    "plt.ylabel('Longitude')\n",
    "plt.xlabel('Number')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are definitely some outliers with latitudes and longitudes. So let's restrict just our points to range -180:180 and -90:90."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Clean up the data, remove anything outside lat lon extent\n",
    "\n",
    "#Find all the \"chemdata\" column 1 (i.e. longitude) data points that are greater than -180, save it in a new variable\n",
    "where_longitude_above_min = chemdata[:,1]>-180\n",
    "cudata=chemdata[where_longitude_above_min]\n",
    "#Repeat for less than 180\n",
    "where_longitude_below_max = cudata[:,1]<180\n",
    "cudata2=cudata[where_longitude_below_max]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can make that a litte clearer for the latitude values to see what is actually going on..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "goodlat=cudata2[:,0]<90\n",
    "print(goodlat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "goodlat.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cudata2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "#Repeat for latitudes less than 90\n",
    "cudata3=cudata2[cudata2[:,0]<90]\n",
    "#Repeat for greater than -90\n",
    "cudata4=cudata3[cudata3[:,0]>-90]\n",
    "\n",
    "\n",
    "print(\"We have removed\", chemdata.shape[0]-cudata4.shape[0], \"samples\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.plot(cudata4[:,1],cudata4[:,0],'k.')\n",
    "plt.title('Copper Deposits from EarthChem.org')\n",
    "plt.ylabel('Latitude')\n",
    "plt.xlabel('Longitude')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "lats=cudata4[:,0]\n",
    "longs=cudata4[:,1]\n",
    "age=cudata4[:,3]\n",
    "copper=cudata4[:,2]\n",
    "\n",
    "#lats_rich=lats[copper>2]\n",
    "\n",
    "fig = plt.figure(figsize=(6,4),dpi=150)\n",
    "\n",
    "plt.scatter(longs,lats,s=age/1000,c=copper,vmin=0, vmax=1000,cmap=plt.cm.copper)\n",
    "plt.title('Copper Deposits from EarthChem.org')\n",
    "plt.ylabel('Latitude')\n",
    "plt.xlabel('Longitude')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "def reject_outliers(data):\n",
    "    m = 2\n",
    "    u = np.mean(data)\n",
    "    print(\"mean is:\", u)\n",
    "    s = np.std(data)\n",
    "    print(\"std is:\", s)\n",
    "    filtered = [e for e in data if (u - 2 * s < e < u + 2 * s)]\n",
    "    print(\"removed:\",np.shape(data)[0] - np.shape(filtered)[0])\n",
    "    return filtered"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "filtered_age=reject_outliers(copper)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.plot(copper[copper>1],'k.')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's make a nicer map"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Import another module called Basemap - great for plotting things on globes\n",
    "from mpl_toolkits.basemap import Basemap\n",
    "from sklearn import preprocessing\n",
    "\n",
    "#Make new variables from our array (so it is easier to see what we are doing)\n",
    "lats=cudata4[:,0]\n",
    "longs=cudata4[:,1]\n",
    "age=cudata4[:,3]\n",
    "\n",
    "#######\n",
    "## Make the figure\n",
    "#######\n",
    "\n",
    "#Create a figure object\n",
    "fig = plt.figure(figsize=(16,12),dpi=150)\n",
    "\n",
    "#Make the basemap, shade it and put down some other map symbols\n",
    "pmap = Basemap(projection='hammer', lat_0=0, lon_0=0, #You can break loooong commands over multiple lines\n",
    "           resolution='l')\n",
    "pmap.drawmapboundary(fill_color='white')\n",
    "pmap.fillcontinents(color='grey', lake_color='white', zorder=0)\n",
    "pmap.drawmeridians(numpy.arange(0, 360, 30))\n",
    "pmap.drawparallels(numpy.arange(-90, 90, 30))\n",
    "\n",
    "#compute native map projection coordinates of lat/lon grid.\n",
    "xh, yh = pmap(longs, lats)\n",
    "\n",
    "#Make a scatter plot of the data coloured by age. Restrict the colour range between 0 and 2000\n",
    "#And also set the 'plot' as a variable so we can reference it\n",
    "mapscat = pmap.scatter(xh,yh,marker=\".\",c=age,vmin=0,vmax=100)\n",
    "\n",
    "#Add a colourbar, \n",
    "cbar=pmap.colorbar(mapscat,location='bottom')\n",
    "cbar.set_label('Age (Ma)')\n",
    "\n",
    "# Add a map title, and tell the figure to appear on screen\n",
    "plt.title('Age of Copper Deposits in the EarthChem.org database')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}