Task2_data_profiling/Data_profiling_student_version.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Processing Big Data - Data profiling\n",
    "\n",
    "© Explore Data Science Academy\n",
    "\n",
    "## Honour Code\n",
    "I {**YOUR NAME**, **YOUR SURNAME**}, confirm - by submitting this document - that the solutions in this notebook are a result of my own work and that I abide by the [EDSA honour code](https://drive.google.com/file/d/1QDCjGZJ8-FmJE3bZdIQNwnJyQKPhHZBn/view?usp=sharing).\n",
    "    Non-compliance with the honour code constitutes a material breach of contract.\n",
    "\n",
    "## Context\n",
    "\n",
    "Having completed the first step - data ingestion, the data now needs to be thoroughly prepared so that it is readable, reliable and robust. As the Data Engineer in the team, this will be your responsibility. The Data Scientists are looking to you to clean this data so that model development and deployment become seamless when the data is used in a production environment. Having completed your Data Engineering course recently, your manager Gnissecorp Atadgid, asks you to create data summaries and perform checks using the six dimensions of data quality.\n",
    "\n",
    "<div align=\"center\" style=\"width: 600px; font-size: 80%; text-align: center; margin: 0 auto\">\n",
    "<img src=\"https://github.com/Explore-AI/Pictures/raw/master/data_engineering/transform/predict/DataQuality.jpg\"\n",
    "     alt=\"Data Quality\"\n",
    "     style=\"float: center; padding-bottom=0.5em\"\n",
    "     width=100%/>\n",
    "     <p><em>Figure 1. Six dimensions of data quality</em></p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import libraries\n",
    "Below we import the libraries required to complete this section of the predict."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from pyspark import SparkContext\n",
    "from pyspark.sql import SparkSession\n",
    "from pyspark.sql import functions as F\n",
    "from pyspark.sql.types import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remember that we need a `SparkContext` and `SparkSession` to interface with Spark.\n",
    "We will mostly be using the `SparkContext` to interact with RDDs \n",
    "and the `SparkSession` to interface with Python objects.\n",
    "\n",
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    ">Initialise a new **Spark Context** and **Session** that you will use to interface with Spark."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parquet files\n",
    "In the previous section of the predict, you generated parquet files to your local directory. You will be making use of these files to continue with this section of the predict. Please make sure that your parquet files are specifically for the year **1962**. Any other year used outside of **1962** will produce incorrect answers and have a negative impact on your overall predict mark.\n",
    "\n",
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> Read the parquet files stored in your directory for the year **1962** into a Spark DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Metadata \n",
    "\n",
    "Metadata is data containing additional information about the data itself. In the cloud storage, there is a metadata file called [`symbols_valid_meta.csv`](https://processing-big-data-predict-stocks-data.s3.eu-west-1.amazonaws.com/symbols_valid_meta.csv) that is collocated with the stock market data. You will need to download this to use when performing your data quality checks.\n",
    "\n",
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> Download the metadata from the S3 bucket and read it into a Spark DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Accuracy\n",
    "Data accuracy is the degree to which data correctly describes a \"real world\" object or event.\n",
    "\n",
    "It is important to do checks to determine the basic integrity of the dataset; do the values fall within expected ranges?\n",
    "\n",
    "Most of the possible errors relating to data accuracy can occur at collection time. In our case, it is not possible to test the collection time accuracy, so we have to infer from ranges and summary statistics. Here you need to look closely at each field to see if its values make sense, with no strange surprises.\n",
    "\n",
    "In assessing accuracy, it is important to look into precision as well. Do you need seven decimals, or will one or two suffice?\n",
    "\n",
    "- **Measured by**: The degree to which the data mirrors the characteristics of the real-world object or objects it represents;\n",
    "- **Units**: The percentage of data entries that pass the data accuracy rules;\n",
    "- **Related to**: Validity, Uniqueness, Consistency."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> Generate summary statistics to explore your data. Make sure you understand the ranges, means, extremums, and deviations found in the data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> Generate histograms for the six numerical attributes found in the data to understand the distribution of values.\n",
    ">\n",
    ">*You may use as many coding cells as necessary.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> Investigate the **open** column to identify stocks that have open values greater than 2, and note any anomalies that you find in the data.\n",
    ">\n",
    ">*You may use as many coding cells as necessary.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " ## 📔️**Notes** 📔️\n",
    "\n",
    " *Use this cell to note down any potential findings.*\n",
    "\n",
    " 1.\n",
    " 2.\n",
    "    ...\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> Investigate **high**, **low**, **close**, and **adj_close** to determine if any stocks may be deviating from the normal ranges of the data set. Note down the stock(s) that you come across.\n",
    ">\n",
    ">*You may use as many coding cells as necessary.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " ## 📔️**Notes** 📔️\n",
    "\n",
    " *Use this cell to note down any potential findings.*\n",
    "\n",
    " 1.\n",
    " 2.\n",
    "    ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "## Completeness\n",
    "\n",
    "Completeness is the proportion of stored data against the potential of “100% complete\". This is the degree to which the required data is in the dataset. \n",
    "\n",
    "Does the dataset have missing values, or if it is time-series data, does it have time period gaps? Has a bias been introduced that may change your assumptions or affect your results?\n",
    "\n",
    "Completeness issues can occur at the row level (gaps within the dataset) or the field level (one entry missing). At the field level, entire fields can being empty, or >80% of a field's data missing. \n",
    "\n",
    "Another issue that may occur is default values. A typical example of this is where a logger sends back a 0 instead of a null value, which can greatly skew any attempts at modelling. This is where it is instrumental to employ domain knowledge when assessing a dataset. \n",
    "\n",
    "- **Measured by**: A measure of the absence of blank (null) values or the presence of non-blank values;\n",
    "- **Units**: Percentage;\n",
    "- **Related to**: Validity and Accuracy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Missing values\n",
    "\n",
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> Write code to identify and count the number of missing values (nulls) in the dataset. Include a percentage to describe the proportion of missing values per column. Output the results in the following manner:\n",
    ">\n",
    "> `There are <number_of_missing_values> (<percentage>) null values in <column_name> column`\n",
    ">\n",
    ">*You may use as many coding cells as necessary.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Columns with missing values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> From the above result, probe the columns that are affected by the missing data to find out which stocks were affected.\n",
    ">\n",
    ">*You may use as many coding cells as necessary.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Fix Completeness\n",
    "\n",
    "How do we deal with incomplete data?\n",
    "- Dropping missing values\n",
    "- Discard the incomplete column\n",
    "- Discard the rows containing missing data\n",
    "- Case deletion\n",
    "\n",
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> Use the appropriate strategy to remedy the missing data. \n",
    ">\n",
    ">*You may use as many coding cells as necessary.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Zero Values\n",
    "\n",
    "Take a deeper look into the entries with many zero values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Fix Completeness\n",
    "\n",
    "Completeness issues can be fixed through imputation of the missing data through:\n",
    "- imputation by mean/mode/median;\n",
    "- regression; or\n",
    "- KNN.\n",
    " \n",
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> Write code to identify and count the number of zeros (0) in the dataset. Include a percentage to describe the proportion of missing values per column. Output the results in the following manner:\n",
    ">\n",
    "> `There are <number_of_zeros> (<percentage>) zero values in <column_name> column`\n",
    ">\n",
    ">*You may use as many coding cells as necessary.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the above section, you find that there are a few columns that contain zero values. However, some of these are true zeros and are explainable. Your task is to distinguish which column should undergo data imputation.\n",
    "\n",
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> Investigate the columns with zero values and determine which one should undergo data imputation. Take note of the stock and month on which zero values occurred.\n",
    ">\n",
    ">*You may use as many coding cells as necessary.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> Once you have identified the column that needs to undergo imputation, update the values for the affected records by using the average value for the affected stock.\n",
    ">\n",
    ">*You may use as many coding cells as necessary.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Consistency\n",
    "\n",
    "Consistency is the absence of difference when comparing two or more representations of a thing against a reference.\n",
    "\n",
    "Data entries that refer to the same record or entity have to be consistent across all entries, e.g., if you are dealing with records from a logger in the field, the entries for that logger have to remain consistent, and the name or primary key of that logger cannot change from one entry to another. \n",
    "\n",
    "For example, 'Logger1', 'Loger1' and 'Logge1' are examples of inconsistent keys. \n",
    "\n",
    "This is not just within a single table but also becomes more important if you are dealing with relational data. In which case, the mappings between tables and systems must be consistent. If not, the relationships will be completely lost between the tables and referential integrity compromised. \n",
    "\n",
    "- **Measured by**: Analysis of pattern and/or value frequency;\n",
    "- **Units**: Percentage;\n",
    "- **Related to**: Accuracy, Validity, and Uniqueness."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> There currently exists a stock that has inconsistent naming. Make use of the metadata to determine which stock is inconsistently named, then update the dataframe appropriately to get rid of this inconsistency.\n",
    ">\n",
    ">*You may use as many coding cells as necessary.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Timeliness\n",
    "\n",
    "Timeliness is the degree to which data represent reality from the required point in time.\n",
    "\n",
    "Timeliness expects that the data within your dataset is sufficiently up to date. If you are trying to answer questions that relate to recent problems, having timely data is extremely important. For example, you cannot use current flight patterns to model how many aeroplanes will be required by a large aeronautics company within the next 5-10 years. \n",
    "\n",
    "Similarly, when answering questions that require real-time answers (e.g., predicting when a pipe will burst in a manufacturing plant), you have to be set up to receive real-time data from sensors and loggers. \n",
    "\n",
    "- **Measured by**: Time difference;\n",
    "- **Units**: Time;\n",
    "- **Related to**: Accuracy because it will decay as time progress. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is important to see the latest value for each of the stocks that we are looking at:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-----+----------+\n",
      "|stock| max(Date)|\n",
      "+-----+----------+\n",
      "|   AA|1962-12-31|\n",
      "|  XOM|1962-12-31|\n",
      "|  DIS|1962-12-31|\n",
      "|   PG|1962-12-31|\n",
      "|   GT|1962-12-31|\n",
      "|   MO|1962-12-31|\n",
      "|  IBM|1962-12-31|\n",
      "|  JNJ|1962-12-31|\n",
      "|  CVX|1962-12-31|\n",
      "|  DTE|1962-12-31|\n",
      "|   BA|1962-12-31|\n",
      "|   GE|1962-12-31|\n",
      "|  HPQ|1962-12-31|\n",
      "| ARNC|1962-12-31|\n",
      "|  CAT|1962-12-31|\n",
      "|   IP|1962-12-31|\n",
      "|   FL|1962-12-31|\n",
      "|   ED|1962-12-31|\n",
      "|  NAV|1962-12-31|\n",
      "|   KO|1962-12-31|\n",
      "+-----+----------+\n",
      "\n"
     ]
    }
   ],
   "source": [
    "df.groupBy('stock').agg(F.max('Date')).show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes, some of these axes of data quality will be less important than others. \n",
    "This is one of those cases where it is less important to have timely data, since \n",
    "we are trying to create a training dataset for a stock market prediction algorithm. \n",
    "\n",
    "It is important to know the context in which you are doing your modelling. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gaps in the dataset\n",
    "\n",
    "Let's see if we can find inconsistencies in the time series by having a look at the number of entries for each of the tickers.\n",
    " \n",
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> Uncomment and use the below code to determine which dates had entries that were not equal to 20. You may have to change the name of the dataframe to see the resultant output\n",
    ">\n",
    ">*You may use as many coding cells as necessary.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#df.orderBy('date').groupby('date').count().where(F.col('count') != 20).show(400)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> From the above result, investigate the number of times a stock appears for the given month. You can infer the months by using the output of the previous cell.\n",
    ">\n",
    ">*You may use as many coding cells as necessary.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Uniqueness\n",
    "\n",
    "Uniqueness requires that nothing will be recorded more than once based upon how that thing is identified. It is the inverse of an assessment of the level of duplication.\n",
    "\n",
    "Each entry within the dataset should only relate to a single event that has occurred and thus should not be duplicated. This is largely mediated by having the appropriate primary key, which means sticking to the requirements of a good primary key. All fields in the tables should be non-transitively dependent on the primary key.\n",
    "\n",
    "As such, deduplication of the dataset may be required. \n",
    "\n",
    "- **Measured by**: Analysis of the number of things assessed in the “real world” compared to the number of records of things in the dataset. This requires a reference dataset which is the ground truth;\n",
    "- **Units**: Percentage;\n",
    "- **Related to**: Consistency."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Duplication Test\n",
    "For time-series data, it is important to check for duplications, as we typically expect all values to be unique within the dataset.\n",
    "\n",
    "The first thing to check will be if the primary key values within the dataset are unique - in our case, that will be a combination of the stock name and the date.\n",
    "\n",
    "Secondly, we want to check if the entries are all unique, which is done by checking for duplicates across that whole dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> Write code to determine if there are any duplicates within the data, and then proceed to correct this by dropping them from the dataframe.\n",
    ">\n",
    ">*You may use as many coding cells as necessary.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Validity\n",
    "Data is valid if it conforms to the syntax (format, type, range) of its definition.\n",
    "\n",
    "Certain values within a field may have specific criteria required to make it valid, e.g., numerical columns cannot contain alphabetical characters, which can occur due to scientific notation.\n",
    "\n",
    "This can be more difficult to determine in stings, in which case you may have to check using regex. \n",
    "\n",
    "- **Measured by**: Comparison between the data and metadata or documentation for the data item;\n",
    "- **Units**: Percentage of data items deemed Valid or Invalid;\n",
    "- **Related to**: Accuracy, Completeness, Consistency, and Uniqueness."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to first define what we expect from our dataset:\n",
    "\n",
    "- stock: string (nullable = true) => Should be contained in the list of expected tickers\n",
    "- date: date (nullable = true) => Should conform to date format, and be in the past\n",
    "- open: double (nullable = true) => Should be positive or 0\n",
    "- high: double (nullable = true) => Should be positive or 0\n",
    "- low: double (nullable = true) => Should be positive or 0 (should be < high)\n",
    "- close: double (nullable = true) => Should be positive or 0 (should be <= high )\n",
    "- adj_close: double (nullable = true) => Should be positive or 0\n",
    "- volume: integer (nullable = true) => Should be positive or 0\n",
    "- high_avg: double (nullable = true) => Derived (not needed to test)\n",
    "- high_imp: double (nullable = true) => Derived (not needed to test)\n",
    "- day_of_week: string (nullable = true) => Derived (not needed to test)\n",
    "\n",
    "*Is there any other logic that we can incorporate?*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    "> Use the metadata to check if all the stocks in your current dataframe are valid. In other words, make sure you have no foreign/unknown tickers in your dataframe.\n",
    ">\n",
    ">*You may use as many coding cells as necessary.*\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    ">\n",
    ">Check if the date column contains only valid dates and all dates are in the past.\n",
    ">\n",
    ">*Valid dates should already be checked in the data reading step.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ℹ️ **Instructions** ℹ️\n",
    ">\n",
    ">Check that all of numerical columns are positive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: Write your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Save Updates\n",
    "\n",
    "With our initial work of checking the various dimensions of the data quality completed, we can now save these results to a CSV file.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd_df = df.toPandas()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd_df.to_csv('./<name>_<surname>_data_profiling.csv')"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "24a0a2ddc4dffcb168e507551dd24967ddc40ea2df7a72a200a74e0ae6d88beb"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}