how_to_log.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logging in python\n",
    "You know when you have coded your biggest project and every time it runs you can barely figure out what is doing only by reading a series of print statements and the creation of strategically saved files?\n",
    "Well if that is the case, you ought to learn logging and step up your game. \n",
    "With a proper system of logging. you will have a consistent, ordered and a more reliable way to understand your own code, to time and track its progression and capture bugs easily.\n",
    "\n",
    "Let’s break down the advantages of logging:\n",
    "1.\tFormatting: Logging allows you to standardize every message using a format of your choosing.\n",
    "2.\tTime tracking: Alongside the message you can add the time when it is generated. \n",
    "3.\tCompact: All messages are gathered in files, you don’t need to scroll up continuously. \n",
    "4.\tVersatility: Print do not works everywhere (i.e., objects without __str__ methods).\n",
    "5.\tFlexibility: Logging allows different levels of importance to your messages so you regulate what to show.\n",
    "With all of this, you won’t be the only one who can understand your code.\n",
    "\n",
    "Let’s start!\n",
    "\n",
    "## Functions of logging\n",
    "“logging” is a standard library of python, therefore, you won’t need to install it. Simply type\n",
    "import logging\n",
    "\n",
    "And you are good to go.\n",
    "At its core there are only five functions of logging that we are going to use all the times, those are:\n",
    "- logging.debug and logging.info: Conceptually, These two are the closest to a normal print. They report events following the normal run of a program.\n",
    "    - Logging.info: It’s meant to confirm that things are working as expected.\n",
    "    - Logging.debug: These are messages expected to arise during debugging and it should carry detailed information.\n",
    "- Logging.warning: This is a low level error message, meaning that there is an issue present that won’t stop the flow of the program but it has to be noted by the user.\n",
    "- logging.error: This message is meant when an issue has arisen that won’t allow the program to execute some functions, but it won’t stop the program. \n",
    "- logging.critical: This the highest level of error messages. A critical means that a serious issue is present and the program may stop.\n",
    "\n",
    "Let’s see an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "logging.debug('A debug message, expected but not welcomed')\n",
    "logging.info('An info message, expected and welcomed')\n",
    "\n",
    "logging.warning('A warning message, an issue may arise')\n",
    "logging.error('An error message, an issue has arise')\n",
    "logging.critical('A critical message, an issue has compromised the program')\n",
    "\n",
    "# WARNING:root:A warning message, an issue may arise\n",
    "# ERROR:root:An error message, an issue has arise\n",
    "# CRITICAL:root:A critical message, an issue has compromised the program"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By running this short code you can see that the first two messages have not been printed, to your terminal or notebook. \n",
    "\n",
    "This is due to the low level of severity that logging gives to these messages by default. \n",
    "These levels of severity are: \n",
    "1.\tdebug (Level 10)\n",
    "2.\tinfo (Level 20)\n",
    "3.\twarning (Level 30)\n",
    "4.\terror (Level 40)\n",
    "5.\tcritical (Level 50)\n",
    "\n",
    "In a sense, python tries to be clever. If a message like debug and info is expected, there is no reason to fill the entire terminal with these (imagine you have hundreds of them).\n",
    "\n",
    "But for all the others the user has to be notified at every stage (However, you can modify the levels for each function if you wish).\n",
    "\n",
    "## Create a log file\n",
    "Now, let’s see how we can create a log file. \n",
    "Note: If you have followed along and tried the previous script, you need to restart your python terminal (or kernel) otherwise the following script won’t work, we will see why shortly.\n",
    "\n",
    "To create a log file we will use the function basicConfig() this should be executed before any other logging function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "\n",
    "logging.basicConfig(filename='my_log.log', level=logging.DEBUG)\n",
    "\n",
    "logging.debug('Message logged in file')\n",
    "logging.error('Message logged in file and console')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let’s see what are basicConfig() has done:\n",
    "\n",
    "With its first parameter “filename” we chose the name of the log file, this is pretty straightforward. But with the second one “level” we have decided what level of message we want our file to have. Indeed, by default it won’t log info() and debug() messages. \n",
    "\n",
    "Other two important parameter of basicConfig() are \n",
    "- filemode: In what mode to open the file. The default is ‘a’, which means append.\n",
    "- format: In what way you want the message string to appear\n",
    "- datefmt: A specified date/time format\n",
    "\n",
    "Let’s see an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "\n",
    "logging.basicConfig(\n",
    "    filename='my_log.log',\n",
    "    level=logging.DEBUG,\n",
    "    filemode= 'w',\n",
    "    format = '%(levelname)s: %(asctime)s %(message)s',\n",
    "    datefmt='%m/%d/%Y'\n",
    ")\n",
    "\n",
    "logging.debug('Bug here')\n",
    "logging.error('Here an error')\n",
    "# DEBUG: 05/10/2020 Bug here\n",
    "# ERROR: 05/10/2020 Here an error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the previous script did not work is because you have not restarted Python. To solve this problem try to run this line:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Logging.root.handlers[:]\n",
    "# [<StreamHandler <stderr> (NOTSET)>]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the result is not an empty list runs the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for handler in logging.root.handlers[:]:\n",
    "    logging.root.removeHandler(handler)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let’s see what all that means later. *\n",
    "# Classes of logging\n",
    "## Logger\n",
    "Until now we have seen the basic functions of logging and every time, they were the modules of a class logger called by default root. \n",
    " \n",
    "It is good practice to define a new name of the class logger, in your code. Indeed, you can customize your logger and adapt to the needs of your project.\n",
    " \n",
    "Commonly a logger is configured using:\n",
    "- logger.setLevel() set the the lowest severity for the log messages similarly to what we have seen with basicConfig()\n",
    "- logging.Formatter(): How to format the messages\n",
    "- logger.addHandler() and Logger.removeHandler() add and remove handler objects from the logger object. \n",
    "- logger.addFilter() and Logger.removeFilter() add and remove filter objects from the logger object. \n",
    "\n",
    "# Handler\n",
    "Handlers are used to send the logs to one or more output destinations. Each logger can have more than one handler. And like with loggers you can configure the level of severity of a message, formats, etc.\n",
    "Therefore, in each project you can have more than one logger, dealing with different parts and more than one handlers to deal with one or more logging files.\n",
    "\n",
    "# Formatter\n",
    "Like we saw with basicConfig() we can format our messages in different ways obtaining different string formats.\n",
    "There are many other classes (e.g.: Filters) but I’ll leave to you to explore there more let’s see now an example: \n",
    "\n",
    "Example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "\n",
    "# create logger                                                                                                      \n",
    "logger = logging.getLogger('name_of_logger')\n",
    "logger.setLevel(logging.DEBUG) # Set to the lowest level                                                             \n",
    "\n",
    "# Create an handler                                                                                                  \n",
    "handler = logging.FileHandler('log_file.log', mode='w')\n",
    "handler.setLevel(logging.DEBUG)\n",
    "\n",
    "# Create a second handler for console printing                                                                         \n",
    "ch = logging.StreamHandler()\n",
    "ch.setLevel(logging.DEBUG)\n",
    "\n",
    "# Create a formatter                                                                                                 \n",
    "formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s',datefmt='%m/%d/%Y')\n",
    "\n",
    "# Add formatter to handlers                                                                                         \n",
    "handler.setFormatter(formatter)\n",
    "ch.setFormatter(formatter)\n",
    "\n",
    "# Add handler to logger                                                                                            \n",
    "logger.addHandler(handler)\n",
    "logger.addHandler(ch)\n",
    "\n",
    "# Now let's start the code                                                                                          \n",
    "logger.debug('debug message')\n",
    "logger.info('info message')\n",
    "logger.warning('warn message')\n",
    "logger.error('error message')\n",
    "logger.critical('critical message')\n",
    "\n",
    "# Result in file log_file.log\n",
    "# 05/10/2020 - name_of_logger - DEBUG - debug message\n",
    "# 05/10/2020 - name_of_logger - INFO - info message\n",
    "# 05/10/2020 - name_of_logger - WARNING - warn message\n",
    "# 05/10/2020 - name_of_logger - ERROR - error message\n",
    "# 05/10/2020 - name_of_logger - CRITICAL - critical message"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using this “Hello World” you should be able to create every logging system you might require.\n",
    "\n",
    "* By using logging.info() before logging.basicConfig() we have already called the logger root, therefore logging.basicConfig() won’t work. We have to remove everything connected to root before starting again."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}