Merge pull request #44 from Praktyka-Zawodowa-2020/dev

OGR v1.0

.gitignore

@@ -0,0 +1,6 @@
+# gitignore file
+__pycache__
+.idea
+venv
+graphs
+*.xlsx

README.md

@@ -1 +1,175 @@
-Optical Graph Recognition (OGR)
+# Optical Graph Recognition (OGR) - script
+Our OGR implementation is a python algorithm for recognising graphs 
+([in terms of discrete mathematics](https://en.wikipedia.org/wiki/Graph_(discrete_mathematics))) in given images and 
+photos (e.g. graphs from publications, hand drawn ones, etc.).
+
+<img align="right" width="62%" src="./readme_imgs/input_and_output.jpg">
+
+Table of contents:
+- [Introduction](#introduction)
+- [Technologies](#technologies)
+- [Running the script](#running-the-script)
+- [Processing phases](#processing-phases)
+  * [Preprocessing](#preprocessing) 
+  * [Segmentation](#segmentation)
+  * [Topology recognition](#topology-recognition)
+  * [Postprocessing](#postprocessing)
+- [Future development](#future-development)
+- [References and Authors](#references-and-authors)
+
+## Introduction
+
+Running the algorithm (script) results in files with
+information about recognised graph. Example is given above - input image on the left, and resulting GraphML file on the 
+right (opened in [yEd Live](https://www.yworks.com/yed-live/) graph editor).
+
+## Technologies 
+This project was developed using following technologies:
+- Language: Python v3.7,
+- Libraries:
+  * OpenCV v4.4.0.40 (opencv-contrib-python),
+  * Numpy v1.19.1.
+- Environment: PyCharm Community v2020.2.1,
+- Version control: git and github.
+
+## Running the script
+### Read before the first use
+The best results of recognition will be achieved when the following conditions are met. Keep in mind that these are 
+perfect properties that are hard to achieve and even in images of poor quality graphs are usually recognised quite well:
+- Image:
+  * is uniformly lighted across whole area,
+  * graph is the only foreground object in the image,
+  * background is uniform across whole area,
+  * vertices (borders if unfilled) and edges color contrasts from background significantly.
+- Vertices:
+  * are **circular** (especially for unfilled vertices),
+  * have similar size,
+  * are filled (although unfilled ones are also recognised),
+  * unfilled ones are closed contours.
+- Edges:
+  * are thicker than background noise (lines, grid, etc),
+  * are straight lines,
+  * do not intersect (although intersecting ones are also recognised).
+
+
+Before running the script make sure that in your system you have installed python and libraries mentioned in the 
+([Technologies](#technologies)) paragraph. 
+
+OGR script has been developed as part of a bigger project ([repository](https://github.com/Praktyka-Zawodowa-2020)) 
+including [mobile application](https://github.com/Praktyka-Zawodowa-2020/optical_graph_recognition_mobApp) and 
+[server](https://github.com/Praktyka-Zawodowa-2020/optical_graph_recognition_server) on which the file format is 
+validated. Therefore in order for the script to work independently:
+
+**Make sure that your input files are images and have .jpg or .png extensions!**
+
+### Arguments
+You can read arguments descriptions below or display them by typing in the command line:
+
+`python <path_to_main.py> -h`
+
+Required:
+
+  - `-p (--path) <absolute_path_to_image>` - **absolute!** path to input image (must be given).
+
+Optional (*default* values are chosen when these arguments are not given):
+
+- `-m (--mode) [mode_option]` - Input mode indicates visual properties of given graph photo. Possible `[mode_option]`
+values:
+  * `grid_bg` - Hand drawn graph on grid/lined piece of paper (grid/lined notebook etc.),
+  * `clean_bg` - Hand drawn graph on empty uniform color background (on board, empty piece of paper, editor (paint)),
+  * `printed` - Graph from a printed source (e.g. from a paper, a publication, a book, etc.),
+  * `auto` - *default* - Mode is chosen automatically between grid_bg and clean_bg modes.
+- `-d (--debug) [debug_opiton]` - Debug mode indicates how much debugging information will be displayed. Possible 
+`[debug_opiton]` values:
+  * `no` - *default* - no windows with debugging information are displayed,
+  * `general` - only windows with general debugging information are displayed,
+  * `full` - all windows with debugging information are displayed.
+### Command line
+To run the script type in the command line:
+
+`python <path_to_main.py> -p <absolute_path_to_image> -m [mode] -d [debug]`
+
+Following example (when in OGR folder) results in 2 additional files and some general debugging windows (see picture below): 
+
+`python ./main.py -p "C:\Users\Filip\Downloads\graphs\graph_on_grid.jpg" -m grid_bg -d general`
+
+Running the OGR script correctly results in 2 files with different extensions, saved in the same directory 
+and with the same name as input image. Those file types are briefly described below (open links for full descriptions):
+- [graph6 (.g6)](http://users.cecs.anu.edu.au/~bdm/data/formats.html) - stores only logical information about vertices 
+and edges (equivalent to adjacency matrix),
+- [GraphML (.graphml)](https://docs.yworks.com/yfiles/doc/developers-guide/graphml.html) - in addition to logical 
+information stores visual and geometrical properties (vertex center coordinates, its color, etc.).
+
+![cmd_example](./readme_imgs/cmd_example.jpg)
+
+## Processing phases
+Image processing has been divided into 4 phases, that work in the simple pipeline:
+
+`Preprocessing -> Segmentation -> Topology Recognition -> Postprocessing`
+
+Before any processing takes place image is loaded. For the purpose of comparison with the results after each phase 
+example input image is given below.
+
+![input](./readme_imgs/input.jpg)  
+### Preprocessing
+The goal of this (first) phase is to prepare image for recognising vertices and edges and to do so image is:
+1. reshaped to a standard size (1280x800) and rotation (horizontal)
+2. converted to grayscale and binarized
+3. if `auto` mode has been selected, then mode is chosen automatically
+4. noise is filtered (eg. grid is removed)
+5. croped to remove unnecessary background around graph
+
+Image after preprocessing (below) is the input of the next (Segmentation) phase.
+
+![preprocessing](./readme_imgs/preprocessing.jpg)
+### Segmentation
+Second phase recognises vertices (divide pixels into vertices and edges ones) in following steps:
+1. Unfilled vertices are filled
+2. Edges are removed 
+3. Vertices are recognised 
+
+Segmentation results with a vertices list, which is the input (along binary image) for the Topology Recognition phase.
+Visualisation of a list (taken from `full` debug) is given below.
+
+![segmentation](./readme_imgs/segmentation.jpg)
+### Topology Recognition
+In this phase edges are recognised (connections between vertices) in following steps:
+1. Vertices pixels are removed from binary image
+2. Lines intersections are removed
+3. Line segments are approximated from contours to 2 endpoints
+4. Edges are created by linking line segments
+5. Connection between vertices are assigned to adjacency list
+
+Topology recognition updates each vertex object in a list with information about adjacent vertices. Updated list is the input 
+for Postprocessing phase (visualisation from `full` debug below).
+
+![topology_recognition](./readme_imgs/topology_recognition.jpg)
+### Postprocessing
+In the postprocessing phase 2 files, storing graph information, are created from adjacency list (description in 
+[Command line](#command-line) paragraph). GraphML file is created accordingly to 
+[yWorks documentation](https://docs.yworks.com/yfiles/doc/developers-guide/graphml.html) and graph6 file accordingly to 
+[official documentation](http://users.cecs.anu.edu.au/~bdm/data/formats.txt).
+
+## Future development
+There are definitely some features to improve, especially handling the intersections which could be improved by 
+storing removed intersection points coordinates, and later using these points to link segments into edges.
+File extension validation can also be implemented, as well as working with relative paths to input image. If you want
+to further develop this solution remember that we created it during apprenticeship at the [Gdansk University of 
+Technology](https://pg.edu.pl/en) and as far as I know this code is their intellectual property (so its best to contact 
+our [project coordinator](https://pg.edu.pl/7c4980df68_kacper.wereszko/wizytowka), at email address: 
+[email protected]).
+
+## References and Authors
+The idea for OGR, and some adapted solutions were taken from publication: 
+"[Optical Graph Recognition](https://link.springer.com/content/pdf/10.1007%2F978-3-642-36763-2_47.pdf)" written by 
+Christopher Auer, Christian Bachmaier, Franz J. Brandenburg, Andreas Gleißner, and Josef Reislhuber.
+
+Some ideas and solutions have been taken and adapted from 
+[yWorks article](https://www.yworks.com/blog/projects-optical-graph-recognition) about "Recognizing graphs from images".
+
+A lot of solutions and knowledge about opencv has been taken from 
+[offical opencv python tutorials](https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_tutorials.html).
+
+This solution has been mainly developed during summer 2020 one month apprenticeship at the [Gdansk University of 
+Technology (GUT)](https://pg.edu.pl/en) by:
+#### Filip Chodziutko and Kacper Nowakowski 2020

Vertex.py

@@ -0,0 +1,29 @@
+"""Module contains class representing vertex"""
+
+
+class Vertex:
+    """
+    Represent vertex recognised from the image.
+
+    Attributes:
+        x, y (int): coordinates of the center
+        r (float): radius
+        is_filled (bool): flag indicating if vertex is filled
+        color (int, int, int): bgr color
+        adjacency_list (list): list of adjacent (connected) vertices
+    """
+    id = -1
+    x = -1
+    y = -1
+    r = -1.0
+    is_filled = False
+    color = (-1, -1, -1)
+    adjacency_list = []
+
+    def __init__(self, x, y, r, is_filled, color):
+        self.x = x
+        self.y = y
+        self.r = r
+        self.is_filled = is_filled
+        self.color = color
+        self.adjacency_list = []

argsparser.py

@@ -0,0 +1,50 @@
+import os
+import argparse
+
+from shared import Mode, Debug
+
+# instance of parser for reading cli arguments
+parser = argparse.ArgumentParser("Optical graph recognition")
+
+parser.add_argument("-p", "--path", help="Absolute path to input image", required=True)
+parser.add_argument("-m", "--mode", help=Mode.HELP, choices=Mode.CHOICES, default=Mode.DEFAULT, type=str.lower)
+parser.add_argument("-d", "--debug", help=Debug.HELP, choices=Debug.CHOICES, default=Debug.DEFAULT, type=str.lower)
+
+
+def parse_argument(args) -> (int, str, str):
+    """
+    Parses the command line arguments
+
+    :param: args: Command line arguments
+    :return: mode, path to photo, path to save the result
+    """
+    save_path = parse_path(args.path)
+    mode = Mode.get_mode(args.mode)
+    debug = Debug.get_debug(args.debug)
+
+    return mode, debug, args.path, save_path
+
+
+def parse_path(file_path: str) -> str:
+    """
+    Checks the path to the photo and specifies the path to save
+
+    :param: file_path: path to photo
+    :return: path to save the result
+
+    """
+    file_path.replace(" ", "")
+    if file_path.count('.') != 1:
+        print("1: File path is incorrect. Must be only one dot.")
+        return ''
+    head, tail = os.path.split(file_path)
+    if len(tail) == 0:
+        print("1: File name no exist")
+        return ''
+
+    file_name, file_ext = os.path.splitext(tail)
+    if len(file_name) == 0:
+        print("1: File name not found")
+        return ''
+    save_path = head + '/' + file_name
+    return save_path

main.py

@@ -0,0 +1,50 @@
+"""Main module containing script entry - main function"""
+import cv2 as cv
+
+from shared import Debug
+from argsparser import parser, parse_argument
+from preprocessing import preprocess
+from segmentation import segment
+from topology_recognition import recognize_topology
+from postprocessing import postprocess
+
+
+def main():
+    args = parser.parse_args()
+    mode, debug, file_path, save_path = parse_argument(args)
+
+    if mode == -1 or debug == -1 or len(save_path) == 0:
+        print("1: Error reading input arguments!")
+        return -1
+
+    source = cv.imread(file_path)
+    if source is not None:  # read successful, process image
+
+        # 1st step - preprocessing
+        source, preprocessed, mode, is_rotated = preprocess(source, mode, debug)
+
+        # 2nd step - segmentation
+        vertices_list, visualised, preprocessed, edge_thickness = segment(source, preprocessed, mode, debug)
+        if len(vertices_list) == 0:
+            print("1: No vertices found")
+            return -1
+
+        # 3rd step - topology recognition
+        vertices_list = recognize_topology(vertices_list, preprocessed, visualised, edge_thickness, mode, debug)
+
+        # 4th step - postprocessing
+        postprocess(vertices_list, save_path, is_rotated)
+
+        # if displaying debug info has been enabled keep displayed windows open until key is pressed
+        if debug != Debug.NO:
+            cv.waitKey(0)
+
+        print("0")
+        return 0
+    else:
+        print("1: Error opening image!")
+        return -1
+
+
+if __name__ == "__main__":
+    main()