Checking-the-quality-of-revit-and-ifc-projects.ipynb

{"metadata":{"hide_input":false,"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.7.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"\nhttps://DataDrivenConstruction.io/index.php/checking-the-quality-of-revit-and-ifc-projects/","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Solution logic\nWith the help of a table describing the parameters in the category to be checked and with the help of Pipelines from DataDrivenConstruction – **we check the projects in Revit and IFC formats**. We automatically generate a PDF document, which contains the basic information about the parameters and categories to be checked.\n\n> In the construction industry, having accurate and high-quality building information models (BIM) is essential to the successful completion of a project. \n\n![](https://DataDrivenConstruction.io/wp-content/uploads/2023/02/Solution-Check-Data-Revit-IFC.gif)\n\n\n#### Read more about DataDrivenConstruction verification solution on the website:\n#### https://DataDrivenConstruction.io/index.php/checking-the-quality-of-revit-and-ifc-projects/","metadata":{}},{"cell_type":"code","source":"### installing the necessary Python libraries via a file requirements.txt\n### uncomment the line (remove the grid sign from the next line)\n# pip install -r requirements.txt\n\n### or install each library individually\n# pip install fpdf\n# pip install # pip install missingno\n# pip install pandas","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Required installation of a library for creating PDF documents\n!pip install fpdf","metadata":{"execution":{"iopub.status.busy":"2023-02-12T10:28:20.911879Z","iopub.execute_input":"2023-02-12T10:28:20.912262Z","iopub.status.idle":"2023-02-12T10:28:33.068672Z","shell.execute_reply.started":"2023-02-12T10:28:20.912233Z","shell.execute_reply":"2023-02-12T10:28:33.066992Z"},"trusted":true},"execution_count":2,"outputs":[{"name":"stdout","text":"Collecting fpdf\n  Downloading fpdf-1.7.2.tar.gz (39 kB)\n  Preparing metadata (setup.py) ... \u001b[?25ldone\n\u001b[?25hBuilding wheels for collected packages: fpdf\n  Building wheel for fpdf (setup.py) ... \u001b[?25ldone\n\u001b[?25h  Created wheel for fpdf: filename=fpdf-1.7.2-py2.py3-none-any.whl size=40725 sha256=2c4d3b9f9c0a3192833bf874e159fcf28fb328b40b378b54f1d0ea8a4d93ccd9\n  Stored in directory: /root/.cache/pip/wheels/d7/ca/c8/86467e7957bbbcbdf4cf4870fc7dc95e9a16404b2e3c3a98c3\nSuccessfully built fpdf\nInstalling collected packages: fpdf\nSuccessfully installed fpdf-1.7.2\n\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n\u001b[0mNote: you may need to restart the kernel to use updated packages.\n","output_type":"stream"}]},{"cell_type":"code","source":"#####################################################################################\n# DataDrivenConstruction\n# URI: https://DataDrivenConstruction.io/\n# If you have a feature enhancement request, request a price by info@DataDrivenConstruction.io\n\n# Pipeline:  Batch reporting for Revit and IFC projects\n# Description: Creating PDF documents by criteria from an Excel spreadsheet for Revit and IFC files\n# Input:  1. Tables with criteria, 2. project files in tabular form\n# Output: PDF report for each file \n\n# This program is free software: you can redistribute it and/or modify\n# it under the terms of the GNU General Public License as published by\n# the Free Software Foundation, either version 3 of the License, or\n# (at your option) any later version.\n#####################################################################################\n\nimport json, os, re, subprocess, warnings, pandas as pd\nfrom datetime import date\nfrom fpdf import FPDF\nimport matplotlib.pyplot as plt\nimport missingno as msno\nimport logging\nlogging.getLogger('matplotlib.font_manager').disabled = True\nwarnings.simplefilter(action='ignore', category=FutureWarning)\n\n\n############################   Parameters    ############################\n\n# Folders where conversion files are located\npath = r'/kaggle/input/check-the-projects-in-revit-and-ifc-formats/'\n\n# Folders where Table with Parameters are located\npathfold = r'/kaggle/input/table-of-parameters-pdf-content-sources/'\nnamemap = pathfold + 'ODB Table_of_Parameters_Revit _IFC_Check.xlsx'\n\n#########################################################################\n","metadata":{"ExecuteTime":{"end_time":"2023-02-08T12:49:41.562546Z","start_time":"2023-02-08T12:49:39.115618Z"},"pycharm":{"name":"#%%\n"},"scrolled":true,"execution":{"iopub.status.busy":"2023-02-12T10:32:53.832152Z","iopub.execute_input":"2023-02-12T10:32:53.832505Z","iopub.status.idle":"2023-02-12T10:32:53.839520Z","shell.execute_reply.started":"2023-02-12T10:32:53.832479Z","shell.execute_reply":"2023-02-12T10:32:53.838504Z"},"trusted":true},"execution_count":6,"outputs":[]},{"cell_type":"markdown","source":"# Prerequisites \n<span style=\"font-size: 14pt;\">Before you dive into running the application, you need to get the <span style=\"color: #333399;\"><strong>data from the project as a CSV or XLSX</strong></span> <code>Excel</code>&nbsp;table. To export Revit or IFC format into tables, you can use any solution you work with:<span style=\"font-size: 14pt;\"><span style=\"color: #333399;\"><strong>👨‍💻 Manual extraction of tables</strong><span style=\"color: #000000;\">:</span> <code>Revit</code> & <code>Dynamo</code>, <code>Revit</code> & <code>Schedule</code>, <code>Revit</code> & <code>ODBC</code>, <code>pyRevit</code>, <code>Forge</code>, <code>SimpleBIM</code>, <code>Desite</code>, <code>IfcOpenShell</code>, <code>IFCjs</code> <span style=\"color: #999999;\">and others</span></span><span style=\"font-size: 14pt;\"><span style=\"color: #333399;\"><strong> ⚙️ Automatic and batch table </strong> <span style=\"color: #000000;\">retrieval:</span></span> using <a href=\"https://DataDrivenConstruction.io/index.php/product/DataDrivenConstruction-converter-kit-lite-version/\"><code>noBIM Lite</code></a> converters <span style=\"color: #999999; font-size: 12pt;\">(<span class=\"css3_grid_vertical_align_table\"><span class=\"css3_grid_vertical_align\">Revit 2018, Revit 2019, Revit 2020, Revit 2021,&nbsp;</span></span></span><span class=\"css3_grid_vertical_align_table\"><span class=\"css3_grid_vertical_align\"><span style=\"color: #999999; font-size: 12pt;\">IFC2X3, IFC4X1, IFC4X, IFC4 - IFC4.3),</span> <a href=\"https://DataDrivenConstruction.io/index.php/product/DataDrivenConstruction-converter\"><code>noBIM Full</code></a> converters <span style=\"color: #999999; font-size: 12pt;\">(</span></span></span><span style=\"color: #999999; font-size: 12pt;\"><span class=\"css3_grid_vertical_align_table\"><span class=\"css3_grid_vertical_align\">Revit 2015, Revit 2016, Revit 2017, Revit 2018, Revit 2019, Revit 2020, Revit 2021, Revit 2022, Revit 2023</span></span><span class=\"css3_grid_vertical_align_table\"><span class=\"css3_grid_vertical_align\">,&nbsp;</span></span><span class=\"css3_grid_vertical_align_table\"><span class=\"css3_grid_vertical_align\">IFC2X3, IFC4X1, IFC4X, IFC4 - IFC4.3)</span></span></span></span> \n    \n![](https://DataDrivenConstruction.io/wp-content/uploads/2023/02/github.com-DataDrivenConstruction-5.gif) ","metadata":{}},{"cell_type":"markdown","source":"## The code below only needs to be used if you have DataDrivenConstruction converter: **noBIM Community, noBIM Lite, noBIM Full**<br>\n### noBIM Lite: https://DataDrivenConstruction.io/index.php/product/DataDrivenConstruction-converter-kit-lite-version/ \n### noBIM Full: https://DataDrivenConstruction.io/index.php/product/DataDrivenConstruction-converter/","metadata":{}},{"cell_type":"code","source":"#  The code below only needs to be used if you have DataDrivenConstruction converter: noBIM Community, noBIM Lite, noBIM Full\n\n\"\"\"\nimport os, subprocess\nimport time\n\n############################   Parameters    ############################\n\n# path to noBIM converter\npath_conv = r'C:\\DataDrivenConstruction\\ODBLite\\noBIM_Lite_v1_23-v2jfja\\\\'\n\n#########################################################################\n\n# output path\noutpath = path\n# Folders where the converter and conversion files are located\nconv_IfcToCsv = path_conv + 'IfcToCsv.exe'\nconv_IfcColladaExporter = path_conv + 'IfcColladaExporter.exe'\nconv_RvtToCsv = path_conv + 'RvtToCsv.exe'\nconv_RvtColladaExporter = path_conv + 'RvtColladaExporter.exe'\n\ndef convert(path, path_conv, outpath):\n    try: os.mkdir(outpath)\n    except: pass\n\n    # Conversion process from RVT and IFC in DAE\n    for file in os.listdir(path):\n        filepath = path + file\n        fileoutpath = outpath + file[:-3]\n        \n        # Conversion process from IFC in DAE and CSV\n        if file.endswith('.ifc'):\n            subprocess.Popen([conv_IfcToCsv, filepath, fileoutpath+'csv'], cwd = path_conv)\n            subprocess.Popen([conv_IfcColladaExporter, filepath, fileoutpath+'dae'], cwd = path_conv)\n            while not os.path.exists(fileoutpath+'dae') and not os.path.exists(fileoutpath+'csv'):\n                time.sleep(0.3)\n            print(\"Conversion Done: \" + file[:-3]+'csv' + ', ' + file[:-3]+'dae' )\n            \n        # Conversion process from Revit in DAE and CSV\n        if file.endswith('.rvt'):\n            subprocess.Popen([conv_RvtColladaExporter, filepath, fileoutpath+'dae'], cwd = path_conv)\n            while not os.path.exists(fileoutpath+'dae'):\n                time.sleep(0.3)\n            print(\"Conversion Done: \" + file[:-3]+'dae' )\n            while not os.path.getsize(fileoutpath+'dae') > os.path.getsize(filepath)*0.7:\n                time.sleep(0.3)\n            time.sleep(1)\n            subprocess.Popen([conv_RvtToCsv, filepath, fileoutpath+'dae', fileoutpath+'csv'], cwd = path_conv)\n            while not os.path.getsize(fileoutpath+'dae') > 1000:\n                time.sleep(0.3)\n            print(\"Conversion Done: \" + file[:-3]+'csv')         \n        time.sleep(7)\n    return\n\n# Run a function to search for any RVT and IFC files in a folder and then convert them\nconvert(path, path_conv, outpath)\n\"\"\"","metadata":{"ExecuteTime":{"end_time":"2023-02-08T12:51:17.123119Z","start_time":"2023-02-08T12:49:42.173365Z"},"scrolled":true,"execution":{"iopub.status.busy":"2023-02-12T10:32:57.925799Z","iopub.execute_input":"2023-02-12T10:32:57.926177Z","iopub.status.idle":"2023-02-12T10:32:57.935775Z","shell.execute_reply.started":"2023-02-12T10:32:57.926148Z","shell.execute_reply":"2023-02-12T10:32:57.934831Z"},"trusted":true},"execution_count":7,"outputs":[{"execution_count":7,"output_type":"execute_result","data":{"text/plain":"'\\nimport os, subprocess\\nimport time\\n\\n############################   Parameters    ############################\\n\\n# path to noBIM converter\\npath_conv = r\\'C:\\\\DataDrivenConstruction\\\\ODBLite\\noBIM_Lite_v1_23-v2jfja\\\\\\'\\n\\n#########################################################################\\n\\n# output path\\noutpath = path\\n# Folders where the converter and conversion files are located\\nconv_IfcToCsv = path_conv + \\'IfcToCsv.exe\\'\\nconv_IfcColladaExporter = path_conv + \\'IfcColladaExporter.exe\\'\\nconv_RvtToCsv = path_conv + \\'RvtToCsv.exe\\'\\nconv_RvtColladaExporter = path_conv + \\'RvtColladaExporter.exe\\'\\n\\ndef convert(path, path_conv, outpath):\\n    try: os.mkdir(outpath)\\n    except: pass\\n\\n    # Conversion process from RVT and IFC in DAE\\n    for file in os.listdir(path):\\n        filepath = path + file\\n        fileoutpath = outpath + file[:-3]\\n        \\n        # Conversion process from IFC in DAE and CSV\\n        if file.endswith(\\'.ifc\\'):\\n            subprocess.Popen([conv_IfcToCsv, filepath, fileoutpath+\\'csv\\'], cwd = path_conv)\\n            subprocess.Popen([conv_IfcColladaExporter, filepath, fileoutpath+\\'dae\\'], cwd = path_conv)\\n            while not os.path.exists(fileoutpath+\\'dae\\') and not os.path.exists(fileoutpath+\\'csv\\'):\\n                time.sleep(0.3)\\n            print(\"Conversion Done: \" + file[:-3]+\\'csv\\' + \\', \\' + file[:-3]+\\'dae\\' )\\n            \\n        # Conversion process from Revit in DAE and CSV\\n        if file.endswith(\\'.rvt\\'):\\n            subprocess.Popen([conv_RvtColladaExporter, filepath, fileoutpath+\\'dae\\'], cwd = path_conv)\\n            while not os.path.exists(fileoutpath+\\'dae\\'):\\n                time.sleep(0.3)\\n            print(\"Conversion Done: \" + file[:-3]+\\'dae\\' )\\n            while not os.path.getsize(fileoutpath+\\'dae\\') > os.path.getsize(filepath)*0.7:\\n                time.sleep(0.3)\\n            time.sleep(1)\\n            subprocess.Popen([conv_RvtToCsv, filepath, fileoutpath+\\'dae\\', fileoutpath+\\'csv\\'], cwd = path_conv)\\n            while not os.path.getsize(fileoutpath+\\'dae\\') > 1000:\\n                time.sleep(0.3)\\n            print(\"Conversion Done: \" + file[:-3]+\\'csv\\')         \\n        time.sleep(7)\\n    return\\n\\n# Run a function to search for any RVT and IFC files in a folder and then convert them\\nconvert(path, path_conv, outpath)\\n'"},"metadata":{}}]},{"cell_type":"markdown","source":"# Excel with checking rules\n<span style=\"font-size: 14pt;\">Any project elements in <code>Revit </code>and <code>IFC </code>format belong to some group (<code>OST_ </code>in <code>Revit</code>, <code>IfcEntity </code>in <code>IFC</code>) or type group (<code>TypeName </code>in <code>Revit</code>, <code>ObjectType </code>in <code>IFC</code>). Also, <span style=\"color: #333399;\"><strong>each element has its own unique set of properties and parmeters</strong> </span>(which are columns in the overall project table when the <code>noBIM </code>concept is applied).</span>\n\n<span style=\"font-size: 14pt;\">Using the check table, <span style=\"color: #333399;\">we specify in the first column of the table, </span><span style=\"color: #000000;\">for the elements to be checked and tested</span>, <span style=\"color: #333399;\">first the category</span> (<code>OST_ </code>in <code>Revit</code>, <code>IfcEntity </code>in <code>IFC</code>) or type (<code>TypeName</code>in <code>Revit</code>, <code>ObjectType </code>in <code>IFC</code>) and in the <span style=\"color: #333399;\"><strong>second column we specify the parameters</strong> </span>that we want to check for this group of elements.</span>\n![](https://DataDrivenConstruction.io/wp-content/uploads/2023/02/Excel-Check-Revit-IFC-Project.gif)\n\n<span style=\"font-size: 14pt;\">For each parameter from the check table in the project will be checked:</span>\n<span style=\"font-size: 14pt;\"><span style=\"color: #333399;\">\n<ul style=\"list-style-type: square;\">\n \t<li><span style=\"font-size: 14pt;\"><span style=\"color: #333399;\"><strong>presence of the parameter</strong> </span>for the group of elements</span></li>\n \t<li><span style=\"font-size: 14pt;\"><span style=\"color: #333399;\"><strong>percentage of the content of the values</strong></span> in the parameter</span></li>\n \t<li><span style=\"font-size: 14pt;\">the <span style=\"color: #333399;\"><strong>unique values</strong></span> of this parameter</span></li>\n</ul>","metadata":{}},{"cell_type":"code","source":"# Where files and photos of project data are created\npathr = pathfold + 'PDF_Content_Sources/'\npathro = path\n#pathro = path + r'Geometry\\\\'\n\n# Variables used in the code\n# For Revit - Category, for IFC - IfcEntity\nparam_group = 'Category'\n\n# Which files will be checked\ncsv_files = []\nfor file in os.listdir(path):\n    if file.endswith('.csv'):\n        csv_files.append(file)\n\ncsv_files","metadata":{"ExecuteTime":{"end_time":"2023-02-08T12:51:17.153687Z","start_time":"2023-02-08T12:51:17.128114Z"},"execution":{"iopub.status.busy":"2023-02-12T10:53:06.440596Z","iopub.execute_input":"2023-02-12T10:53:06.440951Z","iopub.status.idle":"2023-02-12T10:53:06.449449Z","shell.execute_reply.started":"2023-02-12T10:53:06.440925Z","shell.execute_reply":"2023-02-12T10:53:06.448138Z"},"trusted":true},"execution_count":19,"outputs":[{"execution_count":19,"output_type":"execute_result","data":{"text/plain":"['rac_advanced_sample_project_rvt.csv',\n 'rac_basic_sample_project_rvt.csv',\n 'rme_basic_sample_project_rvt.csv',\n 'Technical_school-current_m_rvt.csv']"},"metadata":{}}]},{"cell_type":"code","source":"# Classes used to format a PDF document\nclass PDF(FPDF):\n    def header(self):\n        # Arial bold 15\n        self.set_font('Arial', 'B', 12)\n        # Move to the right\n        # Title\n        self.set_text_color(56, 81, 153)\n        self.cell(1, 0, '', 50, 100, 'L')\n        self.cell(1, 6, 'BIM PARAMETER CHECK', 50, 1, 'L',  link = '')\n        self.cell(1, 6, projectf[:-4], 50, 200, 'L')\n        # Logo\n        self.image(pathr + 'logo_odb.png', 25, 3, 175, )\n        # Line break\n        self.ln(6)\n        self.set_text_color(0, 0, 0)\n\n    # Page footer\n    def footer(self):\n        #if self.page_no() != 1:\n        # Position at 1.5 cm from bottom\n        self.set_y(-20)\n        # Arial italic 8\n        self.set_font('Arial', '', 8)\n        # Page number\n        self.image(pathr + 'footer_odb.png', x = 24, y = None, w = 170, h = 0, type = '', link = 'https://DataDrivenConstruction.io/')\n        self.set_text_color(56, 81, 153)\n        self.cell(170, 4, d2 + '  ' + 'Page  ' + str(self.page_no()) + '/{nb}', 0, 0, 'R')","metadata":{"ExecuteTime":{"end_time":"2023-02-08T12:51:17.169461Z","start_time":"2023-02-08T12:51:17.156460Z"},"pycharm":{"name":"#%%\n"},"scrolled":true,"execution":{"iopub.status.busy":"2023-02-12T10:53:07.697549Z","iopub.execute_input":"2023-02-12T10:53:07.697919Z","iopub.status.idle":"2023-02-12T10:53:07.707090Z","shell.execute_reply.started":"2023-02-12T10:53:07.697890Z","shell.execute_reply":"2023-02-12T10:53:07.705855Z"},"trusted":true},"execution_count":20,"outputs":[]},{"cell_type":"markdown","source":"# Run the code\n <span>With these steps, you should be <span style=\"color: #333399;\"><strong>able to run Python code on a new computer without any issues</strong></span>.</span></br>\n \nAfter the script runs, check the finished reports on the right side in the \"Output\" section</br>\n![](https://DataDrivenConstruction.io/wp-content/uploads/2023/02/github.com-DataDrivenConstruction-6.gif)","metadata":{}},{"cell_type":"code","source":"# Read parameter table Excel\ndf_param =  pd.read_excel(namemap, header = 1)   \nlistpar = list(zip(df_param['REVIT category'], df_param['Parameters to check']))\n\ntoday = date.today()\nd2 = today.strftime(\"%d.%m.%Y\")\n\n# Go through all the files in the folder\nfor file in csv_files:  \n    \n    # Creating a dictionary with all the criteria\n    dictl = {} \n    for el in listpar:\n        dictl.setdefault(el[0], []).append(el[1])\n    \n    pdf = PDF()\n    pdf.alias_nb_pages()\n\n    # Page indentation\n    pdf.set_left_margin(25)\n    pdf.set_right_margin(15)\n    \n    # Project name in the document header\n    projectf = file \n    strnr = 1 \n    countab = 1\n    \n    # Reading a project file\n    df = pd.read_csv(path + file)\n    pathf = '../' + file[:-4] + '_IMG'\n    \n    # Creating folders\n    try: os.mkdir(pathro)\n    except: pass\n    try: os.mkdir(pathf)\n    except: pass   \n\n    file_dae = path + file.replace('.csv', '.dae')\n    try: ImgCapture.load(file_dae)\n    except: print(file_dae)\n    \n    # Parameter\n    ent_value = df[param_group].unique()\n    dict_wrong = {}\n    klnummer = 1\n    \n    # All categories of the project\n    ent_value = [item for item in ent_value if not(pd.isnull(item)) == True]\n    \n    # Going through all categories of the project       \n    for elkl in ent_value:\n        if elkl in dictl.keys():    \n            \n            # Creating a data frame for a specific group\n            klasse_dfmatch, klasse_dfmatchw = [], []\n            dfparam = df.loc[df[param_group] == elkl]\n            dfparam = dfparam.dropna(axis=1, how='all')\n            klasse_dfmatch = dfparam.columns\n            df_klasse = df.loc[:, klasse_dfmatch]\n            df_klasse = df_klasse.loc[df_klasse[param_group] == elkl]\n            df_klasse = df_klasse.dropna(how='all', axis=0)\n\n            # Receiving data that does not meet the criteria\n            n_inproject = []    \n            odb_param = []\n            if elkl in dictl.keys():\n                parameter_tab = dictl[elkl]\n                param_len = len(parameter_tab)\n                for el in df_klasse.columns[:45].to_list():\n                    if el in parameter_tab:\n                        odb_param.append(el)\n                    else:\n                        n_inproject.append(el)\n            \n            # Obtaining the table of parameters distributed by the number of values\n            nval = [df_klasse[col_name].count() for col_name in df_klasse.columns]\n            dict_nv = {}\n            dict_nv = dict(zip(df_klasse.columns, nval))\n            dict_nv = sorted(dict_nv.items(), key=lambda x: x[1], reverse=True)\n            \n            # Parameters that are not in the table to check\n            dict_nvn = []\n            for el in dict_nv:\n                dict_nvn.append(el[0])\n            for el in odb_param:\n                dict_nvn.remove(el)\n            \n            for el in odb_param:\n                dict_nvn.insert(0, el)\n            df_klasse = df_klasse[dict_nvn]\n            \n            # Parameters found in XLSX table to check\n            parameter_s23 = parameter_tab \n            for el23 in odb_param:\n                parameter_s23.remove(el23)\n            \n            # Defining colors for parameters that are found and parameters that were not found\n            colorsd = []\n            for el in odb_param:\n                colorsd.append('#43CD62')\n            for el in range(len(dict_nvn)):\n                colorsd.append('grey')\n            # Creating a picture PNG for all parameters\n            namefig = msno.bar(df_klasse[df_klasse.columns[:45]], figsize=(25,4), fontsize=20, color=colorsd)\n            fig_copy = namefig.get_figure()\n            elklclean = re.sub('[^A-Za-z0-9]+', '', elkl)\n            \n            # Save the picture in the folder\n            fig_copy.savefig(pathf + '\\\\' + elklclean + '_attributesklasse.png', bbox_inches = 'tight')\n            namefig.clear()\n            plt.clf()\n\n            df_klasse = df_klasse.set_index('ID')\n            df_klasse.index = df_klasse.index.astype(str)\n            nodes = list(df_klasse.index)\n            \n            # Forming geometry for elements in the group at the top of the document\n            level_img = pathro + file[:30] + elklclean + '_geometry_image.png'\n            #img = ImgCapture.shoot(level_img, nodes)\n\n            \n            #bookm[klne + ' ' + param_group + ': '+ elkl]=pdf.page_no()\n            \n            ################################################################\n            ###################### PDF document ############################\n            ################################################################\n            \n            # Creating a new page of a PDF document\n            pdf.add_page()\n            klnummer += 1\n            \n            ################################################################\n            # If the group photo is generated\n            try: pdf.image(level_img, x = 55, y = 32, w = 65, type = '')\n            except: pass\n            pdf.set_font('Arial', 'B', 10)\n            pdf.set_text_color(0,102,204)\n            pdf.set_font('','U')\n            pdf.set_font('Arial', '', 10)\n            pdf.set_text_color(0, 0, 0)\n                \n\n            ################################################################\n\n            # Group name at the top of the document\n            klne = str(strnr) + '.' + str(klnummer)    \n            pdf.set_font('Arial', 'B', 12)\n            pdf.set_fill_color(241, 241, 242)\n            pdf.cell(len(param_group) + 24, 7, ' ' + klne + ' ' + param_group + ': ', 0, 0, 'L', fill = True)\n            pdf.cell(3, 7, ' ', 0, 0, 'L')\n            pdf.set_fill_color(210, 255, 224)\n            pdf.cell(len(elkl) + 20, 7, ' ' + elkl, 0, 1, 'C', fill = True)\n            pdf.ln(57)\n            \n             ################################################################\n                \n            pdf.set_font('Arial', '', 10)\n            # Check the number of parameters found, if more than 50% \n            if len(odb_param)/param_len > 0.5:\n                pdf.image(pathr + 'v.png', x = 125, y = 57, w = 75)\n            else:\n                pdf.image(pathr + 'w.png', x = 130, y = 58, w = 70)\n            \n            ################################################################\n            \n            # Forming a table with the statistics of the parameters checked and as a whole\n            \n            pdf.set_font('Arial', '', 8)\n            pdf.cell(32, 7, 'Checking parameters by criteria from an Excel table:', 0, 1, 'L')\n           \n            pdf.set_font('Arial', 'B', 10)\n            pdf.set_fill_color(225, 225, 225)\n            pdf.cell(32, 7, 'Elements', 0, 0, 'C', fill = True)\n            pdf.cell(3, 7, '', 0, 0, 'c') \n            pdf.cell(32, 7,  'Parameters', 0, 0, 'C', fill = True) \n            pdf.cell(3, 7, '', 0, 0, 'c') \n            pdf.cell(32, 7, 'Checked', 0, 0, 'C', fill = True) \n            pdf.cell(3, 7, '', 0, 0, 'c') \n            pdf.set_fill_color(165, 250, 184)\n            pdf.cell(32, 7,  'Project has', 0, 0, 'C', fill = True) \n            if len(parameter_s23) == 0:\n                pdf.set_fill_color(165, 255, 200)\n            else:\n                pdf.set_fill_color(250, 180, 165)\n            strparam = str(parameter_s23)\n            pdf.cell(3, 7, '', 0, 0, 'c') \n            strparam = strparam.replace(\"[\", \"\")\n            strparam = strparam.replace(\"]\", \"\")\n            pdf.cell(32, 7, 'Missing', 0, 0, 'C', fill = True) \n            pdf.set_font('Arial', '', 11)\n            pdf.cell(20, 8, '', 2, 1, 'L')\n\n            ##############\n            \n            pdf.set_font('Arial', '', 10)\n            pdf.set_fill_color(241, 241, 242)\n            pdf.cell(32, 7, str(len(df_klasse)), 0, 0, 'C', fill = True)\n            pdf.cell(3, 7, '', 0, 0, 'c') \n            pdf.cell(32, 7, str(len(df_klasse.columns)), 0, 0, 'C', fill = True) \n            pdf.cell(3, 7, '', 0, 0, 'c') \n            pdf.cell(32, 7, str(param_len), 0, 0, 'C', fill = True) \n            pdf.cell(3, 7, '', 0, 0, 'c') \n            pdf.set_fill_color(210, 255, 224)\n            pdf.cell(32, 7,  str(len(odb_param)), 0, 0, 'C', fill = True) \n            if len(parameter_s23) == 0:\n                pdf.set_fill_color(165, 255, 200)\n            else:\n                pdf.set_fill_color(255, 218, 210)\n            pdf.cell(3, 7, '', 0, 0, 'c') \n            pdf.cell(32, 7, str(len(parameter_s23)), 0, 0, 'C', fill = True) \n            pdf.set_font('Arial', '', 11)\n            pdf.cell(20, 10, '', 2, 1, 'L')\n            \n            ################################################################\n            \n            # Parameters not found in the project \n            if len(parameter_s23) > 0:\n                pdf.cell(150, 9, 'Parameters not found in the project or completely empty values:', 0, 1, 'L')\n                pdf.set_font('Arial', '', 10)\n                for el in parameter_s23:\n                    pdf.set_fill_color(255, 218, 210)\n                    size = len(el)+20\n                    pdf.cell(size, 6, el, 0, 0, 'C', fill = True) \n                    pdf.cell(3, 7, '', 0, 0, 'C', ) \n            else:\n                pdf.cell(150, 12, '', 0, 1, 'L')\n\n            ################################################################\n    \n            # Forming a table with statistics on the volume of groups\n            pdf.ln(4)\n            pdf.set_font('Arial', '', 12)    \n            try:\n                df_klasse['Volume'] = df_klasse['Volume'].astype(str).str.extract(r'([-+]?\\d*\\.?\\d+)').astype('float')\n                total_vol = int(df_klasse['Volume'].sum())\n            except:\n                total_vol = '0'\n                pass\n            try:\n                df_klasse['Area'] = df_klasse['Area'].astype(str).str.extract(r'([-+]?\\d*\\.?\\d+)').astype('float')\n                total_area = int(df_klasse['Area'].sum()) \n            except:\n                total_area = '0'\n                pass\n            try:\n                df_klasse['Length'] = df_klasse['Length'].astype(str).str.extract(r'([-+]?\\d*\\.?\\d+)').astype('float')\n                total_len = int(df_klasse['Length'].sum()) \n            except:\n                total_len = '0'\n                pass\n            pdf.cell(20, 3, '', 2, 1, 'L')\n            pdf.set_font('Arial', '', 8)\n            pdf.cell(150, 7, 'Volumetric parameters for all elements in the groups:', 0, 1, 'С')\n            \n            ########\n            \n            pdf.set_font('Arial', 'B', 10)\n            pdf.set_fill_color(225, 225, 225)\n            pdf.cell(55, 7, 'Sum of volumes', 0, 0, 'C', fill = True)\n            pdf.cell(3, 7, '', 0, 0, 'c') \n            pdf.cell(55, 7,  'Sum of areas', 0, 0, 'C', fill = True) \n            pdf.cell(3, 7, '', 0, 0, 'c') \n            pdf.cell(55, 7, 'Sum of lengths', 0, 0, 'C', fill = True) \n            pdf.cell(3, 7, '', 0, 1, 'c') \n            \n            ########\n            \n            pdf.cell(20, 2, '', 2, 1, 'L')\n            pdf.set_font('Arial', '', 10)\n            pdf.set_fill_color(241, 241, 242)\n            pdf.cell(55, 7, str(total_vol) + ' m³', 0, 0, 'C', fill = True)\n            pdf.cell(3, 7, '', 0, 0, 'c') \n            pdf.cell(55, 7,  str(total_area) + ' m²', 0, 0, 'C', fill = True) \n            pdf.cell(3, 7, '', 0, 0, 'c') \n            pdf.cell(55, 7, str(total_len) + ' m', 0, 0, 'C', fill = True) \n            pdf.cell(3, 7, '', 0, 1, 'c') \n            \n            ################################################################\n\n            # Inserting a picture with the parameters that were checked above\n            pdf.set_font('Arial', '', 11)\n            pdf.cell(20, 5, '', 2, 1, 'L')\n            pdf.set_font('Arial', '', 11)\n            text1 = 'The following diagram shows which parameters in the Project ' + file[:-9] + ' are included and which parameters and how often for ' + param_group + ' ' + elkl +  ' are given:'\n            pdf.multi_cell(w = 160, h =6, txt = text1, border = 0)\n            pdf.cell(20, 1, '', 2, 1, 'L')\n            pdf.set_font('Arial', 'B', 12)\n            pdf.image(pathr + 'logo_atribbuten.png', x = 30, w = 165, type = '', link = '')\n            pdf.image(pathf+ '\\\\' + elklclean+'_attributesklasse.png', x = 25,  h = 50, w = 165, type = '', link = '')\n\n            ################################################################\n            \n            # Formation of the bottom table of statistics for the found parameters\n            # Calculate the number of non-null values for each column in the dataframe using the \n            non_null_counts = dfparam.count()\n            # Calculate the total number of rows in the dataframe using \"len()\", and store the result in \"total_counts\"\n            total_counts = len(dfparam)\n            percent_filled = non_null_counts / total_counts * 100\n            # Reset the index of the dataframe \"filled_df\"\n            filled_df = pd.DataFrame(percent_filled).reset_index()\n            filled_df.columns = ['column_name', 'Percentage filling']\n            # Convert the \"Percentage filling\" column to integer type\n            filled_df['Percentage filling'] = filled_df['Percentage filling'].astype(int)\n\n            for col in dfparam.columns:\n                unique_values = dfparam[col].unique()[:5]\n                filled_df.loc[filled_df['column_name'] == col, 'Example values'] = \", \".join(map(str, unique_values))\n\n            filled_df['Example values'] = filled_df['Example values'].str.replace('[|]|nan ,|nan,|nan', '')\n            filled_df['Example values'] = filled_df['Example values'].str[:50]\n            filled_df = filled_df.set_index('column_name')\n            \n            filled_df2 = filled_df.loc[odb_param]\n            pdf.ln(4)\n\n            ###############\n            if len(filled_df2) > 0:\n                pdf.set_font('Arial', '', 8)\n                pdf.cell(150, 7, 'Statistics on the parameters found:', 0, 1, 'С')\n                pdf.set_font('Arial', 'B', 10)\n                pdf.set_fill_color(165, 250, 184)\n                pdf.cell(55, 6, txt='  Parameter Name', ln=0, align=\"L\", fill = True)\n                pdf.cell(3, 6, '', ln=0, align=\"C\")\n                pdf.set_fill_color(225, 225, 225)\n                pdf.cell(35, 6, txt='Percentage filling', ln=0, align=\"C\", fill = True)\n                pdf.cell(3, 6, '', ln=0, align=\"C\")\n                pdf.cell(70, 6, txt='  Example values', ln=0, align=\"L\", fill = True)\n                pdf.cell(3, 7, '', ln=1, align=\"C\")\n\n                for i in range(len(filled_df2)):\n                    pdf.set_font('Arial', '', 10)\n                    pdf.set_fill_color(210, 255, 224)\n                    pdf.cell(55, 6, txt='  ' + str(filled_df2.index[i][:38]), ln=0, align=\"L\", fill = True)\n                    if filled_df2.iloc[i, 0] < 50:\n                        pdf.set_fill_color(255, 218, 210)\n                    else:\n                        pdf.set_fill_color(210, 255, 224)\n                    pdf.cell(3, 6, '', ln=0, align=\"C\")\n                    pdf.cell(35, 6, txt=str(filled_df2.iloc[i, 0])+'%', ln=0, align=\"C\", fill = True)\n                    pdf.set_fill_color(241, 241, 242)\n                    pdf.cell(3, 6, '', ln=0, align=\"C\")\n                    pdf.cell(70, 6, txt='  ' + str(filled_df2.iloc[i, 1][:38]), ln=0, align=\"L\", fill = True)\n                    pdf.cell(3, 7, '', ln=1, align=\"C\") \n                    \n    strnr +=1\n    pdf.output(file[:-4] + 'ODB_Report.pdf', 'F')\n    print('Report generated: ' + file[:-4] + 'ODB_Report.pdf')\n    # After the script runs, check the finished reports on the right side in the \"Output\" section","metadata":{"ExecuteTime":{"end_time":"2023-02-08T12:51:49.130806Z","start_time":"2023-02-08T12:51:17.173194Z"},"pycharm":{"name":"#%%\n"},"scrolled":true,"execution":{"iopub.status.busy":"2023-02-12T10:54:37.327538Z","iopub.execute_input":"2023-02-12T10:54:37.327911Z","iopub.status.idle":"2023-02-12T10:55:12.230988Z","shell.execute_reply.started":"2023-02-12T10:54:37.327880Z","shell.execute_reply":"2023-02-12T10:55:12.230151Z"},"trusted":true},"execution_count":23,"outputs":[{"name":"stdout","text":"/kaggle/input/check-the-projects-in-revit-and-ifc-formats/rac_advanced_sample_project_rvt.dae\nReport generated: rac_advanced_sample_project_rvtODB_Report.pdf\n/kaggle/input/check-the-projects-in-revit-and-ifc-formats/rac_basic_sample_project_rvt.dae\nReport generated: rac_basic_sample_project_rvtODB_Report.pdf\n","output_type":"stream"},{"name":"stderr","text":"/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py:3552: DtypeWarning: Columns (8,10,13,14,15,17,20,27,28,29,30,31,33,35,36,38,41,48,53,54,56,57,58,59,62,72,73,74,75,80,81,82,83,84,85,86,87,89,90,92,93,95,96,97,98,109,110,114,115,116,117,118,123,125,126,127,128,129,130,132,133,137,138,144,145,150,151,152,173,174,182,183,189,190,191,193,194,195,196,197,198,199,200,202,203,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,276,284,285,286,287,288,289,290,292,295,296,297,298,299,300,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,318,321,322,323,327,328,329,330,331,332,333,335,338,339,340,341,343,344,345,347,351,352,354,355,356,357,359,361,362,363,364,365,367,368,369,370,371,372,373,374,375,376,378,379,380,381,383,388,391,393,394,395,396,399,400,401,402,403,404,405,410,416,429,430,431,433,434,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,468,473,474,486,487,492,493,494,495,496,497,498,499,500,502,505,506,507,509,510,513,514,515,516,517,518,519,520,521,522,523,524,525,526,527,528,529,530,531,532,533,534,535,536,537,538,539,540,541,542,543,544,545,546,547) have mixed types.Specify dtype option on import or set low_memory=False.\n  exec(code_obj, self.user_global_ns, self.user_ns)\n","output_type":"stream"},{"name":"stdout","text":"/kaggle/input/check-the-projects-in-revit-and-ifc-formats/rme_basic_sample_project_rvt.dae\nReport generated: rme_basic_sample_project_rvtODB_Report.pdf\n/kaggle/input/check-the-projects-in-revit-and-ifc-formats/Technical_school-current_m_rvt.dae\n","output_type":"stream"},{"name":"stderr","text":"/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py:3552: DtypeWarning: Columns (8,9,11,12,13,14,75,80,81,82,83,84,85,86,93,94,95,96,99,100,101,103,105,124,127,128,130,133,134,137,140,141,142,143,144,150,151,155,156,157,162,165,166,167,168,169,172,173,175,178,180,181,182,183,184,185,186,187,188,189,190) have mixed types.Specify dtype option on import or set low_memory=False.\n  exec(code_obj, self.user_global_ns, self.user_ns)\n","output_type":"stream"},{"name":"stdout","text":"Report generated: Technical_school-current_m_rvtODB_Report.pdf\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"<Figure size 1800x288 with 0 Axes>"},"metadata":{}}]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}