Abstract

Facilities should be kept in good condition throughout their lifecycle by rigorous inspection processes. The semantic relationships between multiple inspection information during lifecycle phases, and inspection result representation are among the most critical issues that need to be addressed. So far, many studies have been done to identify, analyze, repair, and prevent defects. However, after capturing the defect information, there is a need for an ontology to organize and integrate relevant information and future actions. Additionally, the availability of inspection robots in buildings’ construction and operation phases has led to expanding the scope of applications and increasing technological challenges. BIM models comprise useful information about the building environment’s representation, which can help the inspection robot overcome task complexity. However, the research in this area is still limited and fragmented, and there is a need to develop an integrated ontology to be used as a knowledge model for logic-based inspection of building defects.
Moreover, visual inspection using non-equipped eyes is the principal method of detecting structural surface defects, which is unsafe, time-consuming, expensive, and subjective to human errors. Using remote sensing, such as, cameras and LiDAR scanners, is one solution to overcome these shortcomings. The captured point cloud data from the real environment can assist in detecting the defects and taking further actions. Recently, machine learning methods attracted the attention of researchers for semantic segmentation and classification based on point clouds. However, no deep learning method is currently available for semantic segmentation of concrete surface defects based on raw point cloud data. Furthermore, the BIM model needs to be integrated with the results of defect semantic segmentation after the LiDAR-based inspection.
Addressing the above issues, this research has the following objectives: (1) Developing an ontology for concrete surface defects; (2) Developing BIM-based ontology to cover the different types of information and concepts related to robotic navigation and inspection tasks; (3) Developing a method for point cloud-based concrete surface defects semantic segmentation; and (4) Developing a semi-automated process for as-inspected modeling.
The first part of this research focused on the development of an ontology, called Ontology for Concrete Surface Defects (OCSD), to have a unified knowledge model where all the stakeholders can access information in a systematic manner. OCSD metrics include 333 classes, 51 relations, 27 attributes, and 31 individuals. OCSD comprises high-level knowledge of the concepts and relationships related to surface defects, inspection, diagnosis, and 3R (Repair, Rehabilitation, and Replacement) processes. The application of OCSD was investigated in a case study and a survey was designed to evaluate the semantic representation of OCSD. Based on the evaluation, OCSD was able to provide a clear understanding of the concepts and relationships in the domain, and it can help future asset management systems benefit from the provided knowledge.
The second part of this research focused on the development of an integrated ontology, called Ontology for BIM-based Robotic Navigation and Inspection Tasks (OBRNIT), to extend BIM applications for robotic navigation and inspection tasks. OBRNIT metrics include 386 classes, 45 relations, 52 attributes, and 8 individuals. OBRNIT comprises high-level knowledge of the concepts and relationships related to buildings, robots, and navigation and inspection tasks. BIM is considered as a reference that is integrated with the knowledge model. The semantic representation of OBRNIT was evaluated through a case study and a survey. The evaluation demonstrates that OBRNIT covers the domain’s concepts and relationships up to the point that satisfies the domain experts. Based on the evaluation, OBRNIT was able to give a clear understanding of the concepts and relationships in the domain, and it can help in the future in developing robotic inspection systems.
The last part of this research focused on a method for point cloud-based defect semantic segmentation based on Normal Vector Enhanced Dynamic Graph Convolutional Neural Network (NVE-DGCNN) to automate the inspection process of concrete surface defects, including cracks and spalls. This part investigates two main characteristics related to surface defects, including the normal vector and depth. The network’s performance is improved by modifying the network and augmenting the dataset. Sensitivity analysis is applied to capture the best combination of hyperparameters and investigate their effects on the network performance. NVE-DGCNN resulted in 98.56% and 96.50% recalls for semantic segmentation of cracks and spalls, respectively. Furthermore, post-processing of the results of the defects semantic segmentation is done to semi-automate the process of as-inspected modeling. This semi-automated process made it possible to manage and visualize the detected defects by extracting their dimensions and identifying the conditions on the 3D model.