Fig 5 - available via license: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International
Content may be subject to copyright.
Source publication
Despite advanced construction technologies that are unceasingly filling the city-skylines with glassy high-rise structures, maintenance of these shining tall monsters has remained a high-risk labor-intensive process. Thus, nowadays, utilizing façade-cleaning robots seems inevitable. However, in case of navigating on cracked glass, these robots may...
Context in source publication
Context 1
... MCU (Arduino Mega) and ROS slave (Intel Compute Stick), on board. As well as, between ROS slave and ROS master, communicated by WiFi. ROS master is a server PC where the information from the ROS slave is processed in software which cannot be processed in the ROS slave given the processing resources necessary for the correct operation. As shown in Fig. 5, The communication between TensorFlow and ROS is developed by subscribing to data package topics [40] for identifying the cracked glass by image ...
Similar publications
In this work we present a new framework for neural networks compression with fine-tuning, which we called Neural Network Compression Framework (NNCF). It leverages recent advances of various network compression methods and implements some of them, such as sparsity, quantization, and binarization. These methods allow getting more hardware-friendly m...
We have developed a real-time system for expressing emotion as a portrait selected according to the facial expression while writing a message. The portrait is decided by a hair style, a facial outline, and a cartoon of facial expression. The image signal is analyzed by the system using image processing software (OpenCV). The system selects one port...
The algorithms that extract keypoints
and descriptors in augmented reality applications are getting more and more
important in terms of performance.
Criterions like time and correct matching of points gain more impact according
to the type of application. In this paper, the performance of the algorithms
used to identify an image using keypoint and...
The great development of robot vision represented by deep learning places urgent demands on embedded vision implementation. This article introduces a hardware framework for implementation of embedded vision based on digital signal processor, which can be widely used in robot vision applications. Firstly, the article discusses implementation of a pr...
Nowadays, crime incidents like stealing, fighting and harassment often occur in campus leading to serious consequences. Students do not feel secure to study in campus anymore. Thus, a simple facial emotion detection system using a Raspberry Pi is introduced to help mitigating the issue before getting worse in campus. Two algorithms are used for thi...
Citations
... Automatic glass crack detection for façade-cleaning robot [118] AI Integration of service robots in the smart home by means of UPnP: A surveillance robot case study 2013 Implementing a basic garbage detection routine using built-in camera that allows the smart home system to instruct a service robot to clean whenever garbage is detected. ...
The construction industry faces many challenges, including schedule and cost overruns, productivity constraints, and workforce shortages. Compared to other sectors, it lags in digitalization in every project phase. Artificial Intelligence (AI) and Machine Learning (ML) have emerged as transformative technologies revolutionizing the construction sector. However, a discernible gap persists in systematically categorizing the applications of these technologies throughout the various phases of the construction project life cycle. In response to this gap, this research aims to present a thorough assessment of the deployment of AI and ML across diverse phases in construction projects, with the ultimate goal of furnishing valuable insights for the effective integration of these intelligent systems within the construction sector. A thorough literature review was performed to identify AI and ML applications in the building sector. After scrutinizing the literature, the applications of AI and ML were presented based on a construction project life cycle. A critical review of existing literature on AI and ML applications in the building industry showed that AI and ML applications are more frequent in the planning and construction stages. Moreover, the opportunities for AI and ML applications in other stages were discussed based on the life cycle categorization and presented in this study. The practical contribution of the study lies in providing valuable insights for the effective integration of intelligent systems within the construction sector. Academically, the research contributes by conducting a thorough literature review, categorizing AI and ML applications based on the construction project life cycle, and identifying opportunities for their deployment in different stages.
... However, the crack detection algorithm was not processed onboard but on a separate computer. It also was not able to maintain maximum coverage of the cleaning area while avoiding cracks [81]. ...
Modular self-configurable robot (MSR) systems have been investigated for decades, and their applications have been widely explored to meet emerging automation needs in various applications, such as space exploration, manufacturing, defense, medical industry, entertainment, and services. This paper aims to gain a deep understanding of up-to-date research and development on MSR through a thorough survey of market demands and published works on design methodologies, system integration, advanced controls, and new applications. In particular, the limitations of existing mobile MSR are discussed from the reconfigurability perspective of mechanical structures.
... Li and Zhao (2019) also evaluates learning rate values on crack detection task using deep neural networks. On the other hand, Kouzehgar et al. (2019) analyzes two optimizers (adagrad and adam) for a robot equipped with deep learning for crack detection. In fact, hyperparameters (e.g. ...
... In this sense, the methodology is adopted for recommending configurations of optimizer, learning rate and data augmentation. On the other hand, in general, among the studies analyzed, only one of these hyperparameters was tuned: optimizer (Gopalakrishnan et al. 2017;Kouzehgar et al. 2019), learning rate Li and Zhao 2019) and data augmentation (Ottoni et al. 2023a). In addition, the approach adopted statistical methods (ANOVA and Scott-Knott) to tuning these hyperparameters. ...
Deep learning methods have relevant applications in crack detection in buildings. However, one of the challenges in this field is the hyperparameter tuning process for convolutional neural networks (CNN). Thus, the objective of this paper is to propose a automated hyperparameter tuning approach for crack image classification. For this, a public dataset with 40,000 images of walls and floors of several buildings was used. The images are divided into two classes: negative (non-crack) and positive (crack). In this aspect, statistical methods are used for hyperparameter tuning, such as analysis of variance, Scott–Knott method and HyperTuningSK algorithm. Moreover, three new automated machine learning algorithms are proposed: AutoHyperTuningSK, AutoHyperTuningSK-test and AutoHyperTu-ningSK-DA. CNN architecture from the literature (MobileNet) and three types of hyperparameters (learning rate, optimizer and data augmentation) are analyzed. In general, the recommended configurations reached the best results in relation to unselected hyperparameters. In this regard, a selected combinations achieved a mean accuracy of around 99%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$99\%$$\end{document} (test experiments) in binary crack classification.
... Current crack detection tasks increasingly rely on fast detection devices such as drones (Ding et al., 2023), road measurement vehicles (Guo et al., 2023), and specially customized robots (Kouzehgar et al., 2019), as shown in Figure 1. These edge devices prioritize lightweight and real-time processing, often lacking high computational power. ...
Through extensive research on deep learning in recent years and its application in construction, crack detection has evolved rapidly from rough detection at the image-level and patch-level to fine-grained detection at the pixel-level, which better suits the nature of this field. Despite numerous existing studies utilizing off-the-shelf deep learning models or enhancing them, these models are not always effective or efficient in real-world applications. In order to bridge this gap, we propose a High-resolution model with Semantic guidance, specifically designed for real-time crack segmentation, referred to as HrSegNet. Our model maintains high resolution throughout the entire process, as opposed to recovering from low-resolution features to high-resolution ones, thereby maximizing the preservation of crack details. Moreover, to enhance the context information, we use low-resolution semantic features to guide the reconstruction of high-resolution features. To ensure the efficiency of the algorithm, we design a simple yet effective method to control the computation cost of the entire model by controlling the capacity of high-resolution channels, while providing the model with extremely strong scalability. Extensive quantitative and qualitative evaluations demonstrate that our proposed HrSegNet has exceptional crack segmentation capabilities, and that maintaining high resolution and semantic guidance are crucial to the final prediction. Compared to state-of-the-art segmentation models, HrSegNet achieves the best trade-off between efficiency and effectiveness. Specifically, on the crack dataset CrackSeg9k, our fastest model HrSegNet-B16 achieves a speed of 182 FPS with 78.43% mIoU, while our most accurate model HrSegNet-B48 achieves 80.32% mIoU with an inference speed of 140.3 FPS.
... Although deep learning for crack detection is intensively investigated for concrete civil infrastructure, limited studies explored such techniques for façade crack detection which generally contains more background noises. Existing studies have implemented deep CNN to detect cracked tiles (Shih and Chi 2020) and CNN to detect cracked glasses using a façade-cleaning robot (Kouzehgar et al. 2019). Additionally, generative adversarial networks (GANs) are used to detect cracks in UAV-captured motion blurred concrete façade images ) and a two-step CNN U-Net model to segment crack pixels for different façade systems (Chen et al. 2021b). ...
... Several automation techniques have been developed for automatically analyzing inspection data to extract meaningful information about the project [37][38][39][40]. Vision-based techniques, such as image processing and computer vision, can be used to identify construction defects or deficiencies, such as cracks, and/or missing components through visual analysis of the project site [39,[41][42][43][44]. Image processing employs mathematical algorithms, whereas computer vision employs machine learning models to extract high-level information from images. ...
... Robots are automation tools that have been introduced in construction for various purposes, such as precast concrete production [56], painting (E. [57], cleaning [43], tiling [58], prefabricated building assembly [59], and demolition waste handling [60]. A robot is defined by International Organization for Standardization (ISO) as a "programmed actuated mechanism with a degree of autonomy to perform locomotion, manipulation, or positioning" [61]; p. 1). ...
... The mounts are sealed by special sealants to maintain negative pressure and develop strong adhesion with the wall surface. A similar design was seen in [107]. Their robot Mantis was also designed as three suction modules joined by two links used for the inspection of window frames. ...
... When moving from one frame to another, one of the modules would detach and go over the frame followed by the other two modules one-by-one. Kouzehgar et al. [107] acknowledged that cracked glass windows may create a dangerous situation for their robot; therefore, they also developed a crack-detection algorithm to avoid attaching to cracked surfaces. In contrast, the ROMERIN robot developed by [108] used six inter-linked suction cups and turbines, instead of pumps, to create stronger air flow and negative air pressure. ...
... Robotic inspection also provides a safer alternative to manual inspection. High-rise towers with glass facades, which are a common sight in most cities, require regular maintenance [107]. The classical approach to the maintenance of high-rise buildings poses a high risk to human workers due to high winds [107]. ...
Regular inspection and monitoring of buildings and infrastructure, that is collectively called the built environment in this paper, is critical. The built environment includes commercial and residential buildings, roads, bridges, tunnels, and pipelines. Automation and robotics can aid in reducing errors and increasing the efficiency of inspection tasks. As a result, robotic inspection and monitoring of the built environment has become a significant research topic in recent years. This review paper presents an in-depth qualitative content analysis of 269 papers on the use of robots for the inspection and monitoring of buildings and infrastructure. The review found nine different types of robotic systems, with unmanned aerial vehicles (UAVs) being the most common, followed by unmanned ground vehicles (UGVs). The study also found five different applications of robots in inspection and monitoring, namely, maintenance inspection, construction quality inspection, construction progress monitoring, as-built modeling, and safety inspection. Common research areas investigated by researchers include autonomous navigation, knowledge extraction, motion control systems, sensing, multi-robot collaboration, safety implications, and data transmission. The findings of this study provide insight into the recent research and developments in the field of robotic inspection and monitoring of the built environment and will benefit researchers, and construction and facility managers, in developing and implementing new robotic solutions.
... AGV/AMR technology adopts sensor technology and signal processing technology, and integrates various onboard sensors such as vision and lidar to be applied to the construction field to realize unmanned construction, identify the environment and status of the construction site, and evaluate the acquired road information, vehicle location and obstacle information to make decisions and carry out construction operations [241][242][243][244][245]. ...
Automated guided vehicles (AGVs) and autonomous mobile robots (AMRs) have been widely used recently to solve various engineering problems in logistics, manufacturing, and high-risk labor. This paper reviews the latest research on AGVs and AMRs and includes results from different past and present research areas on AGVs and AMRs. In addition, various navigation principles of AGV are compared. This paper also discusses and compares various types of AGV/AMR visual tracking control technologies and presents three recognition and tracking integration technologies, which improve the accuracy, robustness, and real-time performance of AGV/AMR visual navigation systems. Finally, the application of AGV/AMR technologies in civil engineering is reviewed and discussed, including for road pavements, bridges, and construction. These technologies play an important role in construction, defect detection, and condition inspection, free up manpower and improve the degree of automation in the civil engineering industry.
... However, smashed glass could create hazardous conditions if these robots are allowed to continue their journey. To prevent the affected areas from spreading, it appears necessary to equip them with a crack detection system [2]. This study proposes a deep-learning-based method for detecting cracks in a unique modular facade-cleaning robot developed using convolutional neural networks. ...
Cracks in the pavement first appear in the outer layer, visible through the eyes; they deteriorate on a deeper layer, affecting the entire construction. Nowadays, technical advancement plays a vital role in the detection of cracks. Artificial intelligence and machine learning algorithms are used for pixel-level classification in computer vision and image processing. The detection of cracks in the pavement is a crucial task, and it requires an ample amount of time. In the proposed crack detection model, the feature extraction and feature selection are based on the lion optimization algorithm (LOA), which extracts the best features for the selected area of the bounding box. The parts are removed with the lion’s optimal position and velocity, which can also be used for the best feature selection. The convolutional neural network trains and tests the model with transfer learning methods. The proposed LOA-based crack detection algorithm outperforms the existing crack detection algorithms with a precision of 94.7 percent, a recall of 95.22 percent, and an F1 score of 94.89 percent.
... The learning rate of the SGD method remains constant throughout the training phase, which brings about severe problems. In specific, a small learning rate results in slow convergence, while a large learning rate leads to violent oscillation near the optimum (either global or local) parameter set h*. Improved learning methods like AdaGrad [43] can adjust learning rate automatically according to historical gradients: ...
This paper proposes a real-time prediction method for key monitoring physical parameters (KMPPs) for early warning of fire-induced building collapse using machine learning. Since the actual load distribution and structural material properties of the burning building are usually unknown and uncertain, easy-to-measure parameters of the burning building, including easy-to-measure KMPPs (displacements and displacement velocities) of key joints, and temperatures of key structural members of the building, are incorporated as the input to predict the hard-to-measure KMPPs. The long short-term memory network is adopted in the machine learning framework. The network can be trained offline during the design stage through simulated data and used online with real-time measured data in fire. A portal frame building is numerically examined, and the results indicate that the trained agent can identify unknown and uncertain parameters and predict the hard-to-measure KMPPs with high accuracy and efficiency, enhancing the accuracy and reliability of early warning for fire-induced building collapse.
