Content uploaded by Najiya Koderi Valappil
Author content
All content in this area was uploaded by Najiya Koderi Valappil on Dec 14, 2020
Content may be subject to copyright.
A preview of the PDF is not available
Pavement Crack Detection and Localization using Convolutional Neural Networks (CNNs)
... To conduct classification tasks, the CNN ends with an FC layer with a SoftMax activation function to normalize the output of a network to a probability distribution over predicted output classes [12]. The effectiveness of CNN models in pavement distress detection and classification has been proven in several studies and experiment results [12][13][14][15][16][17][18][19]. ...
... The ASTM D6433 also scales PCI into several ratings of good (85-100), satisfactory (70-85), fair (55-70), poor (40-55), very poor (25)(26)(27)(28)(29)(30)(31)(32)(33)(34)(35)(36)(37)(38)(39)(40), serious (10)(11)(12)(13)(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25), and failed (0-10), which could be treated as a classification task. Therefore, in this research, a feasibility study of CNN-based PCI estimation was conducted on aerial imagery (Google Earth) to rate the PCI at the multi-level as well. ...
... That is the major difference from FCNs, because an FCN model typically does not contain FC, but it uses a convolutional layer with a Sigmoid activation function as the network's end layer for generating the same-sized output results as the input images [27]. Furthermore, CNNs can be used with the sliding window scheme (or overlapping small patches [12,14]) to perform crack and non-crack binary classification tasks [14][15][16][17] and pavement cracking category classification tasks [15] in each small patch of a large-resolution 2D/3D image. Moreover, when the size of the window patches is very small, for example, 13 × 13 pixels [14], the CNN-based image patch classification results would be properly annotating cracks on the large-resolution images [14,16]. ...
This paper aims to explore and evaluate aerial imagery and deep learning technology in pavement condition evaluation. A convolutional neural network (CNN) model, named PCIer, was designed to process aerial images and produce pavement condition index (PCI) estimations, which are classified into four scales of Good (PCI ≥ 70), Fair (50 ≤ PCI < 70), Poor (25 ≤ PCI < 50), and Very Poor (PCI < 25). In the experiment, the PCI datasets were retrieved from the published pavement condition report by the City of Sacramento, CA. Following the retrieved datasets, the authors also collected the corresponding aerial image datasets containing 100 images for each PCI grade from Google Earth. An 80% proportion of datasets were used for PCIer model training, and the remaining were used for testing. Comparisons showed using a 128-channel heatmap layer in the proposed PCIer model and saving the PCIer model with the best validation accuracy would yield the best performance, with a testing accuracy of 0.97, and a weighted average precision, recall, and F1-score of 0.98, 0.97, and 0.97, respectively. Moreover, future research recommendations are provided in the discussion for improving the effectiveness of pavement evaluation via aerial imagery and deep learning.
... The reviewed studies have achieved various pavement evaluation objectives with the remote sensing and nondestructive testing techniques in pavement surface properties evaluation, pavement geometrical properties evaluation, and other related objectives. There were different types of data sources used in different pavement evaluation objectives, while the most common two types are: a) 2D imagery that was captured by digital cameras (smart phones) that were carried by operators, mounted on vehicles, and carried by drones (Ali et al., 2019;Dadrasjavan et al., 2019;Dorafshan et al., 2019;Y. Liu et al., 2020); b) 3D imagery that was directly generated from laser line profile sensors (Edmondson et al., 2019;Zhou & Song, 2020a, 2020b, and converted from point clouds by 3D laser scanner (Edmondson et al., 2019) and SfM photogrammetry (Edmondson et al., 2019;Roberts et al., 2020). ...
... Liu et al., 2019), and Temple University (F. . Another approach is capturing 2D images through a drone mounted digital camera (Ali et al., 2019;Dadrasjavan et al., 2019;Dorafshan et al., 2019;Y. Liu et al., 2020), which can minimize manual operation in most conditions, but is not a safe choice for surveying a roadway with a high traffic volume. ...
... CNNs can be used with sliding window scheme (or overlapping small patches Protopapadakis et al., 2019)) to perform crack and non-crack binary classification tasks (Ali et al., 2019;Maniat, 2019;Protopapadakis et al., 2019;Zhou & Song, 2020a) and pavement cracking categories classification tasks (Maniat, 2019) in each smallpatch of a large resolution 2D/3D image. Moreover, when the size of the window patches are very small, like 13×13pixel (Protopapadakis et al., 2019), the CNN-based image patch classification results would be properly annotating cracks on the large resolution images (Protopapadakis et al., 2019;Zhou & Song, 2020a). ...
This review paper aims to evaluate the remote sensing and neural network-driven pavement evaluation (RSNNPE) techniques. The collected studies were published in recent years and mainly relate to pavement distresses detection, classification, and quantification. The data collections were conducted by remote sensing and non-destructive techniques, including photography, hyperspectral imagery, satellite imagery, photogrammetry, laser scanning, ground penetrating radar, and laminography. The data analysis was conducted by neural network (NN) modeling, image filtering, threshold segmentation, template matching, SVM, and random forest. The NN architectures include MLP, RNN for structured data; CNN, Faster R-CNN, NIN for 2D/3D imagery patch-wise or object-orientated pavement distresses detection; and FCN, U-net, SegNet, PSPNet, DeepLabv3+, Mask-RCNN, DeepCrack, CrackSeg, FPHBN, CrackGAN for 2D/3D imagery pixelwise segmentation. Moreover, this paper discusses drone photogrammetry-based data acquisition, data preparation, and NN architecture selection for pavement evaluation. Based on the results of the review, future research recommendations are proposed.
... Deep learning used in asset management and detection systems is successful nowadays due to advances in artificial intelligence and computer hardware [9], [10], [11]. Several different techniques are used in pavement crack detection, including Convolutional neural networks (CNN) [12], [13], [14]. Two-stage CNN [15], faster R-CNN [16], a Network CNN [13] and others of its kind exist to aid in pavement crack detection. ...
... Several different techniques are used in pavement crack detection, including Convolutional neural networks (CNN) [12], [13], [14]. Two-stage CNN [15], faster R-CNN [16], a Network CNN [13] and others of its kind exist to aid in pavement crack detection. The faster processing involves less fine-tuning. ...
Monitoring asset conditions is a crucial factor in building efficient transportation asset management. Because of substantial advances in image processing, traditional manual classification has been largely replaced by semi-automatic/automatic techniques. As a result, automated asset detection and classification techniques are required. This paper proposes a methodology to detect and classify roadway pavement cracks using the well-known You Only Look Once (YOLO) version five (YOLOv5) and version 8 (YOLOv8) algorithms. Experimental results indicated that the precision of pavement crack detection reaches up to 67.3% under different illumination conditions and image sizes. The findings of this study can assist highway agencies in accurately detecting and classifying asset conditions under different illumination conditions. This will reduce the cost and time that are associated with manual inspection, which can greatly reduce the cost of highway asset maintenance.
... A great number of studies have incorporated Deep Neural Networks together with image computing techniques, for the purposes of defect detection in historical building façades. The most common deep learning approaches that can be used for detection of objects and defects from the 2D/3D images in the architecture, engineering and constructing industry are the convolutional neural networks (CNNs)based image classification and patch-wise segmentation [31] and fully convolutional networks (FCNs)-based pixelwise segmentation [32,33]. Therefore, a great number of authors have made contributions such as an automated defect detection and classification method from closed-circuit television (CCTV) inspections based on a deep convolutional neural network (DCNN) that takes advantage of the large volume of inspection data [34], a conditionaware model of structures that incorporated a textured 3D building model with defects detected by deep learning models and mapped using UV mapping [35], a vision-based method for concrete crack detection and density evaluation using a deep fully convolutional network (FCN) [37], a transfer-learning method based on multiple DCNN knowledge for crack detection [38] and an automated defect detection system with an object detector based on a CNN [39]. ...
The creation of geometric digital twins of residential buildings from pictures of their facades is studied. After a short literature review, an application of image analysis based on YOLO neural networks object detection software is presented. A quick CAD model, including essential elements and damages is created, which can be used for preliminary strength analysis and evaluation. A fine-tuned YOLOv7 model, trained on the Cracks Dataset, was employed for crack detection, demonstrating notable performance with F1-score and mAP50 scores of 0.80 and 0.82, respectively.
... DL algorithms are particularly effective in detecting features in images because they can automatically learn the feature representations from the data itself. Various works [17]- [21] have been presented in the literature using various DL based Convolutional Neural Network (CNN) models for crack detection in civil structures. However, these approaches suffer from localized receptive field problems in which the feature are not extracted in a global context. ...
... Various factors influence CNN performance, including hyperparameter selection and architecture fine-tuning [18]. Initial research concentrated on patch-based crack identification utilizing datasets consisting of crack and non-crack patches [19][20][21][22][23][24]. However, a pixel-wise crack-detection approach is necessary to perform crack localization and assess the crack widths, lengths, and propagation directions. ...
Recently, deep-learning (DL)-based crack-detection systems have proven to be the method of choice for image processing-based inspection systems. However, human-like generalization remains challenging, owing to a wide variety of factors such as crack type and size. Additionally, because of their localized receptive fields, CNNs have a high false-detection rate and perform poorly when attempting to capture the relevant areas of an image. This study aims to propose a vision-transformer-based crack-detection framework that treats image data as a succession of small patches, to retrieve global contextual information (GCI) through self-attention (SA) methods, and which addresses the CNNs’ problem of inductive biases, including the locally constrained receptive-fields and translation-invariance. The vision-transformer (ViT) classifier was tested to enhance crack classification, localization, and segmentation performance by blending with a sliding-window and tubularity-flow-field (TuFF) algorithm. Firstly, the ViT framework was trained on a custom dataset consisting of 45K images with 224 × 224 pixels resolution, and achieved accuracy, precision, recall, and F1 scores of 0.960, 0.971, 0.950, and 0.960, respectively. Secondly, the trained ViT was integrated with the sliding-window (SW) approach, to obtain a crack-localization map from large images. The SW-based ViT classifier was then merged with the TuFF algorithm, to acquire efficient crack-mapping by suppressing the unwanted regions in the last step. The robustness and adaptability of the proposed integrated-architecture were tested on new data acquired under different conditions and which were not utilized during the training and validation of the model. The proposed ViT-architecture performance was evaluated and compared with that of various state-of-the-art (SOTA) deep-learning approaches. The experimental results show that ViT equipped with a sliding-window and the TuFF algorithm can enhance real-world crack classification, localization, and segmentation performance.
Pavement defect detection is of profound significance regarding road safety, so it has been a trending research topic. In the past years, deep learning based methods have turned into a key technology, with advantages of high accuracy, strong robustness, and adaptability to complex pavement environments. This paper first reviews the methods based on image processing and 3D imaging. As for image-based processing techniques, they can be classified into three types based on how to label the collected data: fully supervised learning, unsupervised learning, and other methods. Different methods are further classified and compared with each other. Second, the pavement detection methods based on 3D data are sorted out, thereby summarizing their benefits, drawbacks, and application scenarios. Third, the study proposed the major challenges in the field of pavement defect detection, introduced validated datasets and evaluation metrics. Finally, on the basis of reviewing the literature in pavement defect detection, the promising direction is put forward.
Pavement maintenance poses serious economic consequences as they have been reported to cost annually billions of dollars in the US alone. Traditional approaches of handling pavement cracks surveys and inventory are tedious, ineffective, including safety risks and error prone. Investing in machine intelligent solutions should effectively address all of these problems and concerns. In this chapter, we present a proposed solution of the survey and inventory of pavement cracks based on Deep Learning (DL) models implementing Convolution Neural Networks (CNN) to detect and classify five types of road problems including longitudinal, traverse, block, alligator cracks and potholes. The development process is detailed and the experimental work, results and observations are discussed.KeywordsPavement cracksAutomatic detectionIntelligent surveyCrack inventoryDeep learningMachine intelligence
This paper proposes an effective and efficient model for concrete crack detection. The presented work consists of two modules: multi-view image feature extraction and multi-task crack region detection. Specifically, multiple visual features (such as texture, edge, etc.) of image regions are calculated, which can suppress various background noises (such as illumination, pockmark, stripe, blurring, etc.). With the computed multiple visual features, a novel crack region detector is advocated using a multi-task learning framework, which involves restraining the variability for different crack region features and emphasizing the separability between crack region features and complex background ones. Furthermore, the extreme learning machine is utilized to construct this multi-task learning model, thereby leading to high computing efficiency and good generalization. Experimental results of the practical concrete images demonstrate that the developed algorithm can achieve favorable crack detection performance compared with traditional crack detectors.
A number of image processing techniques (IPTs) have been implemented for detecting civil infrastructure defects to partially replace human-conducted on-site inspections. These IPTs are primarily used to manipulate images to extract defect features, such as cracks in concrete and steel surfaces. However, the extensively varying real-world situations (e.g., lighting and shadow changes) can lead to challenges to the wide adoption of IPTs. To overcome these challenges, this article proposes a vision-based method using a deep architecture of con-volutional neural networks (CNNs) for detecting concrete cracks without calculating the defect features. As CNNs are capable of learning image features automatically , the proposed method works without the conjugation of IPTs for extracting features. The designed CNN is trained on 40 K images of 256 × 256 pixel resolutions and, consequently, records with about 98% accuracy. The trained CNN is combined with a sliding window technique to scan any image size larger than 256 × 256 pixel resolutions. The robustness and adaptability of the proposed approach are tested on 55 images of 5,888 × 3,584 pixel resolutions taken from a different structure which is not used for training and validation processes under various conditions (e.g., strong light spot, shadows , and very thin cracks). Comparative studies are conducted to examine the performance of the proposed CNN using traditional Canny and Sobel edge detection methods. The results show that the proposed method shows
Automatic detection of pavement cracks is an important task in transportation maintenance for driving safety assurance. However, it remains a challenging task due to the intensity inhomogeneity of cracks and complexity of the background, e.g., the low contrast with surrounding pavement and possible shadows with similar intensity. Inspired by recent success on applying deep learning to computer vision and medical problems, a deep-learning based method for crack detection is proposed in this paper. A supervised deep convolutional neural network is trained to classify each image patch in the collected images. Quantitative evaluation conducted on a data set of 500 images of size 3264 × 2448, collected by a low-cost smart phone, demonstrates that the learned deep features with the proposed deep learning framework provide superior crack detection performance when compared with features extracted with existing hand-craft methods.
Cracks cause deterioration of road performance and functional or structural failure if not managed in a timely manner. This paper proposes an automated crack detection method using a car black box camera to address this problem. The proposed method uses a deep learning model [i.e., convolutional neural network (CNN)] composed of segmentation and classification modules. The segmentation process is performed to extract only the road surface in order to remove elements that interfere with crack detection in the black box image. Then, cracks are detected through analysis of patch units within the extracted road surface. The proposed CNN architecture classifies the elements of the road surface into three categories (i.e., crack, road marking, and intact area) with 90.45% accuracy. The results of the proposed CNN architecture are better than those of previous studies.
Due to the complexity and diversity of pavement surfaces, pavement cracking detection is a challenging task even for human operators. The automation of pavement cracking detection generally requires robust algorithms with high level of intelligence. From such a perspective, Deep Learning, a promising branch of Artificial Intelligence, can serve as an advanced approach to high level of intelligence by learning from huge amount of historical data and enhancing the capability of behaving correctly under unforeseen and complex environments. This paper proposes a convolutional neural network (CNN) for recognizing cracks on asphalt surfaces at subdivided image cells. The proposed CNN consists of 3 convolution layers and 2 fully-connected layers with 1,246,240 parameters in total. Feeding the proposed CNN with 5,000 manually-processed example images recursively, the accuracy of the proposed CNN is progressively improved, and the generalization is also enhanced. It is demonstrated in the paper that the trained CNN can achieve high accuracies 96.32 and 94.29% on training data and testing data respectively.
Among the various defects of asphalt pavement distress, much attention has been paid to cracks which often cause significant engineering and economic problems. Crack detection is not an easy task since images of road pavement surface are very difficult to analyze. In this paper, a highly efficient pavement crack detection system is proposed, which has the following distinguishing features. Firstly, a new description of the cracks is proposed based on the spatially clustered pixels with similar grey levels. Secondly, an adaptive thresholding method is presented for image segmentation by comprehensively taking into account the spatial distribution, intensities and geometric features of cracks. Thirdly, a new concept termed Region of Belief (ROB) is introduced to facilitate the subsequent detection by defining some credibility factors which indicate the reliability that a region could be labeled as a distress region which contains cracks, and an algorithm to extract such ROBs is devised accordingly. Lastly, a novel region growing algorithm is propounded for crack detection, which features starting with an ROB seed, determining the searching scope with a specially devised rule, and searching and merging a ROB with different regions following a similarity criterion which synthetically takes different cues into consideration. Two different types of experiments were conducted. The first one was carried out using 10,000 of our field-captured images which were taken from different road conditions and environments. The second one was completed using a benchmark dataset for a comparison with other recent publications. The evaluation performance is satisfactory for a variety of different cracks. For our own data, the detection accuracy is over 95% and more than 90% of coherent cracks without disconnected fragments have been correctly detected as the integrated ones. For the benchmark data, our detection performance also outperforms previously published results. Currently, our approach has been widely applied in China.
Visual inspection of bridges is customarily used to identify and evaluate faults. However, current procedures followed by human inspectors demand long inspection times to examine large and difficult to access bridges. Also, highly relying on an inspector's subjective or empirical knowledge induces false evaluation. To address these limitations, a vision-based visual inspection technique is proposed by automatically processing and analyzing a large volume of collected images. Images used in this technique are captured without controlling angles and positions of cameras and no need for preliminary calibration. As a pilot study, cracks near bolts on a steel structure are identified from images. Using images from many different angles and prior knowledge of the typical appearance and characteristics of this class of faults, the proposed technique can successfully detect cracks near bolts.
