Figure 5 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Source publication
In this paper, a complete system based on computer vision and deep learning is proposed for surface inspection of the armatures in a vibration motor with miniature volume. A device for imaging and positioning was designed in order to obtain the images of the surface of the armatures. The images obtained by the device were divided into a training se...
Contexts in source publication
Context 1
... traditional computer vision, the information such as edges, textures, and colors are extracted and summarized. In our study, we used Sobel edge detection (Figure 5b), binarization (Figure 5d), and Canny edge detection ( Figure 5e) to extract edges and textures information. Harris detection was used to detect the corners (Figure 5a). ...
Context 2
... traditional computer vision, the information such as edges, textures, and colors are extracted and summarized. In our study, we used Sobel edge detection (Figure 5b), binarization (Figure 5d), and Canny edge detection ( Figure 5e) to extract edges and textures information. Harris detection was used to detect the corners (Figure 5a). ...
Context 3
... traditional computer vision, the information such as edges, textures, and colors are extracted and summarized. In our study, we used Sobel edge detection (Figure 5b), binarization (Figure 5d), and Canny edge detection ( Figure 5e) to extract edges and textures information. Harris detection was used to detect the corners (Figure 5a). ...
Context 4
... our study, we used Sobel edge detection (Figure 5b), binarization (Figure 5d), and Canny edge detection ( Figure 5e) to extract edges and textures information. Harris detection was used to detect the corners (Figure 5a). Because the main features of the inspected area were copper wire and tin, we took advantage of histogram backprojection to extract the areas that are similar to the tin in color ( Figure 5c). ...
Context 5
... detection was used to detect the corners (Figure 5a). Because the main features of the inspected area were copper wire and tin, we took advantage of histogram backprojection to extract the areas that are similar to the tin in color ( Figure 5c). In addition, the copper wire and tin are different in values between the R channel and B channel. ...
Context 6
... addition, the copper wire and tin are different in values between the R channel and B channel. Therefore, we subtracted the R and B channels to highlight the position of the copper wire ( Figure 5f). As can be seen from Figure 5, all methods suffer from a degree of information loss due to the effects of areas other than copper wires and tin, so the RGB image keeps the features integrated. ...
Context 7
... we subtracted the R and B channels to highlight the position of the copper wire ( Figure 5f). As can be seen from Figure 5, all methods suffer from a degree of information loss due to the effects of areas other than copper wires and tin, so the RGB image keeps the features integrated. We need a more effective feature extraction method for feature extraction of RGB images. ...
Citations
... The deep network has a low resolution and learns semantic features. The higher the level, the greater the abstraction of the features, the smaller the size of the feature map, and small-sized objects are easily missed (15). The location and size of the gallbladder in this study were different, and the deep residual convolutional network was relatively easy to false alarms, making the detection results inaccurate and affecting the classification accuracy of the model. ...
Background
Many screening and diagnostic methods are currently available for biliary atresia (BA), but the early and accurate diagnosis of BA remains a challenge with existing methods. This study aimed to use deep learning algorithms to intelligently analyze the ultrasound image data, build a BA ultrasound intelligent diagnostic model based on the convolutional neural network, and realize an intelligent diagnosis of BA.
Methods
A total of 4,887 gallbladder ultrasound images of infants with BA, non-BA hyperbilirubinemia, and healthy infants were collected. Two mask region convolutional neural network (Mask R-CNN) models based on different backbone feature extraction networks were constructed. The diagnostic performance between the two models was compared through good-quality images at the image level and the patient level. The diagnostic performance between the two models was compared through poor-quality images. The diagnostic performance of BA between the model and four pediatric radiologists was compared at the image level and the patient level.
Results
The classification performance of BA in model 2 was slightly higher than that in model 1 in the test set, both at the image level and at the patient level, with a significant difference of p = 0.0365 and p = 0.0459, respectively. The classification accuracy of model 2 was slightly higher than that of model 1 in poor-quality images (88.3% vs. 86.4%), and the difference was not statistically significant (p = 0.560). The diagnostic performance of model 2 was similar to that of the two radiology experts at the image level, and the differences were not statistically significant. The diagnostic performance of model 2 in the test set was higher than that of the two radiology experts at the patient level (all p < 0.05).
Conclusion
The performance of model 2 based on Mask R-CNN in the diagnosis of BA reached or even exceeded the level of pediatric radiology experts.
... To address the issue of imbalanced data and enhance the accuracy and generalization capability of the model in real-world PV segmentation scenarios, we propose GenPV, a deep learning method integrated with multi-scale feature learning, inductive learning, and hard sample mining. As illustrated in Figure 5, GenPV is capable of handling multiple resolution images through the use of data augmentation and a ResNet-101 backbone [45], which enables the extraction of abundant features. Subsequently, multi-scale feature learning is employed to enhance the encoder process, with a particular focus on the improved FPN. ...
... FPN leverages the feature hierarchy within the network that generates feature maps with varying resolutions to build the pyramid. To incorporate contextual information across multiple scales, FPN blends features of different scales by upsampling and summing them in a top-down pathway [45]. In addition, FPN aims to extract features from a single input image to create the pyramid format without sacrificing representational power, speed, and memory, unlike hierarchical images that have limitations in terms of memory [21]. ...
Accurate and generalizable photovoltaic panel segmentation using deep learning for imbalanced datasets, Renewable Energy (2023), doi: https://doi. Abstract: The widespread adoption of photovoltaic (PV) technology for renewable energy necessitates accurate segmentation of PV panels to estimate installation capacity. However, achieving highly efficient and precise segmentation methods remains a pressing challenge. Recent advancements in artificial intelligence and remote sensing techniques have shown promise in PV segmentation. Nevertheless, real-world scenarios introduce complexities such as diverse sensing platforms, sensors, panel categories, and testing regions. These factors contribute to resolution, size, and foreground-background class imbalances, impeding accurate and generalized PV panel segmentation over large areas. To address these challenges, we propose GenPV, a deep learning model that leverages data distribution analysis and PV panel characteristics to enhance segmentation accuracy and generalization. GenPV employs a multi-scale feature learning approach, utilizing an enhanced feature pyramid network to fuse data features from multiple resolutions, effectively addressing resolution imbalance. Moreover, inductive learning is employed through a multitask approach, facilitating the detection and identification of both small and large-sized PV panels to mitigate size imbalance. To address significant class imbalance in PV panel recognition tasks, we integrate the Focal loss function for effective hard sample mining. Through experimental evaluation conducted in Heilbronn, Germany, our proposed method demonstrates superior performance compared to state-of-the-art approaches in PV panel segmentation. The results exhibit progressively higher accuracy and improved generalization capability. These findings highlight the potential of our method to serve as an advanced and practical tool for PV segmentation in the renewable energy field. J o u r n a l P r e-p r o o f 2 Highlights 1. Analysis of the imbalance problems in real-world PV semantic segmentation scenarios; 2. Creation of an open-source PV panel dataset to advance renewable energy development; 3. Introducing a novel end-to-end DL model named GenPV for PV panel segmentation; 4. Improved accuracy and generalization in PV segmentation across unaligned datasets.
... The FPN network model is divided into two network routes. One of the network routes produces multi-scale features from bottom to top, connecting the high-level features with high semantics and low resolution and the low-level features with high resolution and low semantics [33]. Another network route is from top to bottom; after some layer changes, the rich semantic information contained in the upper layer is transferred layer by layer to the low-layer features for fusion [34]. ...
In the process of image shooting, due to the influence of angle, distance, complex scenes, illumination intensity, and other factors, small targets and occluded targets will inevitably appear in the image. These targets have few effective pixels, few features, and no obvious features, which makes it difficult to extract their effective features and easily leads to false detection, missed detection, and repeated detection, thus affecting the performance of target detection models. To solve this problem, an improved faster region convolutional neural network (RCNN) algorithm integrating the convolutional block attention module (CBAM) and feature pyramid network (FPN) (CF-RCNN) is proposed to improve the detection and recognition accuracy of small-sized, occluded, or truncated objects in complex scenes. Firstly, it incorporates the CBAM attention mechanism in the feature extraction network in combination with the information filtered by spatial and channel attention modules, focusing on local efficient information of the feature image, which improves the detection ability in the face of obscured or truncated objects. Secondly, it introduces the FPN feature pyramid structure, and links high-level and bottom-level feature data to obtain high-resolution and strong semantic data to enhance the detection effect for small-sized objects. Finally, it optimizes non-maximum suppression (NMS) to compensate for the shortcomings of conventional NMS that mistakenly eliminates overlapping detection frames. The experimental results show that the mean average precision (MAP) of target detection of the improved algorithm on PASCAL VOC2012 public datasets is improved to 76.2%, which is 13.9 percentage points higher than those of the commonly used Faster RCNN and other algorithms. It is better than the commonly used small-sample target detection algorithm.
... In addition, we also compare the proposed method with other deep learning models, including Faster R-CNN [39], YOLOv3 [40], Inception-v3 [41], ResNet-50, ResNet-101 [42], Ref [26], and Ref [43]. Furthermore, the effectiveness of the mixed dataset augmentation and the ensemble learning proposed in this study is also demonstrated by ablative analysis. ...
Automatically and accurately detecting the self‐blast state of glass insulators is of great significance to operation and maintenance of transmission lines. To solve the shortcomings of the existing open‐loop cognitive models to detect the self‐blast state of glass insulators, this study explores a mixed data augmentation‐based intelligent recognition method to detect the self‐blast state of the glass insulator, by imitating the human cognitive mode. Firstly, generative adversarial network is utilised to obtain the high‐quality generative self‐blast samples of the glass insulator, and the non‐generative data augmentation techniques is used to obtain rich sample features. Secondly, considering the characteristics of aerial images such as large scale variations, variable shooting angles and complex backgrounds, feature maps with strong semantics and adaptive multi‐scale fusion are extracted using the feature pyramid network with adaptive hierarchy and the multi‐deformable convolutional network. Then, the extracted feature maps are transmitted to a two‐dimensional stochastic configuration network that can adaptively generate hidden nodes and basis functions so as to develop the self‐blast state classification criteria with universal approximation capability. Thirdly, based on the generalised error and entropy theory, the semantic error entropy evaluation indices of recognition results are defined to evaluate in real time, the credibility of the uncertain recognition results for the self‐blast state of the glass insulator. Then, based on transfer learning and the established self‐optimising feedback mechanism for feature pyramid network, the self‐optimising adjustment and reconstruction of the feature map space with strong semantics and multi‐scale fusion and its classification criteria are realised. Finally, the stacking method is applied to integrate the recognition results of the feature pyramid network with adaptive hierarchy and multi‐channel deformable convolutional networks to improve the robustness of the recognition model. Results of experimental comparison with other machine learning and deep learning methods verify the feasibility and effectiveness of the proposed method.
... en, the fixed convolution layer and pooling layer are replaced with deformable convolution layer and deformable pooling layer. Finally, the FPN [28][29][30] network is used for multifeature fusion, and the soft nonmaximum suppression (soft NMS) [31] is used to reduce the confidence of the detection frame larger than the threshold, so as to alleviate the situation of the target missing detection. According to the test results on the open dataset NEU-DET, the proposed algorithm can effectively detect a variety of defects on steel surface, which is higher than the ordinary steel surface detection algorithms in accuracy. ...
In recent years, more and more scholars devoted themselves to the research of the target detection algorithm due to the continuous development of deep learning. Among them, the detection and recognition of small and complex targets are still a problem to be solved. The authors of this article have understood the shortcomings of the deep learning detection algorithm in detecting small and complex defect targets and would like to share a new improved target detection algorithm in steel surface defect detection. The steel surface defects will affect the quality of steel seriously. We find that most of the current detection algorithms for NEU-DET dataset detection accuracy are low, so we choose to verify a steel surface defect detection algorithm based on machine vision on this dataset for the problem of defect detection in steel production. A series of improvement measures are carried out in the traditional Faster R-CNN algorithm, such as reconstructing the network structure of Faster R-CNN. Based on the small features of the target, we train the network with multiscale fusion. For the complex features of the target, we replace part of the conventional convolution network with a deformable convolution network. The experimental results show that the deep learning network model trained by the proposed method has good detection performance, and the mean average precision is 0.752, which is 0.128 higher than the original algorithm. Among them, the average precision of crazing, inclusion, patches, pitted surface, rolled in scale and scratches is 0.501, 0.791, 0.792, 0.874, 0.649, and 0.905, respectively. The detection method is able to identify small target defects on the steel surface effectively, which can provide a reference for the automatic detection of steel defects.
In recent years, with the development of remote sensing technology and the enhancement of the value of remote sensing images in military and civil fields, remote sensing image object segmentation has also received more and more attention. This paper mainly studies the application of instance segmentation based on deep convolutional neural network in the remote sensing image. This paper proposes an attention balanced feature pyramid module, which strengthens multi-level features and uses the attention module to suppress the interference features of noise in the complex background. In addiction, Soft-NMS is introduced to improve the performance of the network, and GIoU loss is introduced to improve the effect of object detection. The proposed network improves the average detection and segmentation accuracy (mAP) values from [Formula: see text] and [Formula: see text] to [Formula: see text] and [Formula: see text], respectively.
The operating state of insulators is directly related to the stability of power transmission line. The existing methods for insulator state recognition cannot achieve satisfactory performance. In this paper, the self-blast state recognition of glass insulators is investigated by using an adaptive learning representation. To increase the adaptability of the network to different scales, we propose a solution based on multi-scale information throughout the entire process, beginning from a low-scale to high-scale subnetworks. The multi-scale information is aggregated in parallel way to take advantage of rich information representation. Then, an imitation of the human thinking pattern is employed. Utilizing entropy-based cost function, we update the parameters of the learner model in real-time. Based on the constraint of the evaluation index, adaptive depth representation for training glass insulators that are unsatisfied with the reliability evaluation is constructed to realize the self-optimizing regulation of feature space. Correspondingly, a stochastic configuration networks (SCNs) classifier is re-constructed to fit for the update multi-hierarchies knowledge space to carry out the re-recognition process. Finally, fuzzy integration is employed to ensemble multi-hierarchies network to improve the model’s generalization. The recognition results on aerial dataset of insulators images demonstrate the effectiveness of our proposed approach.
