Figure - available from: Journal of Intelligent Manufacturing
This content is subject to copyright. Terms and conditions apply.
Source publication
The classification of defect types during LCD panel production is very important because it is closely related to deciding whether a defect panel is restorable. But since defect areas are very small compared to the panel area, it is hard to classify defect types by images. Therefore, we need to eliminate the background pattern of the panel, but thi...
Citations
... To surmount the hurdles of incomplete defect information extraction in ADC research (Zhi et al., 2023), the issues of low accuracy in classifying defects against complex backgrounds, and the challenge of slow model convergence (Kim et al., 2020), this study introduces a novel lightweight deep learning network architecture. This architecture synergistically combines a CNN model with a Transformer module. ...
Efficient management and control of wafer defects are paramount in enhancing yield in IC chip manufacturing. Scanning Electron Microscope imagery of wafer surfaces, however, presents a challenge due to complex backgrounds and a minimal presence of actual defects. This complexity often hampers traditional convolutional neural networks tasked with defect classification and segmentation, making them prone to disturbances from background elements. To address this issue, we introduce a novel interwoven network architecture that synergizes convolution and Transformer models. This integrated approach is specifically designed to surmount the dual challenges of classification and joint segmentation in wafer defects, achieving a balance between computational efficiency and prediction accuracy. Our research, grounded in real-world production line data from IC chip manufacturing, demonstrates that our network attains a segmentation accuracy of 83.15% and a classification accuracy of 96.88%. The proposed method for automatic defect information extraction is shown to be viable for industrial application. The merger of convolutional neural networks with Transformer models in this innovative architecture shows considerable promise for enhancing wafer defect analysis, thereby improving the precision of defect classification and segmentation in semiconductor manufacturing processes.
... However, manual inspection methods are easily affected by human factors such as subjective experience and eye fatigue, which can result in misjudgments or missed detection (Dong et al., 2020;Pratt et al., 1998). Surface defect detection has seen a growing application of deep learning methods in recent years (Kim et al., 2020;Lu & Su, 2021). However, current deep learning methods have a major disadvantage-namely, large numbers of both normal and defect samples are typically needed to train models, and marking defect samples is time consuming and laborious. ...
This paper proposes an unsupervised method for automatically labelling and obtaining image masks in defect detection. Since it is very labour intensive to acquire the image masks needed for deep learning (e.g. in semantic segmentation tasks) via manual labelling, we propose a method that utilize a generative adversarial network to obtain image masks automatically. Using this method, it is only necessary to input a considerable defect-free image to train. Then the proposed method can generate defect-free image samples like the input defect images, and the defect's location can be determined by comparing the input sample image containing defects with the generated sample image, thereby obtaining the input image mask. Our proposed method has been validated through experimental results, demonstrating its effectiveness. In addition to automatically and obtaining the required masks, our method achieved greater detection accuracy using the deep learning model Mask R-CNN compared with the manual labelling supervised method and a semi-supervised method.
... In recent years, deep learning methods have gained prominence and have gradually been employed for LCD defect detection [6][7][8]. However, the drawback of current deep learning methods is their reliance on a significant number of positive and negative samples for model training. ...
When applying deep learning methods to detect micro defects on low-contrast LCD surfaces, there are challenges related to imbalances in sample datasets and the complexity and laboriousness of annotating and acquiring target image masks. In order to solve these problems, a method based on sample and mask auto-generation for deep generative network models is proposed. We first generate an augmented dataset of negative samples using a generative adversarial network (GAN), and then highlight the defect regions in these samples using the training method constructed by the GAN to automatically generate masks for the defect images. Experimental results demonstrate the effectiveness of our proposed method, as it can simultaneously generate liquid crystal image samples and their corresponding image masks. Through a comparative experiment on the deep learning method Mask R-CNN, we demonstrate that the automatically obtained image masks have high detection accuracy.
... In recent years, deep learning methods have gained prominence and have gradually been employed for LCD defect detection [6][7][8]. However, the drawback of current deep learning methods is their reliance on a significant number of positive and negative samples for model training. ...
When applying deep learning methods to detect micro defects on low-contrast LCD surfaces, there are challenges related to the imbalance in samples dataset, as well as the complexity and laboriousness of annotating and acquiring target image masks. In order to solve these problems, a method based on sample and mask auto-generation for deep generative network models is proposed. We first generate an augmented dataset of negative samples using a generative adversarial network(GAN), and then highlight the defect regions in these samples using the training method constructed by the GAN to generate masks for the defect images automatically. Experimental results shows the effectiveness of our proposed method, as it allows for the simultaneous generation of LCD image samples and their corresponding image masks. Through a comparative experiment on the deep learning method Mask R-CNN, we demonstrate that the automatically obtained image masks have high detection accuracy.
... A CNN-based inspection system was proposed in Nguyen et al. (2021) to achieve defect classification in casting products, but the CNN deep learning model performed well only with a large volume of high-quality data. In Kim et al. (2020), authors proposed an indicator to differentiate between defects and the background area for the classification of defect types in thin-film-transistor liquid-crystal display panels. For industrial production processes, automatic defect classification was performed based on a CNN. ...
... In recent years, deep learning methods have gained prominence and have gradually been employed for LCD defect detection [6][7][8]. However, the drawback of current deep learning methods is their reliance on a significant number of positive and negative samples for model training. ...
When applying deep learning methods to detect micro defects on low-contrast LCD surfaces, there are challenges related to the imbalance in samples dataset, as well as the complexity and laboriousness of annotating and acquiring target image masks. In order to solve these problems, a method based on sample and mask auto-generation for deep generative network models is proposed. We first generate an augmented dataset of negative samples using a generative adversarial network(GAN), and then highlight the defect regions in these samples using the training method constructed by the GAN to generate masks for the defect images automatically. Experimental results shows the effectiveness of our proposed method, as it allows for the simultaneous generation of LCD image samples and their corresponding image masks. Through a comparative experiment on the deep learning method Mask R-CNN, we demonstrate that the automatically obtained image masks have high detection accuracy.
... Thin film transistor liquid crystal display (TFT-LCD) is a device that uses thin film transistor technology to display images. It is widely used in smartphones, televisions, and various vision displays (Kim et al., 2020;Lee & Chien, 2022;Xia et al., 2022;Yue et al., 2023). ...
The inspection of conductive particles after Anisotropic Conductive Film bonding is a crucial step in TFT-LCD manufacturing for quality assurance. Manual inspection under microscope is labor-intensive, time-consuming and error prone. Automatic inspection methods have been proposed by researchers including deep learning methods. However, inspection results are case dependent and existing deep learning-based methods heavily rely on large training dataset which is not given in many real applications. This is because the data available for analysis is limited on the manufacturing lines. To take on this challenge, this paper proposes a novel deep learning method based on modified Mask R-CNN algorithm which performs pixel-level segmentation to detect conductive particles. Under the proposed method, training dataset is augmented by applying novel parametric space envelope technique through a label-preserving transformation. This helps address small sample size prediction problem as well as class imbalance issue within the training dataset. Experimental results show significant improvement over existing methods under real-world constraint of limited training data (i.e., 99.25% overall particle detection accuracy compared with ~ 90% from existing template matching based auto-inspection method). The developed method provides industries an intelligent way to inspect conductive particle in TFT-LCD manufacturing.
... The stacking ensemble model is designed to obtain better results by using previously trained subnets together. It is possible to combine different models with the stacking ensemble, as well as combine the same model for different inputs [28]. In this study, an ensemble model was created by combining the model results obtained because of filtering with two different kernel values used in the preprocessing stage. ...
Traffic sign recognition techniques aim to reduce the probability of traffic accidents by increasing road and vehicle safety. These systems play an essential role in the development of autonomous vehicles. Autonomous driving is a popular field that is seeing more and more growth. In this study, a new high-performance and robust deep convolutional neural network model is proposed for traffic sign recognition. The stacking ensemble model is presented by combining the trained models by applying improvement methods on the input images. For this, first of all, by performing preprocessing on the data set, more accurate recognition was achieved by preventing adverse weather conditions and shooting errors. In addition, data augmentation was applied to increase the images in the data set due to the uneven distribution of the number of images belonging to the classes. During the model training, the learning rate was adjusted to prevent overfitting. Then, a new stacking ensemble model was created by combining the models trained with the input images that were subjected to different preprocessing. This ensemble model obtained 99.75% test accuracy on the German Traffic Sign Recognition Benchmark (GTSRB) dataset. When compared with other studies in this field in the literature, it is seen that recognition is performed with higher accuracy than these studies. Additively different approaches have been applied for model evaluation. Grad-CAM (gradient weighted class activation mapping) was used to make the model explainable. Evidential deep learning approach was applied to measure the uncertainty in classification. Results for safe monitoring are also shared with SafeML-II, which is based on measuring statistical distances. In addition to these, the migration test is applied with BTSC (Belgium Traffic Sign Classification) dataset to test the robustness of the model. With the transfer learning method of the models trained with GTSRB, the parameter weights in the feature extraction stage are preserved, and the training is carried out for the classification stage. Accordingly, with the stacking ensemble model obtained by combining the models trained with transfer learning, a high accuracy of 99.33% is achieved on the BTSC dataset. While the number of parameters the single model is 7.15 M, the number of parameters of the stacking ensemble model with additional layers is 14.34 M. However, the parameters of the models trained on a single preprocessed dataset were not trained, and transfer learning was performed. Thus, the number of trainable parameters in the ensemble model is only 39,643.
... A CNN-based inspection system was proposed in [49] to achieve defect classification in casting products, but the CNN deep learning model performed well only with a large volume of high-quality data. In [25], authors proposed an indicator to differentiate between defects and the background area for the classification of defect types in thin-film-transistor liquid-crystal display panels. For industrial production processes, automatic defect classification was performed based on a CNN. ...
Visual quality inspection in high performance manufacturing can benefit from automation, due to cost savings and improved rigor. Deep learning techniques are the current state of the art for generic computer vision tasks like classification and object detection. Manufacturing data can pose a challenge for deep learning because data is highly repetitive and there are few images of defects or deviations to learn from. Deep learning models trained with such data can be fragile and sensitive to context, and can under-detect new defects not found in the training data. In this work, we explore training defect detection models to learn specific defects out of context, so that they are more likely to be detected in new situations. We demonstrate how models trained on diverse images containing a common defect type can pick defects out in new circumstances. Such generic models could be more robust to new defects not found data collected for training, and can reduce data collection impediments to implementing visual inspection on production lines. Additionally, we demonstrate that object detection models trained to predict a label and bounding box outperform classifiers that predict a label only on held out test data typical of manufacturing inspection tasks. Finally, we studied the factors that affect generalization in order to train models that work under a wider range of conditions.
... Dong et al. (2020) proposed pyramid feature fusion and a global context attention network to achieve defect segmentation. Kim et al. (2020) used histogram equalization and binarization to extract defect features and combined them with stacking ensemble technology to improve the defect classification performance of the network. Yang et al. (2020) proposed an adversarial network based on unsupervised anomalous feature editing to detect various texture defects. ...
The mura defects of thin film transistor-liquid crystal display (TFT-LCD) panels have low contrast and random locations, which makes it impossible for us to correctly evaluate the number and type of mura defects on the image in the field inspection. In response to the above problems, this paper proposes a lightweight YOLO-ADPAM detection method based on an attention mechanism. First, we designed a K-means-ciou++ clustering algorithm using the Complete-Intersection-Over-Union loss function to cluster the anchor box size of the display defect dataset, making the bounding box regression more accurate and stable and improving the recognition and positioning accuracy of the algorithm. Second, we design a parallel attention module, combining the advantages of the channel and spatial attention mechanisms to effectively extract helpful information from feature maps. The channel attention branch can compensate for the defect information lost by global average pooling to a certain extent, and selecting a larger convolution kernel in the spatial attention branch is beneficial to retain crucial spatial information. Third, using atrous spatial pyramid pooling and depthwise separable convolution in the Neck network can further improve the receptive field of the feature map and improve the detection accuracy of the network. The experimental results show that the mAP of our proposed YOLO-ADPAM algorithm in TFT-LCD defect detection reaches 98.20%, and the detection speed reaches 83.23 FPS, which meets the detection accuracy and real-time requirements of TFT-LCD defect detection tasks.
