Figure 1 - uploaded by Du-Ming Tsai
Content may be subject to copyright.
Source publication
Purpose
The purpose of this paper is to develop a robot vision system for surface defect detection of 3D objects. It aims at the ill‐defined qualitative items such as stains and scratches.
Design/methodology/approach
A robot vision system for surface defect detection may counter: high surface reflection at some viewing angles; and no reference mar...
Contexts in source publication
Context 1
... image taken by the camera is 1,600 £ 1,200 pixels in size with 8-bit gray levels. Figure 1 shows the configuration of the robot vision system implemented in this study. ...
Context 2
... cover of the phone is made of white leather with two black stripes on it. Figure 5(b1) is the same rear surface of the cellular phone taken at an inclined angle of 858. Figure 5(a2) and (b2) is the corresponding reflection images, and Figure 5(a3) and (b3) is the respective reflection regions detected with a control constant K ¼ 1.5. The detection result in Figure 5(a3) shows that the proposed reflection-detection method will not classify the pure white region of a surface as reflection. ...
Context 3
... practical implementation, the control constant C can be learned from a set of defect-free sample images by setting the parameter to the minimum value that results in no false alarms for all the test samples. Figure 10 shows an electrical adapter used to evaluate the effect of changes in the value of control constant C, where image (a) is the template image, and images (b1) and (c1) are defect-free and detective test samples at the same viewing angle. Figure 10(b2)-(b6) and (c2)-(c6) is, respectively, the detection results of defect-free sample (b1) and defective sample (c1) with parameter C varying from 0.5 to 3.0. ...
Context 4
... 10 shows an electrical adapter used to evaluate the effect of changes in the value of control constant C, where image (a) is the template image, and images (b1) and (c1) are defect-free and detective test samples at the same viewing angle. Figure 10(b2)-(b6) and (c2)-(c6) is, respectively, the detection results of defect-free sample (b1) and defective sample (c1) with parameter C varying from 0.5 to 3.0. The detection results show that a very small value of C ¼ 0.5 generates noisy points for the defect-free image, as shown in Figure 10(b2). ...
Context 5
... 10(b2)-(b6) and (c2)-(c6) is, respectively, the detection results of defect-free sample (b1) and defective sample (c1) with parameter C varying from 0.5 to 3.0. The detection results show that a very small value of C ¼ 0.5 generates noisy points for the defect-free image, as shown in Figure 10(b2). The detected defect size is gradually reduced as the parameter value of C increases. ...
Context 6
... detected defect size is gradually reduced as the parameter value of C increases. Because the defect size is generally very small with respect to the whole object size in the image, the control constant C can be set in the range between 1 and 2. The experimental results in Figure 10 also show that a C value in the range between 1 and 2 can produce good detection results. ...
Context 7
... the experiments on reflection detection, the same Gaussian filter of size 11 £ 11 with scale parameter s ¼ ffiffiffi 2 p was applied to all test samples. Figure 11 Effect of changes in the value of control constant K for reflection segmentation Effect of changes in the control constant C for defect segmentation ...
Context 8
... C = 3 (c6) C = 3 Notes: (a) Template image of an electrical adapter; (b1) defect-free test image; (c1) defective test image with a bold scratch; (b2)-(b6) detection results of defect-free image (b1) with varying C values; (c2)-(c6) detection results of defective image (c1) with varying C values of the reflection regions as binary images. All the reflection regions are reliably detected with a control constant K ¼ 1. Figure 13(a1)-(d1) further shows the sensed images of a white plastic mouse at viewing angles of 808, 888, 958 and 1008 (in a horizontal scanning direction of the image). The convex surface of the mouse presents a circular spark over a wide range of viewing angles. ...
Context 9
... convex surface of the mouse presents a circular spark over a wide range of viewing angles. The detection results in Figure 13(a2)-(d2) shows that the proposed method can also detect well the reflection on a curved surface. The detected reflection regions are superimposed on the original gray-level images, so that the location changes of the reflection regions at different viewing angles can be easily observed. ...
Context 10
... detected reflection regions are superimposed on the original gray-level images, so that the location changes of the reflection regions at different viewing angles can be easily observed. As seen in Figure 13(a3)-(d3), the detected reflection regions are significantly shifted from the left to the right in the image when the viewing angle is changed from 808 to 1008. The printed characters cannot be observed at viewing angle 888 (in image (b1)) when they are concealed by the reflection. ...
Context 11
... to the template-matching process, the whole inspection procedure from reflection detection, marker selection to image registration described previously is applied to all test samples discussed in this subsection. Figure 14 shows the defect detection results of the battery charger. The template image of the charger at the viewing angle of 578 is shown in Figure 14(a). ...
Context 12
... 14 shows the defect detection results of the battery charger. The template image of the charger at the viewing angle of 578 is shown in Figure 14(a). Figure 14(b1) is a defect-free test sample, and Figure 14(c1) is a defective sample with a thin scratch on the surface. ...
Context 13
... template image of the charger at the viewing angle of 578 is shown in Figure 14(a). Figure 14(b1) is a defect-free test sample, and Figure 14(c1) is a defective sample with a thin scratch on the surface. The value of the control constant C for the threshold of similarity measure is determined such that the defect-free test sample generates the least noise and the defect is well presented in the segmented image for individual comparative methods. ...
Context 14
... template image of the charger at the viewing angle of 578 is shown in Figure 14(a). Figure 14(b1) is a defect-free test sample, and Figure 14(c1) is a defective sample with a thin scratch on the surface. The value of the control constant C for the threshold of similarity measure is determined such that the defect-free test sample generates the least noise and the defect is well presented in the segmented image for individual comparative methods. ...
Context 15
... value of the control constant C for the threshold of similarity measure is determined such that the defect-free test sample generates the least noise and the defect is well presented in the segmented image for individual comparative methods. The binary images in Figure 14(b2)-(b4) are detection results of the defect-free test sample from the NCC method with C ¼ 2, the SAD method with C ¼ 3, and the optical-flow measure with C ¼ 1, respectively. The binary images in Figure 14(c2)-(c4) are the detected defect of the defective test sample from the three comparative methods. ...
Context 16
... binary images in Figure 14(b2)-(b4) are detection results of the defect-free test sample from the NCC method with C ¼ 2, the SAD method with C ¼ 3, and the optical-flow measure with C ¼ 1, respectively. The binary images in Figure 14(c2)-(c4) are the detected defect of the defective test sample from the three comparative methods. The detection results show that the optical-flow matching method can detect the thin scratch well, without presenting noise. ...
Context 17
... NCC method is also sensitive to object edges, and is computationally expensive with a large neighborhood window size. Figure 15 shows the defect detection results of the cellular phone at the viewing angle of 798. Figure 15(a) is the template image. Figure 15(b1) and (c1) shows a defect free-and a defective test sample, respectively. ...
Context 18
... 15 shows the defect detection results of the cellular phone at the viewing angle of 798. Figure 15(a) is the template image. Figure 15(b1) and (c1) shows a defect free-and a defective test sample, respectively. ...
Context 19
... 15 shows the defect detection results of the cellular phone at the viewing angle of 798. Figure 15(a) is the template image. Figure 15(b1) and (c1) shows a defect free-and a defective test sample, respectively. The resulting binary images in Figure 15(b2)-(b4) and (c2)-(c4) were obtained from NCC, SAD and optical-flow matching. ...
Context 20
... 15(b1) and (c1) shows a defect free-and a defective test sample, respectively. The resulting binary images in Figure 15(b2)-(b4) and (c2)-(c4) were obtained from NCC, SAD and optical-flow matching. Again, the optical-flow measure can reliably detect the small defect without showing any noise. ...
Context 21
... SAD method can also identify the defect with noisy points on the two vertical edges of the phone. Figure 16 further shows the defect detection results of the electrical adapter at the viewing angle of 908. Figure 16(a) is the template image. Figure 16(b1) is a defect-free sample, and Figure 16(c1) is a defective sample with a bold scratch. ...
Context 22
... 16 further shows the defect detection results of the electrical adapter at the viewing angle of 908. Figure 16(a) is the template image. Figure 16(b1) is a defect-free sample, and Figure 16(c1) is a defective sample with a bold scratch. ...
Context 23
... 16 further shows the defect detection results of the electrical adapter at the viewing angle of 908. Figure 16(a) is the template image. Figure 16(b1) is a defect-free sample, and Figure 16(c1) is a defective sample with a bold scratch. The detection results in Figure 16(b2)-(b4) and (c2)-(c4) from the three comparative methods also reveal that the optical-flow matching method can accurately detect the scratch defect. ...
Context 24
... 16 further shows the defect detection results of the electrical adapter at the viewing angle of 908. Figure 16(a) is the template image. Figure 16(b1) is a defect-free sample, and Figure 16(c1) is a defective sample with a bold scratch. The detection results in Figure 16(b2)-(b4) and (c2)-(c4) from the three comparative methods also reveal that the optical-flow matching method can accurately detect the scratch defect. ...
Context 25
... 16(b1) is a defect-free sample, and Figure 16(c1) is a defective sample with a bold scratch. The detection results in Figure 16(b2)-(b4) and (c2)-(c4) from the three comparative methods also reveal that the optical-flow matching method can accurately detect the scratch defect. ...
Citations
... The captured image that is mostly grayscale is imported to computer to be processed in order to extract desired information e.g., abnormalities. There are a lot of image processing techniques and this field has been studied comprehensively in the literatures not only for quality control but also for variety of applications such as medical image analyzing, robotic, construction engineering, etc. Tian et al. [23] for automatic surface defect detection of stamping grinding flat parts employed template matching method, in which the similarity between the defect-free template image and the scene image under test is compared [24]. According to [25], two general approaches, namely moving window and region of interest)ROI(based methods work well to detect specific defect types but they do not work as well to detect defects of different sizes and shapes; therefore Fadel et al. proposed a method to detect emergence of fault by subtracting a stream of images from the nominal image and then finding the ROI, in which the maximum generalized likelihood ratio as a statistic parameter is higher than a threshold. ...
Online inspection and surface characterization including 3D defect detection of surfaces using 3D scanning and
coordinate metrology data has crucial applications in today’s Industry 4.0. Although 3D vision-based metrology
methods are superior to 2D in providing spatial information, its processing remains challenge. A novel automated
Detailed Deviation Zone Evaluation method is proposed in this paper in which 3D unorganized PC is converted
into 2D grayscale image such that the intensity variation of image is proportional to the surface topography. This
developed image is addressed as “skin Image” of the scanned surface. The considered point clouds include only
XYZ-coordinates. No prior knowledge of defects, and no training set is required. The methodology is fully
implemented, and verified by inspecting point clouds of several workpieces with predefined defects. The
experimental results show high efficiency of the developed methodology in defect detection and 3D-feature
identification irrespective of shape and size.
... Stereoscopic vision [140][141][142][143][144] Suitable for areas with large texture variations and is very sensitive to normal surface disturbances ...
... The experimental results show that the running time of this method for a test image is shorter than that of other methods that fuse the 2D image and 3D depth information. Using robots to shoot from multiple perspectives and template matching technology to detect local defects of 3D objects [141,142] is also a general method. For example, Tsai et al. [141] first acquired images of each perspective of a defect-free object, stored them as comparison templates, compared each acquired image with the corresponding template image, and then used template matching technology to identify the local defects of two comparison images in each perspective. ...
... Using robots to shoot from multiple perspectives and template matching technology to detect local defects of 3D objects [141,142] is also a general method. For example, Tsai et al. [141] first acquired images of each perspective of a defect-free object, stored them as comparison templates, compared each acquired image with the corresponding template image, and then used template matching technology to identify the local defects of two comparison images in each perspective. For quickly calculating the large dataset of a 3D point cloud, Enzberg et al. [143] provided a reference surface to measure the measured surface with a model-based surface quality detection method, compared its 3D coordinate data with reference data, and accelerated the calculation of 3D point cloud data through dual-eigenvalue decomposition. ...
The computer-vision-based surface defect detection of metal planar materials is a research hotspot in the field of metallurgical industry. The high standard of planar surface quality in the metal manufacturing industry requires that the performance of an automated visual inspection system and its algorithms are constantly improved. This paper attempts to present a comprehensive survey on both two-dimensional and three-dimensional surface defect detection technologies based on reviewing over 160 publications for some typical metal planar material products of steel, aluminum, copper plates and strips. According to the algorithm properties as well as the image features, the existing two-dimensional methodologies are categorized into four groups: statistical, spectral, model, and machine learning-based methods. On the basis of three-dimensional data acquisition, the three-dimensional technologies are divided into stereoscopic vision, photometric stereo, laser scanner, and structured light measurement methods. These classical algorithms and emerging methods are introduced, analyzed, and compared in this review. Finally, the remaining challenges and future research trends of visual defect detection are discussed and forecasted at an abstract level.
... In the case of detecting defects of 3D objects, the device requirements and image acquisition methods are quite different from that of planar objects, which also leads to more difficulties in the detection process [36][37][38][39][40]. In practical applications such as defect detection of three-dimensional curved glass, an allin-focus image at varying distances in a scene cannot be captured. ...
... The image acquired will get blurred due to the height difference between upper and lower surfaces. Faced with such a detection problem, much attention has been focused on exploiting the multisensor method or the single sensor with multiposition method [38][39][40]. All these methods above are dedicated to obtaining a complete defect image of three dimensional objects prepared for subsequent defect identification. ...
We propose a multifocus image fusion method for achieving all-in-focus images of three-dimensional objects based on the combination of transform domain and spatial domain techniques. First, the source images are decomposed into low-frequency and high-frequency components by the discrete wavelet transform technique. Next, a correlation coefficient is employed to obtain the maximum similarity among low-frequency components. Then, in order not to interrupt the correlations among decomposition layers, the comparison among high-frequency components is executed by transforming them to spatial domain. In addition, a sliding window is used to evaluate the local saliency of the pixels more accurately. Finally, the fused image is synthesized from source images and the saliency map. The variance, entropy, spatial frequency, mutual information, edge intensity, and similarity measure ( Q A B / F ) are used as metrics to evaluate the sharpness of the fused image. Experimental results demonstrate that the fusion performance of the proposed method is enhanced compared with that of the other widely used techniques. In the application of three-dimensional surface optical detection, the proposed method is suitable for obtaining the complete image at varying distances in the same scene, so as to prepare for subsequent defect identification.
We have fabricated the Al/Al2O3/GeON/SiO2/Si charge-trapping flash memory with different annealing temperatures for the GeON charge storage layer. The physical structure of memory device was studied by transmission electron microscopy and the chemical composition of the GeON film was investigated by X-ray photoelectron spectroscopy. The proposed device that had been annealed at 600 °C exhibited a large 5.75-V initial memory window, a 3.77-V 10-year extrapolated retention window, a 6.08-V endurance window at 105 cycles under very fast (100-μs) and low-voltage (±16-V) program/erase. The excellent properties are due to charge traps with desirable energy levels generated by optimal annealing, indicating that GeON is a potential candidate as the charge storage layer for flash memory applications.
