Figure 8 - available via license: CC BY
Content may be subject to copyright.
Rectified images of the left and right camera (brightness and contrast adjusted for better visibility).
Source publication
Pattern projection-based 3D measurement systems are widely used for contactless, non-destructive optical 3D shape measurements. In addition, many robot-operated automation tasks require real-time reconstruction of accurate 3D data. In previous works, we have demonstrated 3D scanning based on statistical pattern projection-aided stereo matching betw...
Context in source publication
Context 1
... recorded a data set of 400 image pairs at a resolution of 1 Megapixel per image while the projector was running. Each image pair contains a different aperiodic sinusoidal stripe pattern (Figure 8). We rectified the images and worked exclusively with this rectified version throughout the survey. ...
Similar publications
The paper presents an analysis of reluctance actuator for speckle reducing in laser based projectors, head-up displays, metrology and 3D scanning units to improve the quality of the generated images by a moving diffuser. Design concepts for one-dimensional laser speckle reducer actuator are analyzed through different designs structures. There are t...
Citations
... It is composed of a real-time NIR 3D sensor consisting of a NIR GOBO projector at 850 nm (Heist et al., 2018) and two NIR high-speed cameras at the same wavelength. This 3D sensor can reconstruct and transmit 3D images with very low latency at a framerate of 15 Hz using the stereo matching acceleration algorithm BICOS (Dietrich et al., 2019) and the graphics processing unit NVIDIA RTX 2080. The real-time 3D data enable the analysis of patients' head motion. ...
Obstructive sleep apnea (OSA) is a common sleep-related breathing disorder characterized by the collapse of the upper airway and associated with various diseases. For clinical diagnosis, a patient’s sleep is recorded during the night via polysomnography (PSG) and evaluated the next day regarding nocturnal respiratory events. The most prevalent events include obstructive apneas and hypopneas. In this paper, we introduce a fully automatic contactless optical method for the detection of nocturnal respiratory events. The goal of this study is to demonstrate how nocturnal respiratory events, such as apneas and hypopneas, can be autonomously detected through the analysis of multi-spectral image data. This represents the first step towards a fully automatic and contactless diagnosis of OSA. We conducted a trial patient study in a sleep laboratory and evaluated our results in comparison with PSG, the gold standard in sleep diagnostics. In a study sample with three patients, 24 hours of recor ded video materials and 245 respiratory events, we have achieved a classification accuracy of 82 % with a random forest classifier.
... The GOBO projector and the stereo-vision camera setup contain band-pass optical filters at 850 nm with a FWHM of 50 nm, so there is no crosstalk between projected GOBO patterns and the NIR cameras at 750 nm and 950 nm. Using a GOBO pattern set with a length of N = 10, a 3D image can be reconstructed using the correspondence search method of Dietrich (27). The 2D frame rate of the cameras of the stereo-vision setup was set to 300 Hz, so the stereo-vision setup can deliver 3D image data at a frame rate of 30 Hz. ...
Newborns and preterm infants require accurate and continuous monitoring of their vital parameters. Contact-based methods of monitoring have several disadvantages, thus, contactless systems have increasingly attracted the neonatal communities' attention. Camera-based photoplethysmography is an emerging method of contactless heart rate monitoring. We conducted a pilot study in 42 healthy newborn and near-term preterm infants for assessing the feasibility and accuracy of a multimodal 3D camera system on heart rates (HR) in beats per min (bpm) compared to conventional pulse oximetry. Simultaneously, we compared the accuracy of 2D and 3D vision on HR measurements. The mean difference in HR between pulse oximetry and 2D-technique added up to + 3.0 bpm [CI−3.7 – 9.7; p = 0.359, limits of agreement (LOA) ± 36.6]. In contrast, 3D-technique represented a mean difference in HR of + 8.6 bpm (CI 2.0–14.9; p = 0.010, LOA ± 44.7) compared to pulse oximetry HR. Both, intra- and interindividual variance of patient characteristics could be eliminated as a source for the results and the measuring accuracy achieved. Additionally, we proved the feasibility of this emerging method. Camera-based photoplethysmography seems to be a promising approach for HR measurement of newborns with adequate precision; however, further research is warranted.
... The drawback of aperiodic sinusoidal fringe projection systems is the increased depth reconstruction error for the same number of temporal measurements, compared to periodic fringe projection methods [5]. Another less commonly addressed issue is the computational burden associated with the algorithms used for disparity calculation, as highlighted in [6]. Compared to algorithms used in periodic fringe projection [1], the computational requirements of temporal pixel-to-pixel correlation limit the portability of such approaches to high-end computing devices. ...
... The scope of this paper is limited to aperiodic sinusoidal temporal pattern projection. Currently, some publications are evaluating the use of such methods [4][5][6]8]. However, the main subject of these publications mostly stems from the field of optics. ...
... However, the main subject of these publications mostly stems from the field of optics. The closest research work that addresses the issue of computational complexity in temporal statistical pattern stereo was published by Dietrich et al. [6]. The main contribution of this work was the incorporation of an extended census transform over the temporal axis and the use of the Hamming distance as the matching cost metric when calculating disparity. ...
The presented paper describes a hardware-accelerated field programmable gate array (FPGA)–based solution capable of real-time stereo matching for temporal statistical pattern projector systems. Modern 3D measurement systems have seen an increased use of temporal statistical pattern projectors as their active illumination source. The use of temporal statistical patterns in stereo vision systems includes the advantage of not requiring information about pattern characteristics, enabling a simplified projector design. Stereo-matching algorithms used in such systems rely on the locally unique temporal changes in brightness to establish a pixel correspondence between the stereo image pair. Finding the temporal correspondence between individual pixels in temporal image pairs is computationally expensive, requiring GPU-based solutions to achieve real-time calculation. By leveraging a high-level synthesis approach, matching cost simplification, and FPGA-specific design optimizations, an energy-efficient, high throughput stereo-matching solution was developed. The design is capable of calculating disparity images on a 1024 × 1024(@291 FPS) input image pair stream at 8.1 W on an embedded FPGA platform (ZC706). Several different design configurations were tested, evaluating device utilization, throughput, power consumption, and performance-per-watt. The average performance-per-watt of the FPGA solution was two times higher than in a GPU-based solution.
... [21][22] These high-speed cameras and GOBO projector work at 850 nm and are thus irritation-free. Using a pattern set with a length N = 10, a 3D point cloud can be reconstructed using the stereo image correlation method in Ref. 23 and the BICOS algorithm 24 for acceleration. With the support of graphics processing unit (GPU) a 3D frame rate up to 30 Hz can be realized. ...
... We have developed a two-step thermal technique [12][13][14][15][16] based on the scanning from heating approach in combination with a corresponding pixel search by temporal cross-correlation [17][18][19][20][21][22]: (1) projection of areal aperiodic sinusoidal multifringe thermal patterns onto the measurement object and (2) stereo recording of the thermal radiation that is absorbed and re-emitted by the object surface. ...
Three-dimensional (3D) shape measurement systems based on diffuse reflection of projected structured light do not deliver reliable data when measuring glossy, transparent, absorbent, or translucent objects. In recent years, we have developed a method based on stereo recording with infrared cameras and projection of areal aperiodic sinusoidal thermal patterns to detect such objects. However, the measurements took longer than 10 s, up to minutes; moreover, the measurement accuracy was improvable. Now, we have succeeded in both drastically reducing measurement time and significantly increasing measurement quality. This finally provides a technique for reliably measuring transparent objects, e.g., in series production. We demonstrate measurement examples achieved within 1 s and with 3D standard deviations less than 10 µm.
... This 3D sensor works at 850 nm and is thus irritation-free. Using a set of 10 projected light patterns, a 3D point cloud can be calculated using the stereo image correlation method in [23] with the acceleration algorithm in [24]. With the use of a graphics processing unit (GPU), a 3D frame rate up to 30 Hz can be realized, giving a real-time 3D video. ...
The contactless estimation of vital signs using conventional color cameras and ambient light can be affected by motion artifacts and changes in ambient light. On both these problems, a multimodal 3D imaging system with an irritation-free controlled illumination was developed in this work. In this system, real-time 3D imaging was combined with multispectral and thermal imaging. Based on 3D image data, an efficient method was developed for the compensation of head motions, and novel approaches based on the use of 3D regions of interest were proposed for the estimation of various vital signs from multispectral and thermal video data. The developed imaging system and algorithms were demonstrated with test subjects, delivering a proof-of-concept.
... In the first step of the 3D reconstruction, corresponding pixels in both cameras that originate from the same object point have to be found. The corresponding pixels are detected by the maximum of the normalized cross-correlation function 13,16,[21][22][23] between N temporal gray values of the camera pixels of both cameras: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 4 ; 1 1 6 ; 2 6 9 ρ ¼ ...
... For the computationally intensive steps 2-4 (see section 1.2), an analysis regarding their feasibility for FPGA implementation was conducted. Specifically, the coarse and fine point-correspondence search are computationally intensive [10] . In these steps, based on multiple images over time from both cameras, with different projected patterns, points with similar gray value stacks are identified [11] . ...
... Finally, a 3D point cloud can be calculated from matched image point pairs with the use of the camera parameters of the stereo-vison camera setup. Using an accelerated algorithm for the coarse correspondence search 17 and graphics processing unit (GPU), a 3D frame rate of 30 Hz can be achieved. What's more, the pattern projection occurs in the invisible NIR range and is irritation-free, so this 3D sensor can be immediately used for human-machine interaction. ...
