Figure 2 - uploaded by Min H. Kim
Content may be subject to copyright.
Image (a) shows a schematic diagram of compressive sensing. Image (b) represents the optical paths from the coded aperture (right) to the detector (left). Inset (b): a snapshot of the coded aperture with a monochromatic light (bandwidth: 10nm from 560-570nm). Image (c) shows a photograph of our 2D imaging unit.
Contexts in source publication
Context 1
... spectroscopy uses a diffraction grating (or a prism) and a solid-state array to measure spectrum dispersed by the diffrac- tion grating (as shown in Fig. 2). Dispersion-based imaging spec- troscopy was introduced to measure a spatially-varying spectral pattern on a 2D surface . Imaging spectroscopy dis- perses spatially-varying radiance through a coded-aperture ...
Context 2
... to resolve spatio-spectral information for sampling. We then solve the under- determined system by solving sparsity-constrained optimization problems. Here we briefly describe our mathematical model that follows the single disperser design [Wagadarikar et al. 2009], to help understand the basic operations implemented by the optical elements. See Fig. 2 for an ...
Context 3
... square. A Newport X-Y piezo translation stage modulates the aperture by 160µm travel per axis or 21 pixels on the detector. The aperture code is 1:1 mapped onto the Imperx (w/o microlenses) 2048⇥2048 pixel, 15.15mm square, monochromatic detector (7.4µm pixel pitch). The internal optical design includes relay lenses and a double Amici prism (See Fig. 2(b)). The prism is made of fused silica (FS) and calcium fluoride (CaF 2 ). The field lenses are made of FS. The Cooke triplets are made of CaF 2 and BK7. A Coastal Optics 60mm f/4 UV-VIS-IR lens that is apochromatic from approximately 315nm to 1.1µm is mounted to our imager. Three bandpass filters, placed in front of the objective lens, ...
Context 4
... to evaluate SFRs of our 3DIS system and two reference imaging systems: the Nikon and the QSI cameras ( Fig. 7(f)). The horizontal and vertical SFR of our hyperspectral imaging system rivals that of a commodity RGB camera, sufficient for high-resolution hyperspec- tral 3D measurements of small-sized objects (Nazca cup: 9.76cm tall, shown in Fig. 12). The horizontal SFR (along the spectral dispersion axis) exceeds the vertical SFR due to the image recon- struction algorithm described in Sec. 3.1. Note that the commodity RGB camera captures the highest spatial ...
Context 5
... Others have explored extending these functions into 3D [Brusco et al. 2006]. Our 3DIS system is capable of capturing hyperspectral datasets of complete three-dimensional cultural objects (polychromal sculpture, ceramics, etc) to assist archaeologists, art historians, and conser- vators in their study and preservation of objects of cultural value. Fig. 12 demonstrates 3D renderings of an excavated and restored Nazca cup (in Nazca valley, Peru in 100 B.C.-600 A.D.) evidencing two repaired regions near its ...
Similar publications
代工、品牌、標準、還是帶領需求 蘋果iPhone率先採用金屬機殼 工業4.0到底是不是萬靈丹 程天縱說：不是所有產品、數量都用工業4.0生產，它並非萬能丹，應該要看產品性質及數量來決定 巨型工廠將消失 台灣適合的是小型產線 3D列表機崛起：個人化生產時代來臨 供應鏈優勢還是智慧生產 思考下一個大產品（Next Big Product）是什麼? 創造核心價值、不斷創新
Citations
... Hyperspectral imaging (HSI) captures a number of narrow spectral images within a continuous spectral range, and constructs a three-dimensional (3D) spatio-spectral data cube of the target scene. HSIs provide much richer and more detailed spectral information than RGB images, and have been widely used in object detection [1], image recognition [2], remote sensing [3] and other fields. Conventional HSI systems obtain the 3D spatio-spectral data cube by scanning the target scene along the spectral or spatial directions, which is not suitable to capture the dynamic scenes [4]- [6]. ...
p>Coded aperture snapshot spectral imager (CASSI) can recover three-dimensional hyperspectral images (HSIs) from two-dimensional compressive measurements. Recently, deep unfolding approaches were shown impressive reconstruction performance among various algorithms. Existing deep unfolding methods usually employ linear projection methods to guide the iterative learning process. However, the linear projections do not include trainable parameters and ignore the essential characteristics of HSI. This paper proposes a novel learning-based deep estimation-enhancement unfolding (DEEU) framework to improve the HSI reconstruction. The deep estimation-enhancement (DEE) module is used to guide the iterative learning process of the network based on the prior information of the CASSI system, and then exploits the intrinsic features of the reconstructed HSI in both spectral and spatial dimensions. In addition, a multi-prior ensemble learning module is proposed to further improve the reconstruction performance without increasing runtime. As with most of deep unfolding methods, we plug a convolutional neural network as a denoiser in each stage of the DEEU framework, which finally forms the proposed DEEU-Net. Comprehensive experiments on both simulation and real datasets demonstrate that the effectiveness of our DEEU framework, and our DEEU-Net can achieve both high reconstruction quality and speed, outperforming the state-of-the-art methods.</p
... Hyperspectral imaging (HSI) captures a number of narrow spectral images within a continuous spectral range, and constructs a three-dimensional (3D) spatio-spectral data cube of the target scene. HSIs provide much richer and more detailed spectral information than RGB images, and have been widely used in object detection [1], image recognition [2], remote sensing [3] and other fields. Conventional HSI systems obtain the 3D spatio-spectral data cube by scanning the target scene along the spectral or spatial directions, which is not suitable to capture the dynamic scenes [4]- [6]. ...
p>Coded aperture snapshot spectral imager (CASSI) can recover three-dimensional hyperspectral images (HSIs) from two-dimensional compressive measurements. Recently, deep unfolding approaches were shown impressive reconstruction performance among various algorithms. Existing deep unfolding methods usually employ linear projection methods to guide the iterative learning process. However, the linear projections do not include trainable parameters and ignore the essential characteristics of HSI. This paper proposes a novel learning-based deep estimation-enhancement unfolding (DEEU) framework to improve the HSI reconstruction. The deep estimation-enhancement (DEE) module is used to guide the iterative learning process of the network based on the prior information of the CASSI system, and then exploits the intrinsic features of the reconstructed HSI in both spectral and spatial dimensions. In addition, a multi-prior ensemble learning module is proposed to further improve the reconstruction performance without increasing runtime. As with most of deep unfolding methods, we plug a convolutional neural network as a denoiser in each stage of the DEEU framework, which finally forms the proposed DEEU-Net. Comprehensive experiments on both simulation and real datasets demonstrate that the effectiveness of our DEEU framework, and our DEEU-Net can achieve both high reconstruction quality and speed, outperforming the state-of-the-art methods.</p
... Some systems have also been proposed for spectral 3D acquisition [19,21,27,29,39,49]. However, they rely on a dedicated setup using a multispectral camera [27,39,49] or a multispectral light source [19,21,29], which makes the system impractical and expensive for most users. ...
... Some systems have also been proposed for spectral 3D acquisition [19,21,27,29,39,49]. However, they rely on a dedicated setup using a multispectral camera [27,39,49] or a multispectral light source [19,21,29], which makes the system impractical and expensive for most users. ...
... Spectral 3D acquisition systems: Existing systems for spectral 3D acquisition are roughly classified into photometric stereo-based [29,39], SfM and multi-view stereobased [21,49], and active lighting or scanner-based [19,27] systems. The photometric stereo-based systems [29,39] can acquire dense surface normals for every pixels of the single-view image. ...
In this paper, we propose a novel projector-camera system for practical and low-cost acquisition of a dense object 3D model with the spectral reflectance property. In our system, we use a standard RGB camera and leverage an off-the-shelf projector as active illumination for both the 3D reconstruction and the spectral reflectance estimation. We first reconstruct the 3D points while estimating the poses of the camera and the projector, which are alternately moved around the object, by combining multi-view structured light and structure-from-motion (SfM) techniques. We then exploit the projector for multispectral imaging and estimate the spectral reflectance of each 3D point based on a novel spectral reflectance estimation model considering the geometric relationship between the reconstructed 3D points and the estimated projector positions. Experimental results on several real objects demonstrate that our system can precisely acquire a dense 3D model with the full spectral reflectance property using off-the-shelf devices.
... Based on geometrical optics, various hyperspectral imaging systems have been developed for snapshot imaging of dynamic objects and include various optical elements: a dispersive optical element (prism or diffraction grating), a coded aperture mask, several relay lenses, and an objective imaging lens. The dimensions of a typical compressive hyperspectral imager are larger than those of a conventional camera; for instance, its length is greater than a meter [Kim et al. 2012a; Lee and Kim 2014;Lin et al. 2014]. Actual imaging applications are limited to laboratory environments. ...
... The grating splits the collimated incident light into diffraction patterns in different directions while sacrificing the spatial resolution for computed tomography. Coded aperture snapshot spectral imaging (CASSI) [Gehm et al. 2007;Jeon et al. 2016;Kim et al. 2012a;Wagadarikar et al. 2008] was introduced for capturing dynamic objects. A dispersive optical element is coupled with a coded aperture through relay lenses to encode spectral or spatial-spectral signatures. ...
... Different from conventional RGB cameras, snapshot spectral imagers capture compressed signals of dense spectral samples, which need to be reconstructed by a post process. Since hyperspectral reconstruction is a severely ill-posed problem (inferring dense spectral information from a monochromatic, encoded image), several optimization approaches have been proposed by defining a data fidelity term and specific image priors, such as a total variation (TV) l 1 -norm regularization [Jeon et al. 2016;Kim et al. 2012a;Kittle et al. 2010] or pretrained dictionary [Lin et al. 2014]. A common characteristic of these approaches is the tradeoff between spatial resolution and spectral accuracy in the reconstructed results. ...
Traditional snapshot hyperspectral imaging systems include various optical elements: a dispersive optical element (prism), a coded aperture, several relay lenses, and an imaging lens, resulting in an impractically large form factor. We seek an alternative, minimal form factor of snapshot spectral imaging based on recent advances in diffractive optical technology. We thereupon present a compact, diffraction-based snapshot hyperspectral imaging method, using only a novel diffractive optical element (DOE) in front of a conventional, bare image sensor. Our diffractive imaging method replaces the common optical elements in hyperspectral imaging with a single optical element. To this end, we tackle two main challenges: First, the traditional diffractive lenses are not suitable for color imaging under incoherent illumination due to severe chromatic aberration because the size of the point spread function (PSF) changes depending on the wavelength. By leveraging this wavelength-dependent property alternatively for hyperspectral imaging, we introduce a novel DOE design that generates an anisotropic shape of the spectrally-varying PSF. The PSF size remains virtually unchanged, but instead the PSF shape rotates as the wavelength of light changes. Second, since there is no dispersive element and no coded aperture mask, the ill-posedness of spectral reconstruction increases significantly. Thus, we propose an end-to-end network solution based on the unrolled architecture of an optimization procedure with a spatial-spectral prior, specifically designed for deconvolution-based spectral reconstruction. Finally, we demonstrate hyperspectral imaging with a fabricated DOE attached to a conventional DSLR sensor. Results show that our method compares well with other state-of-the-art hyperspectral imaging methods in terms of spectral accuracy and spatial resolution, while our compact, diffraction-based spectral imaging method uses only a single optical element on a bare image sensor.
... While hyperspectral imaging has also found application in computer vision tasks [14,25,31], its widespread adoption has been hindered due to inherent challenges in acquisition them. Capturing a HSI requires sampling of a very high dimensional signal; for example, mega-pixel images at hundreds of spectral bands, a process that is daunting to do at video rate. ...
Many materials have distinct spectral profiles. This facilitates estimation of the material composition of a scene at each pixel by first acquiring its hyperspectral image, and subsequently filtering it using a bank of spectral profiles. This process is inherently wasteful since only a set of linear projections of the acquired measurements contribute to the classification task. We propose a novel programmable camera that is capable of producing images of a scene with an arbitrary spectral filter. We use this camera to optically implement the spectral filtering of the scene's hyperspectral image with the bank of spectral profiles needed to perform per-pixel material classification. This provides gains both in terms of acquisition speed --- since only the relevant measurements are acquired --- and in signal-to-noise ratio --- since we invariably avoid narrowband filters that are light inefficient. Given training data, we use a range of classical and modern techniques including SVMs and neural networks to identify the bank of spectral profiles that facilitate material classification. We verify the method in simulations on standard datasets as well as real data using a lab prototype of the camera.
... HSI is already an established tool in a widespread area of applications, such as agriculture [1,2], food quality control [3,4], remote sensing [5,6], surveillance and astronomy [7] as well as biomedical imaging [8]. ...
In this paper we present results from process observations of deep penetration laser welding using a hyperspectral imaging (HSI) technique. For monitoring different types of laser-based material processing, we developed an appropriate high-speed camera-based HSI system. This system enables us to derive spectra of the deep penetration welding process with high time resolution. These spectra can be used for temperature determination or to extract spectral characteristics, both of which can help to further investigate process dynamics and properties of the vapor. The presented results contain particularly HSI-derived spectral information of the evolving vapor plume that is related to corresponding high-speed images and spectra acquired with a conventional VIS-spectrometer.
... With the rapid development of spectrometry and its comprehensive wide utilities, field spectral imaging [1][2][3][4][5], aiming at daily life vision applications, has been more and more affordable, so as to be used for classification [6,7], recognition [8], and tracking [9] by taking advantage of richer color channels than traditional trichromatic imaging. Unfortunately, different from remote sensing, the complicated nature of field spectral imaging, e.g., scene geometry (shading), inter-reflections, and complicated artificial illumination, makes the further utility of field spectral imaging a very challenging problem. ...
In this paper, we present a spectral intrinsic image decomposition (SIID) model, which is dedicated to resolve a natural scene into its purely independent intrinsic components: illumination, shading, and reflectance. By introducing spectral information, our work can solve many challenging cases, such as scenes with metameric effects, which are hard to tackle for trichromatic intrinsic image decomposition (IID), and thus offers potential benefits to many higher-level vision tasks, e.g., materials classification and recognition, shape-from-shading, and spectral image relighting. A both effective and efficient algorithm is presented to decompose a spectral image into its independent intrinsic components. To facilitate future SIID research, we present a public dataset with ground-truth illumination, shading, reflectance and specularity, and a meaningful error metric, so that the quantitative comparison becomes achievable. The experiments on this dataset and other images demonstrate the accuracy and robustness of the proposed method on diverse scenes, and reveal that more spectral channels indeed facilitate the vision task (i.e., segmentation and recognition).
Spectral imaging has revolutionisedvarious fields by capturing detailed spatial and spectral information. However, its high cost and complexity limit the acquisition of a large amount of data to generalise processes and methods, thus limiting widespread adoption. To overcome this issue, a body of the literature investigates how to reconstruct spectral information from RGB images, with recent methods reaching a fairly low error of reconstruction, as demonstrated in the recent literature. This article explores the modification of information in the case of RGB-to-spectral reconstruction beyond reconstruction metrics, with a focus on assessing the accuracy of the reconstruction process and its ability to replicate full spectral information. In addition to this, we conduct a colorimetric relighting analysis based on the reconstructed spectra. We investigate the information representation by principal component analysis and demonstrate that, while the reconstruction error of the state-of-the-art reconstruction method is low, the nature of the reconstructed information is different. While it appears that the use in colour imaging comes with very good performance to handle illumination, the distribution of information difference between the measured and estimated spectra suggests that caution should be exercised before generalising the use of this approach.
