Content uploaded by Jason T. Smith
Author content
All content in this area was uploaded by Jason T. Smith on Jun 04, 2022
Content may be subject to copyright.
Wide-field Diffuse Optical Tomography Using Deep Learning
Navid Ibtehaj Nizam†, *, Marien Ochoa†, Jason T. Smith, and Xavier Intes
Center for Modeling, Simulation and Imaging in Medicine, Rensselaer Polytechnic Institute, Troy, NY 12180, USA
†These authors contributed equally.
*nizamn@rpi.edu
Abstract: A modified AUTOMAP architecture, with micro-CT validation, is used for 3D
reconstructions in Diffuse Optical Tomography, employing wide-field illumination and detection.
The performance is compared with a regularized Least Squares technique.
1. Introduction
Diffuse Optical Tomography (DOT) enables monitoring of the physiological state of deep tissues with high
sensitivity. DOT performances depend on the spatial, spectral, and temporal density of the data acquired. Recently, a
hyperspectral wide-field time-domain single-pixel instrumental strategy has been proposed to collect dense data
along all these dimensions efficiently [1]. Still, the image formation component is challenging. Traditionally,
techniques like the least-squares (LSQ), conjugate gradient (CGS), and total-variation minimization (TVAL) have
been mainly deployed to solve the optical inverse problem. However, the major drawback of these techniques is that
there is no “one-size-fits-all” approach. Additionally, optimizing the parameters associated with these traditional
solvers is often an expert and cumbersome process. Because of the availability of high-powered GPUs and the
associated rise in computational power, there has been an increased interest in investigating the potential of Deep-
Learning (DL)-based approaches for 3D image formation [2]. Herein, we report on modified AUTOMAP (ModAM)
[3], a Convolutional Neural Network (CNN)-based architecture, that directly reconstructs the δμa contrast based on
single-pixel tomographic data sets [4].
2. Methods
For simplicity, in this report, we consider a Continuous-wave (CW) approach throughout. An in silico workflow is
used to train the network, which remains suitable for an in vitro experiment. Additionally, widefield illumination
and detection strategies are utilized leveraging sparse and low-frequency patterns (total of 36) as in [2] (shown in
Fig. 1(a)). We use the open-source Monte Carlo (MC) based software, MCX [5], to generate large volumes of in
silico optical phantoms. The homogenous embeddings in the phantoms are generated from binary characters
obtained from the EMNIST dataset (Fig. 1(b), introduced in [6] and shown to contain enough spatial heterogeneity
to cover the complex 3D biodistributions associated with in vivo imaging). These in silico phantoms have a range of
values for δµa and depth. MCX is deployed to generate perturbed (φ) and unperturbed (φo) measurement vectors for
each in silico phantom (Fig. 1(c)). The ModAM network is trained using 5,000 measurement vectors under the
Rytov approximation (log φo/ φ) with an 80/20 training/validation split and an Adam optimizer. The overall
structure of the network is illustrated in Fig. 1(d).
For experimental validation, an agar-based in vitro phantom is prepared. A mixture of ink and intralipid solution is
used to generate the absorption contrast and scattering, respectively. In a homogenous background, two thin
capillaries are filled with the same absorption contrast and embedded at a high depth (for diffuse optics) of 8.5 mm
from the illumination plane (δµa=0.176 mm-1 and reduced scattering, μs’=1 mm-1). A schematic of the in vitro
phantom is shown in Fig. 2(a). Our single-pixel hyperspectral system (equipped with Digital Micro-Mirror Devices)
is used for projecting the 36 illumination and detection patterns (same as the ones shown in Fig. 1(a)). The perturbed
and unperturbed measurement vectors are recorded with a 16-channel PMT. The details of the experimental
Fig. 1. (a) Illumination and detection
patterns (36 each). (b) The EMNIST
dataset used for training. (c) Perturbed
(red) and Unperturbed (blue)
measurements simulated using MCX.
(d) The ModAM architecture. (e) The
reconstructed iso-volumes with their
GT. (f) The 2D cross-sections at a
depth of 2 mm. (g) Table showing the
quantitative results in terms of the VE
and
the maximum reconstructe
d
δμ
a
.
OW4D.7 Biophotonics Congress: Biomedical Optics (Translational,
Microscopy, OCT, OTS, BRAIN) © Optica Publishing Group 2022
© 2022 The Author(s)
apparatus and protocol can be found in [4]. Additionally, to obtain the exact position, depth, and separation of the
two tubes, a micro-CT scan is carried out (Fig. 2(b)). The volume obtained from the micro-CT is treated as the
Ground-Truth (GT) for the experiment (to calculate the Volume Error (VE)).
3. Results
Representative reconstruction results for an in silico phantom, not part of the training dataset, are presented in Figs.
1(e)-1(g) for both the ModAM network and the regularized LSQ-based technique. The in silico phantom has a
dimension of 30x40x20 mm3 (to match the in vitro experiment) and a homogenous embedding in a homogenous
background (δµa=0.2 mm-1). The embedding (thickness=3 mm) is placed at a shallow depth of 2 mm from the
illumination plane. We present the results in terms of the iso-volume (Fig. 1(e)), the 2D cross-sections at the 2 mm
depth (Fig. 1(f)), and the quantitative evaluation of the reconstructions in terms of the maximum value of the
reconstructed δµa and the VE (Fig. 1(g)). The results obtained from the ModAM network are superior to the
regularized LSQ, both in terms of the VE and the maximum value of δµa reconstructed. Although the ModAM
network takes approximately 5.25 hours to train (NVIDIA RTX 2080 Ti), the reconstruction time for the ModAM
network is a few milliseconds, while that for the regularized LSQ is approximately 20 minutes. The reconstruction
results of the in vitro experiment are presented in Figs. 2(c) and 2(d) (the iso-volumes and the 2D cross-section at a
depth of 10 mm, respectively). Here, the ModAM results are significantly better than those obtained from the
regularized LSQ (as shown quantitatively in Fig. 2(e)). The time advantage in reconstruction is similar to the in
silico case, with the added bonus that the ModAM network need not be re-trained for the in vitro experiment.
Fig. 2. (a) Schematic of the in vitro phantom (b)
Volume generated from micro-CT scan. (c)
Reconstructed iso-volumes. (d) 2D cross-sections at a
depth of 10 mm. (e) Table showing the quantitative
results in terms of the VE and the maximum
reconstructed δμa.
4. Discussion/Conclusion
Our presented in silico and in vitro results reveal that the ModAM network can lead to faster and more accurate δμa
reconstructions than the traditional techniques even at high depths (hence, high scattering). It has also been
demonstrated that the ModAM network, enhanced by the spatial heterogeneity in the EMNIST dataset, can
reconstruct a wide array of structures, both in silico and in vitro. However, re-training the network will be necessary
to change the source-detector configuration and/or phantom dimensions. A detailed investigation, with more in vitro
data and pre-clinical in vivo imaging, will be carried out in future works.
Acknowledgements
The authors acknowledge the funding support from National Institutes of Health (NIH) under grants R01CA237267,
R01CA207725 and R01CA250636. We would like to thank Mr. Mengzhou Li and Mr. Xiaodong Guo for providing
the raw micro-CT data.
References
[1] Q. Pian, et al., “Compressive Hyperspectral Time-resolved Wide-Field Fluorescence Lifetime Imaging,” Nature Photonics 11, 411-417
(2017).
[2] L. Tian, et al., “Deep learning in biomedical optics,” Laser in Surgery and Medicine 53(6), 748-775 (2021).
[3] B. Zhu, et al., “Image reconstruction by domain-transform manifold learning,” Nature 555, 487-492 (2018).
[4] Q. Pian, et al., “Hyperspectral wide-field time domain single-pixel diffuse optical tomography platform,” BOE 9(12), 6258-6272 (2018).
[5] R.Yao et al. "Direct approach to compute Jacobians for diffuse optical tomography using perturbation Monte Carlo-based photon “replay”."
Biomedical optics express 9(10), 4588-4603 (2018).
[6] R. Yao et al. "Net-FLICS: fast quantitative wide-field fluorescence lifetime imaging with compressed sensing–a deep learning approach."
Light: Science & Applications 8(1), 1-7 (2019).
OW4D.7 Biophotonics Congress: Biomedical Optics (Translational,
Microscopy, OCT, OTS, BRAIN) © Optica Publishing Group 2022
