ArticlePDF Available

A hybrid strategy to integrate surface-based and mutual-information-based methods for co-registering brain SPECT and MR images

Authors:

Abstract and Figures

Co-registration of brain SPECT and MR images has been used extensively in clinical applications. The complementary features of two major co-registration methods--surface- and mutual-information-based (MI-based)--motivated us to study a hybrid-based scheme that uses the surface-based method to achieve a quick alignment, followed by the MI-based method for fine tuning. Computer simulations were conducted to evaluate the accuracy and robustness of surface-, MI-, and hybrid-based registration methods by designing different levels of noise and mismatch in the registration experiments. Results demonstrated that the hybrid surface-MI-based scheme outperforms both the surface- and MI-based methods in providing superior accuracy and success rates. Specifically, the translational and rotational errors were no more than 1 mm and 2°, respectively, with consistent success rates over 98%. Besides, the hybrid-based method saved 12-53% of the computation efforts, compared with using the MI-based method alone. We recommend the use of hybrid-based method when the orientational differences between the floating and reference images exceed 10°.
Content may be subject to copyright.
ORIGINAL ARTICLE
A hybrid strategy to integrate surface-based
and mutual-information-based methods
for co-registering brain SPECT and MR images
Yuan-Lin Liao Yung-Nien Sun Wan-Yuo Guo
Yuan-Hwa Chou Jen-Chuen Hsieh
Yu-Te Wu
Received: 14 May 2010 / Accepted: 9 December 2010
ÓInternational Federation for Medical and Biological Engineering 2010
Abstract Co-registration of brain SPECT and MR images
has been used extensively in clinical applications. The
complementary features of two major co-registration meth-
ods—surface- and mutual-information-based (MI-based)—
motivated us to study a hybrid-based scheme that uses the
surface-based method to achieve a quick alignment, fol-
lowed by the MI-based method for fine tuning. Computer
simulations were conducted to evaluate the accuracy and
robustness of surface-, MI-, and hybrid-based registration
methods by designing different levels of noise and mismatch
in the registration experiments. Results demonstrated that
the hybrid surface-MI-based scheme outperforms both the
surface- and MI-based methods in providing superior accu-
racy and success rates. Specifically, the translational and
rotational errors were no more than 1 mm and 2°, respec-
tively, with consistent success rates over 98%. Besides, the
hybrid-based method saved 12–53% of the computation
efforts, compared with using the MI-based method alone.
We recommend the use of hybrid-based method when the
orientational differences between the floating and reference
images exceed 10°.
Keywords Image registration Mutual information
Surfaces Single photon emission computed tomography
Magnetic resonance imaging
1 Introduction
The use of multimodal imaging for etiological diagnosis
and treatment planning has been increasing in recent years
[41]. Image fusion becomes one of the essential steps to
integrate structural and functional information [37,49].
Single photon emission computed tomography (SPECT)
and magnetic resonance imaging (MRI) are two popular
image modalities for medical examination. The former
provides functional information with regard to the cerebral
blood flow and metabolism of specific regions, and the
latter supplies the neuroanatomic knowledge that can be
used to better localize the activation or lesion foci in the
SPECT image. Co-registration of brain SPECT and mag-
netic resonance (MR) images has been used extensively in
a number of clinical applications [2,13,21,26,33,35].
Y.-L. Liao Y.-N. Sun (&)
Department of Computer Science and Information Engineering,
National Cheng Kung University, No. 1, Dasyue Rd., East
District, Tainan City 70101, Taiwan
e-mail: ynsun@mail.ncku.edu.tw
W.-Y. Guo
Department of Radiology, Taipei Veterans General Hospital, No.
201, Sec. 2, Shipai Rd., Beitou District, Taipei City 11217,
Taiwan
Y.-H. Chou
Department of Psychiatry, Taipei Veterans General Hospital,
No. 201, Sec. 2, Shipai Rd., Beitou District, Taipei City 11217,
Taiwan
J.-C. Hsieh Y.-T. Wu
Integrated Brain Research Laboratory, Department of Medical
Research and Education, Taipei Veterans General Hospital, No.
201, Sec. 2, Shipai Rd., Beitou District, Taipei City 11217,
Taiwan
J.-C. Hsieh
Institute of Brain Science, National Yang-Ming University, No.
155, Sec. 2, Linong St., Beitou District, Taipei City 11221,
Taiwan
Y.-T. Wu (&)
Department of Biomedical Imaging and Radiological Sciences,
National Yang-Ming University, No. 155, Sec. 2, Linong St.,
Beitou District, Taipei City 11221, Taiwan
e-mail: ytwu@ym.edu.tw
123
Med Biol Eng Comput
DOI 10.1007/s11517-010-0724-9
Two major co-registration methods, surface- and mutual
information (MI)-based methods, are widely used in the
medical imaging community. In the surface-based method,
the brain surface is first extracted semi-automatically and
transformed into a chamfer distance map for subsequent
alignment [6,9]. Mutual information, rooted in information
theory, is another similarity measure employed in mul-
timodality image registration [20,25]. The MI-based
method has been shown to be robust to intensity variations
and adaptable to image noise [19,40,48,50].
Although the surface- and MI-based methods come from
different perspectives, they are complementary. MI-based
registration is advantageous in fully automation, i.e., it does
not require prior segmentation of the brain surface, and
affords superior precision since intensities within the brain
area are more informative than those on the brain surface.
However, the need for iteratively updating the joint histo-
gram in calculating the MI is time consuming. Furthermore,
Pluim et al. found the different patterns of artifacts when
trilinear and partial volume interpolation is used in esti-
mating entropy-based metrics by joint histogram [24]. This
interpolation artifact occurs prominently near the grid-
aligning transformation and creates local extremes which
deteriorate the registration performance. It followed that
several researchers proposed solutions to this problem by
either using different interpolation schemes when con-
structing joint histograms [4,18,31,42] or additional
procedures with image interpolation [29,30]. On the other
hand, the surface-based method, using distance maps cre-
ated from the MR and SPECT images for characterizing the
three-dimensional (3D) surfaces, is computationally effec-
tive and much less sensitive to the initial guess. However,
owing to its lower resolution and blurred contents in SPECT
images, the surface may not be correctly demarcated. As a
consequence, the matching results would be less accurate
than those of the MI-based method.
S
ˇkerl et al. [36] proposed and made available a protocol
to quantitatively evaluate the robustness of similarity
measures for rigid registrations (http://lit.fe.uni-lj.si/
Evaluation). Its capture range and the risk of nonconver-
gence are two useful properties. The capture range is
defined as the smallest distance between the global maxima
and the closet local minima; the risk of nonconvergence
describes the behavior around the global maximum. Note
that a smaller risk of nonconvergence and a larger capture
range indicate better similarity. In this study, we compared
the surface- and MI-based methods using a synthetic pair
of MR/SPECT images. The results show that the surface-
based method had a larger capture range (54.6 vs. 2.8 mm)
and smaller values of risk of nonconvergence (Fig. 1),
suggesting that fewer local minima existed in the vicinity
of ground truth and the registration would converge to the
ground truth through iterative optimization. The comple-
mentary features of these two methods motivated us to
explore the use of a hybrid strategy that uses the surface-
based method for a quick alignment, followed by fine
tuning with the MI-based method.
Some of the relevant studies used the surface informa-
tion in the registration in addition to the MI-based tech-
nique. Lee et al. [15] first applied thresholding to obtain an
Fig. 1 Risk of nonconvergence
for the surface- and MI-based
cost functions computed from
the synthetic pair of MR/SPECT
images as a function of distance
from the ground truth. Note that
the definition of risk of
nonconvergence is the average
of positive gradients within a
specific distance from each of
the global maxima [36]. The
smaller values (all zeros) of risk
of nonconvergence suggest that
fewer local minima exist in the
vicinity of ground truth for the
surface-based cost function
Med Biol Eng Comput
123
approximate brain volume in positron emission tomogra-
phy (PET) image and defined a shell volume consisting of
voxels in the vicinity of brain surface. The PET image was
registered to the computed tomography (CT) or MR image
using the normalized MI criterion based on the shell vol-
ume. Compared to our method, the shell registration was
merely a volume-based registration method, in which the
transformation parameters were not initialized by any other
method. Moreover, in the approach of this study, the
information of the whole brain is used to maximize the MI
cost function for further refinement, rather than using part
of the information (shell volume) which was more likely to
be trapped into local extremes. Eldeib et al. [7] used the
surface point signature (SPS), which captured the 3D cur-
vature information, to initially align the CT and MR ima-
ges, and enhanced the results with MI-based registration.
Although our method is similar to the two-stage registra-
tion proposed by Eldeib et al., we do not use the curvature
information since in the functional (SPECT) images the
delineated surface is more blurred than that obtained from
anatomic (CT/MR) images, and it is problematic to identify
important points (landmarks) therein. Sinha et al. [34] used
the iterative closest point (ICP) algorithm in image guided
neurosurgery to match the textured surface obtained from a
laser range scanner with a 3D surface extracted from MR
images. The results were subsequently refined by a con-
strained MI-based registration to match the textured sur-
face with the intensities on the MR surface. Wang et al.
[44] used the MI to match general 3D surfaces in the
parametric domain with applications on face recognition
and subcortical shape analysis. The proposed hybrid-based
registration is used to register medical volumes, rather than
align the surface data in the study of Sinha et al. (point
clouds) and Wang et al. (surface mesh). There were some
other relevant studies that combined or compared intensity-
based metrics with the anatomic features or surface-based
criteria. Lee et al. [14] proposed a new similarity metric
combining anatomic features in specific binding regions
with the normalized MI in co-registering
99m
Tc-TRODAT-
1 SPECT and T2 MR images. West et al. [45] compared 14
existing surface- and volume-based methods on their inter-
modality registration, such as CT versus MR and PET
versus MR. They concluded that the volume-based meth-
ods tended to be significantly or slightly more accurate.
Pfluger et al. [23] quantified and assessed the error of
interactive (manual) matching, surface-based matching,
and intensity-based uniformity index matching in MRI–
SPECT image registration. They found that interactive
matching provided the highest accuracy, whereas surface
matching had the largest error and required the most time.
In Kagadis et al. [11], a comparative study was conducted
with CT and SPECT images, where only the normalized
correlation coefficient was employed for quantitative
analysis and the volume-based method performed better
than either the ICP algorithm or surface-based method.
The aim of this study is to comprehensively evaluate
and compare the performance of the surface-based,
MI-based, and the proposed hybrid surface-MI-based
method, in terms of accuracy, computational efficiency,
and robustness. We expand our previous study [16]by
using an additional simulated dataset, adopting multi-res-
olution scheme, and investigating the effect of image noise.
The proposed hybrid-based registration is illustrated by a
detailed flowchart (Fig. 2). This article is organized as
follows. First, the procedures for image acquisition, pre-
processing, and registration for all three strategies are
presented. Experimental designs are then described, the
results are then presented, and followed by the discussion.
Finally, this study is concluded.
2 Methods
2.1 Image acquisition
Two volunteers were recruited to participate in this study.
Their written informed consent was obtained before the
experiment. For the SPECT imaging, the subjects lay on the
scanner table for 20 min, being blindfolded and their ears
plugged, before injection and SPECT scanning. The
99m
Tc-
ECD-SPECT scanning was performed 30 min after injection
with 740 MBq (20 mCi) per unit dose. The image acquisi-
tion procedure used the routine regional cerebral flow
(rCBF) clinical protocol. A dual head system (E.CAM;
Siemens Medical Solutions USA, Inc.) equipped with fan
beam collimators was used, and the photon window was
140 keV ±10%. The radius of camera rotation in ECD
scanning was fixed at 13.5 cm. The camera scanned with
step-and-shoot mode, and the camera zoom was 1.23, the
image matrix was 128 9128 with a 3.9 93.9 mm
2
pixel
size. The ECD scanning data were collected from 60 views of
each camera head with 180°rotation. Projection data were
reconstructed using an ordered subset expectation maximi-
zation (OSEM) algorithm implemented in the image recon-
struction toolbox (MATLAB software, available at http://
www.eecs.umich.edu/*fessler/code/) followed by 3D
Gaussian smoothing with the convolution kernel set to three.
Structural MR images were acquired using a 1.5 Tesla MR
scanner (General Electric, Milwaukee, WI, USA) with a 3D
fast-spoiled-gradient-recalled (FSPGR) T1 sequence (TR/
TE/TI =8.54/1.84/400 ms, FOV =260 mm, matrix
size =256 9256, slice thickness =1.5 mm, NEX =1,
flip angle =15°), to obtain 124 axial slices with an in-plane
resolution of 1.02 91.02 mm
2
.
Med Biol Eng Comput
123
2.2 Image pre-processing
Both the MR and SPECT images were first resampled to
make their voxel sizes identical. In the surface-based
method, the brain regions were extracted before registra-
tion. The reference image was defined as one left
unchanged during the registration process; the floating
image is the one to be transformed toward the reference
image. After the brain areas are segmented from the ref-
erence and floating images, the distance maps of brain
surfaces can be constructed based on the morphological
dilation operation. Details of each pre-processing proce-
dure are elaborated in the following sections.
2.2.1 Image resampling
Since the voxel sizes in SPECT and MR images differed
and the voxels in MR images were anisotropic, a modified
sinc interpolation to resample the images of both modali-
ties to ease the subsequent calculation is adopted. The
intensity transformation of each voxel from the original
image coordinates (X,Y,Z) to the resampled image coor-
dinates (x,y,z) is given by
where Wdefines a sinc function multiplied by the apodizing
Hanning function in the following [8,39]:
Wða;AÞ¼sinðpðaAÞÞ
pðaAÞ1
21þcos pðaAÞ
Rþ1

ð2Þ
where Rcan be viewed as the range of influence for
truncating the interpolation. Each voxel of the SPECT and
MR images was resampled into a cubic size of 2 mm. The
resolutions of SPECT and MR images were adjusted to
128 9128 9N, where Nare image depths representing
the slice numbers in volumetric data.
2.2.2 Brain volume of interest (VOI) extraction
Fully automatic approaches were applied to ensure repro-
ducibility and avoid manual intervention. Image thres-
holding is an efficient way to extract an approximate VOI
and can be classified into histogram shape-based, cluster-
ing-based, entropy-based, object attribute-based, etc. (see
[32] for a survey). The optimal (iterative) thresholding [28],
Otsu’s thresholding [22], minimum error thresholding [12],
and fuzzy clustering methods [47] are the four popular
clustering-based methods used in image analysis. The first
Fig. 2 The detailed flowchart
of the proposed hybrid-based
registration algorithm. HR and
LR represent the high- and low-
resolutions, respectively
Iðx;y;zÞ¼PXPYPZIðX;Y;ZÞ Wðx;XÞ
jj
Wðy;YÞ
jj
Wðz;ZÞ
jj
PXPYPZWðx;XÞ
jj
Wðy;YÞ
jj
Wðz;ZÞ
jj ð1Þ
Med Biol Eng Comput
123
three methods presume a normal distribution for each
population in image histogram and work well even when
the image histogram is not bi-modal. The fuzzy clustering
method takes into account structural details, which are
embedded in the intensity distribution, through the assign-
ment of fuzzy membership to each voxel. Sezgin and
Sankur found that minimum error thresholding had the best
performance in the case of nondestructive images, and text
document images degraded with noise and blur [32]. We
utilized the Otsu’s thresholding to separate the brain region
from air background in the ECD-SPECT images by maxi-
mizing the following formula:
r¼lBðtÞxOðtÞlOðtÞxBðtÞ½
2
xOðtÞxBðtÞð3Þ
where tis the threshold to be determined, the resulting
background and object means are l
B
and l
O
, and the
probability of the background and object occurring are x
B
and x
O
, respectively. By evaluating the corresponding
distance measure rwith respect to different thresholds, we
can automatically obtain the optimal threshold that maxi-
mizes the separation between the two classes and mini-
mizes the sizes of the two classes. For MR images, we first
obtain the probabilistic gray matter (GM) and white matter
(WM) images by executing the segment function in the
SPM5 package (MATLAB software, available at
http://www.fil.ion.ucl.ac.uk/spm/) with the default setting.
Voxels with a probability exceeding a predefined threshold
(128 in an 8-bit integer image in this study) in either the
GM or the WM image were viewed as object voxels and
included in the brain region to produce a skull stripped
image. Both extracted brain SPECT and MR images were
post-processed by identifying the maximum connected
component and the morphological region growing to
remove the isles and fill the holes. Figure 3displays the
segmented brain regions from the selected slices of
resampled ECD-SPECT and T1-MR images.
2.2.3 Surface extraction and distance transform
After image resampling and cortex segmentation, the cor-
tical surface was extracted and a distance map was con-
structed. The cortical surface can easily be obtained by
applying a 3D border detector [17] to the binary images.
The distance map encapsulated each voxel a value, which
represents its distance from the surface, and served as a
distance measure between two surfaces. A technique based
on morphological dilation, similar to the chamfer method
[3], was developed to create the distance map. The struc-
tural element used in the implementation was the 6-con-
nected octahedron. Figure 4illustrates the construction
process—the blue voxels represent the cortical surface with
zero distance values. The distance value for each voxel
outside or inside the surface, labeled d, was defined as the
number of morphological dilations from the surface. For
example, the distance values for the green and orange
voxels are d=1 and 5, respectively, since they are reached
after one and five dilation repetitions, respectively. In other
words, the voxels have larger distance values if they are
farther away from the surface. The procedure continues
until all the voxels are labeled. The surface extraction
results and distance maps for axial slices of ECD-SPECT
and T1-MR images are depicted in the third and fourth
rows, respectively, in Fig. 3a, b.
2.3 Image registration
In this section, the surface- and MI-based methods are
briefly reviewed. In the registration, rigid body transfor-
mations were used for intra-subject registration with three
translational and three rotational parameters, since the
floating and reference images were from the same subject.
Powell’s method [27] was used for searching for the
optimal rigid transformation parameters in the surface- and
MI-based methods. Finally, a hybrid-based strategy is
introduced.
2.3.1 Surface-based method
Surface-based registration aims to exploit surface infor-
mation encoded in the distance maps for aligning two
volumetric images. The cost function to be minimized was
defined as the sum of distance values in the pre-computed
distance map, D
R
, for the reference image. Specifically, for
each voxel with position x
s
on the surface of the floating
image F, its new position, represented by T
a
(x
s
), via the
estimated rigid body transformation Twith parameter ais
first calculated. If the new position T
a
(x
s
) is an integer, the
distance value D
R
(T
a
(x
s
)) can be immediately obtained
from a look-up table, since D
R
was already computed and
saved in a buffer; otherwise, the distance value is calcu-
lated by the trilinear interpolation. The cost function
X
xs2F
DRTaðxsÞðÞ ð4Þ
is calculated with respective to the transformation Tuntil it
reaches its minimum. It is obvious that, when two images
are misregistered, most of the surface voxels in the floating
image do not coincide with those of the reference image—
the distance values of these voxel locations are large,
leading to a large cost function (4). However, two closely
registered images, where the two surfaces are nearly
overlapped (see Fig. 5), provide a much smaller cost
function.
Med Biol Eng Comput
123
2.3.2 MI-based method
The MI-based method, developed from information theory
[5], is a common approach for registering multimodal
images. Let two marginal probability distributions be
denoted by p
A
(a) and p
B
(b), with random variables, Aand
B, respectively, and the joint probability distribution be
denoted by p
A,B
(a,b). The mutual information between
Aand B, denoted by I(A,B), is a criterion for estimating the
degree of mutual dependence by measuring the distance
between the joint distribution p
A,B
(a,b) and the estimated
distribution in the case of independence, p
A
(a)p
B
(b), based
on the Kullback–Leibler divergence [43]. It is defined by
IðA;BÞ¼X
a;b
pA;Bða;bÞlog pA;Bða;bÞ
pAðaÞpBðbÞð5Þ
In implementing MI-based registration, Aand Bstand
for the floating and reference images, aand bare the
intensities gathered from images, and the probability values
are estimated by individual and joint histograms. Partial
Fig. 3 Sample slices of aECD-SPECT and bT1-MR images. The resampled images, extracted brain areas, corresponding contours, and
distance maps are shown in the first, second, third, and fourth rows, respectively, on both panels
Med Biol Eng Comput
123
volume interpolation [19] is adopted when the transformed
positions from the floating image do not coincide with the
integer grids in the reference image.
2.3.3 Hybrid surface-MI-based registration strategy
Previous studies have reported that the advantage of
surface-based chamfer matching method is that it is com-
putationally efficient, since the registration involves only
pre-computed distance maps and simple trilinear interpo-
lation [6]. On the other hand, intensity-based MI method
makes use of the overall intensity information, which
provides better registration precision. These complimen-
tary features motivated us to use the surface-based method
for providing a good initial guess of transformation
parameters for guiding subsequent MI-based registration,
which is more likely to be trapped in the local extreme
values if the transformation parameters are not initialized
appropriately. We used Powell’s method [27] to search for
the optimal rigid transformation parameters to minimize
the cost function, based on surface distance maps, and then
maximized the mutual information. The Powell’s conjugate
directions method only requires evaluations of the
N-dimensional cost function fitself and not of the deriva-
tives of f. This method finds the N-dimensional minimum
of fby repeatedly minimizing fin one dimension along a
set of ndifferent directions, each time starting from the
minimum found in the previous direction using a 1D
minimization method such as the Brent’s method.
2.4 Multiresolution framework
The authors adopted a multiresolution scheme in image
registration implementation in this study to avoid the local
minima and speed convergence. A two-level framework,
where a low resolution image is created from the original
volume (dimension: 128 9128 9N) by reducing the
image dimension by half (dimension: 64 964 9N/2) with
Gaussian smoothing was used. Surface- and MI-based
registrations perform the pre-processing and registration
procedures at this low resolution level, and the results are
used in the high resolution level as initial parameters for
co-registering the original volumes. For hybrid-based reg-
istration, the surface-based method was applied at both
levels, followed by MI-based registration at high resolution.
Note that Otsu’s thresholding to extract the brain VOI in the
MR images at the low resolution level, rather than SPM
segmentation, has been used here since the image resolution
is too low to achieve a correct tissue classification by SPM5.
2.5 Experiments
Computer simulations were designed and conducted to
evaluate the performance of surface-, MI-, and hybrid
surface-MI-based algorithms for recovering the mismatch
between the floating and reference images. The experi-
mental images included one set of paired MR/SPECT
synthetic images and two sets of paired MR/SPECT real
images. To test the accuracy and robustness of the regis-
tration algorithms, a set of values representing the trans-
lational offsets and the rotational deviations were
generated. We designed three scales (0, 10, and 20) for
translations Tand four scales (0, 10, 20, and 30) for rota-
tions R, resulting in 3 94=12 combinations. Six random
values, three for translation and three for rotation, were
added to each combination. Each random value was gen-
erated by a Gaussian random variable with zero mean and
standard deviation r=2. This generation of random
deviation was repeated 10 times and added to each com-
bination to obtain 120 sets of artificial misregistration
parameters. Note that these misregistration parameters
were used for all registration methods, and all data pairs for
a fair comparison. In addition, different noise levels were
added to the synthetic and clinical projection data to further
test these algorithms. All the experiments were carried out
by an in-house program implemented using Microsoft
Visual C?? 6.0.
2.5.1 The synthetic images
An image pair, termed s1n0 (s1 is the code name of the
synthetic data set and n0 indicates that the images are free
from artificial noise), was first used, constructed from
Fig. 4 Illustration of distance computation via dilation (simplified as
a 2D case for easy interpretation). The original contour pixels are in
blue, and then dilate inward and outward to the 4-neighborhood as
pixels in green, where their distance values equal to one. After five
dilations, the dilated front reaches the position where pixels in orange,
and are labeled as five. These steps are repeated until all the pixels in
the image have obtained a distance value
Med Biol Eng Comput
123
digital simulators [1] (Fig. 6a, b). The misregistration
parameters generated as described above were added to the
known ground truth parameters to initialize the registration
process. A total of 120 trials were conducted to test each
registration method.
2.5.2 The real data
As the ground truth for the real image pair is unknown, two
image pairs were manually registered: c1n0, and c2n0.
Then, an exhaustive search for the six rigid transformation
parameters was performed around the manual solution until
the maximal MI value was obtained. This result was further
examined by an experienced radiologist to confirm its
accuracy. The resultant transformation parameters were
considered as ground truth.
2.5.3 Adding noise
Another important issue is how the noise content affects
the registration results. To investigate the registration
performance with respect to different noise levels, two
noise acute counterparts of the SPECT image were simu-
lated, as in Kadrmas et al. [10]. Each intensity value in the
projection data was divided by a constant, referred to as the
divisor constant (DC), to mimic the control of noise by
reducing photon counts. Then, each value which served as
the mean of a Poisson distribution was fed into the Poisson
random number generator to create a new value in the noise
acute sinogram. This projection data was reconstructed into
a simulated volume with a controlled noise content. The
noise levels were characterized by the number of counts
controlled by DCs. As the value of DC increased, the noise
Fig. 5 Illustration of the
relative contour position with
aaxial and bsagittal view and
their corresponding distance
errors, which are 136588 and
19970 in misregistration (the
upper two rows in each panel)
and well-registration (the lower
two rows in each panel),
respectively
Med Biol Eng Comput
123
level became worse. In our simulations, DCs were set to
values between 20 and 40. Two noisy volumes generated
from s1n0 were denoted s1n1 (DC =20) and s1n2
(DC =40). The clinical images were similarly denoted
c1n0 and c2n0; c1n1 and c2n1 (DC =20); and c1n2 and
c2n2 (DC =40). Figure 6c, e shows several noisy slices
constructed from the synthetic SPECT image, and Fig. 6d,
f also displays their corresponding extracted surfaces,
which are more distorted and irregular than those (the third
row in Fig. 3a) created from real data.
3 Results
The registration results of c1n0 obtained from manual
alignment followed by a locally exhaustive search are
displayed in Fig. 7, where the SPECT and MR images are
transparently fused (upper row), superimposed in a
checkerboard manner (middle row), and their surface
contours overlaid to provide better visualization (lower
row). The indianred regions (in the upper row) in the
SPECT image aligned with the gray matter and cerebellum
in the MR image, and the border of the activated region in
the SPECT image approximated the pial surface in the MR
image (see the middle and lower rows), suggesting accurate
registration.
The registration error can be quantified based on the root
mean square error (RMSE) of translation RMSE
t
and
rotation RMSE
r
defined by
RMSEt¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
1
3txt0
x

2þtyt0
y2þtzt0
z

2
hi
rð6Þ
RMSEr¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
1
3rxr0
x

2þryr0
y2þrzr0
z

2
hi
rð7Þ
where (t
x
,t
y
,t
z
,r
x
,r
y
,r
z
) are estimated parameters and
t0
x;t0
y;t0
z;r0
x;r0
y;r0
z

are true parameters. Trained clinicians
have been reported to be able to detect differences from the
registration parameter of 4°in the x- and y-rotation angles,
2°in the z-rotation angle, 2 mm in the x- and y-translations,
Fig. 6 The synthetic aperfusion SPECT and bT1-MR images. Noise
is added to create the simulated noisy SPECT images with two noise
levels controlled by the divisor constant (DC): cDC =20 and
eDC =40. The corresponding surfaces of noisy SPECT images are
also shown in (d) and (f)
Med Biol Eng Comput
123
and 3 mm in the z-translation [46]. Therefore, similar cri-
teria for computing the RMSE
t
(*2.38 mm) and RMSE
r
(*3.65°) thresholds were adopted—when both the RMSE
t
and RMSE
r
were under these thresholds, it was regarded as
a success.
The registration accuracy was evaluated with the
RMSEs, and the results of all trials of synthetic and real
image pairs are displayed as box-and-whisker plots in
Fig. 8. The medians and interquartertile ranges of the
RMSEs in all the trials are summarized in Table 1. Note
that medians and interquartertile ranges, instead of means
and standard deviations, have been adopted to better
describe the error variations. In Fig. 8a, the RMSEs of the
MI-based results increased as the noise content spoiled the
image. Median RMSE
t
and RMSE
r
for MI-based method
magnified from 0.003 to 0.934 mm and 0.008°to 2.193°,
respectively (see Table 1). The surface-based registration
resulted in the largest RMSE
t
(1.082 mm) among the three
methods for s1n2; the hybrid-based method demonstrated
superior performance with consistently small RMSEs,
which also presented (Table 1) that the medians and
interquartertile ranges were less than 0.01 mm/°. The
MI-based results, Fig. 8b, exhibited the largest errors; the
surface-based ones were smaller; and the hybrid-based
method appeared to produce the highest accuracy for all
image pairs. The median RMSE
r
of the surface- and
hybrid-based methods were comparable, which were
between 1.113°and 1.442°. However, the median RMSE
t
of the hybrid-based method were smaller (between 0.207
and 0.609 mm in Table 1) than that of the surface-based
method (between 0.748 and 1.072 mm in Table 1). As
shown in Fig. 8c, the hybrid-based outperformed the
surface- and MI-based methods, with the smallest RMSEs
less than 0.04 mm/°(see Table 1). We also noted that the
MI-based results were unstable, producing the largest
variances in all the synthetic and clinical images. After
excluding the failed registrations, the average RMSE
t
and
RMSE
r
values of the hybrid-based method were less than
1 mm and 2°, respectively.
The issue of computational efficiency was addressed
based on the number of iterations in a successful registra-
tion. The surface- and MI-based methods have comparable
numbers of iterations, but they inevitably increased with
larger rotational deviations. For deviations from 0°to 30°,
the average number of iterations for registering the syn-
thetic image series at the low resolution level grew from
6.1 to 8.0 and 4.8 to 7.0 for surface- and MI-based meth-
ods, respectively, whereas it took only from 1.2 to 1.6 and
1.8 to 2.0 iterations at the high resolution level. More
iterations were required to register the two clinical data
sets, ranging from 4.4 to 9.3 and 3.1 to 9.4 iterations at low
resolution, and from 3.1 to 5.5 and 2.1 to 3.8 at high res-
olution. In the hybrid-based registration, the average
number of iterations at the surface-based stage was the
same as the surface-based method since they started from
the same point, and the MI-based process took two to three
extra iterations on average—a prominent reduction of
computation load compared to the MI-based method.
Figure 9shows the time required for the three methods for
different rotational deviations, where the values in the
vertical axis were calculated by multiplying the average
number of iterations by the normalized time units per
iteration. In the experiments of this study, the time required
for one iteration in the MI-based method was 60 and 72
Fig. 7 The ground-truth overlapping of c1n0 pair in transparent fusion (upper), checkerboard display (middle), and contour overlay (lower,
SPECT in magenta and MR in cyan)
Med Biol Eng Comput
123
Fig. 8 The boxplots of RMSE
t
and RMSE
r
of the as1, bc1,
and cc2 image series for the
surface-, MI-, and hybrid-based
methods. Note that the ends of
whiskers represent the
minimum within 1.5 times the
interquartile range of the lower
quartile, and the maximum
within 1.5 times the
interquartile range of the upper
quartile. Outliers (red crosses)
are identified as those data
beyond the upper and lower
ends of the whiskers
Med Biol Eng Comput
123
times more than that for the surface-based method at low
and high resolution, respectively, on a personal computer
with an Intel Pentium D CPU 3.4 GHz. The exact nor-
malized time units (tu) are 1 and 5 for surface-based iter-
ations at low and high resolutions, respectively, and 60 and
360 for MI-based iteration (1 tu &0.5 s). The surface-
and hybrid-based methods require extra computation, but
this required only an additional 1 tu for all the prepro-
cessing steps, including surface extraction and distance
transform, and 450 tu for MR surface segmentation using
SPM5. Not counting the extra time of pre-processing steps
and surface segmentation, the surface-based method was
the most computationally efficient, whereas the computa-
tional time of the MI-based method increased significantly
as the rotational deviation increased due to the demand for
more iterations. The computational cost of the hybrid-
based method was reduced significantly in comparison
with that of the MI-based method, since it drastically
reduced the number of iterations in the MI-based stage.
The analytic results of the study of robustness are pre-
sented with the success rates. The success rates of surface-
based registration, ranging from 97 to 100%, were, in
general, higher than those of the MI-based method. For the
MI-based method, the success rate varied from 98–100%
for synthetic pairs, 51–85% for the c1 pairs, and 62–90%
for the c2 pairs. The hybrid-based method performed
consistently with success rates above 98%. We further
observed that, for the nine pairs of test images, the hybrid-
based method recovered 168 from 170 (99%) failed MI-
based trials.
4 Discussion
In this study, different levels of translational displacement
and rotational deviation were used to evaluate the perfor-
mance of surface-, MI-, and hybrid-based registration
methods. The resulting translational and rotational errors
that were less than predefined tolerances defined the suc-
cess rate, which in turn was regarded as an index for the
assessment of registration robustness and accuracy. A high
success rate indicates that a method can register two ima-
ges with modest, as well as large, positional and orienta-
tional deviations. Although the delineation of surface from
the ECD-SPECT image relied strongly on the image
quality, the surface-based method performed more con-
sistently than the MI-based method in terms of less widely
dispersed distributed errors, and failed trials appeared
mostly in image pairs with high noise levels (three in c1n2
and four in c2n2). This explained why the surface-based
method achieved higher success rates despite its larger
median errors in the c2 image series. On the other hand, the
MI-based method was likely to be affected by the
Table 1 The medians of the RMSE
t
and RMSE
r
for surface-based (S), MI-based (M), and hybrid-based (H) registration methods
Image s1n0 s1n1 s1n2
Method: S M H S M H S M H
RMSE
t
(mm) 0.133 (0.298) 0.003 (0.004) 0.001 (0.003) 0.005 (0.009) 0.281 (0.804) 0.001 (0.002) 1.082 (0.251) 0.934 (1.061) 0.003 (0.008)
RMSE
r
(°) 0.001 (0.003) 0.008 (0.007) 0.004 (0.006) 0.001 (0.002) 0.411 (1.886) 0.001 (0.002) 0.002 (0.012) 2.193 (2.858) 0.002 (0.004)
Image c1n0 c1n1 c1n2
Method: S M H S M H S M H
RMSE
t
(mm) 1.072 (0.410) 2.276 (1.706) 0.609 (0.275) 0.748 (0.127) 1.014 (1.006) 0.207 (0.097) 1.002 (0.131) 1.222 (0.837) 0.426 (0.137)
RMSE
r
(°) 1.442 (0.156) 3.405 (2.405) 1.408 (0.790) 1.204 (0.287) 1.639 (0.816) 1.113 (0.759) 1.286 (0.594) 2.289 (1.212) 1.230 (0.315)
Image c2n0 c2n1 c2n2
Method: S M H S M H S M H
RMSE
t
(mm) 1.411 (0.044) 0.396 (1.609) 0.175 (0.051) 1.273 (0.042) 0.406 (0.317) 0.332 (0.059) 1.541 (0.063) 0.275 (0.191) 0.234 (0.122)
RMSE
r
(°) 1.738 (0.055) 0.375 (3.674) 0.275 (0.101) 1.499 (0.081) 0.827 (0.642) 0.207 (0.189) 1.740 (0.058) 0.665 (0.220) 0.346 (0.303)
The interquarter ranges are shown in parentheses
Med Biol Eng Comput
123
interpolation artifact, which can lead to convergence in
local extremes during the optimization process [24]. Sat-
isfactory success rates were obtained only in the synthetic
image series. As the rotational degrees increasingly devi-
ated from 0°to 30°in the c1 image series, the number of
successful trials decreased from 86, 69, 65, to 33 (out of 90
trials), suggesting that the MI-based method is less adap-
tive to large rotational deviations. To clarify how the noise
influenced the MI cost function, we used the simulated s1
image series in which the ground truths of transformation
parameters were all zeros. We examined the changes of MI
values by varying a translational or a rotational parameter
away from zero while keeping the remaining transforma-
tion parameters unchanged. We found that the different
noise levels resulted in different MI values where the
higher levels of noise produced lower MI values. However,
the largest MI value was still produced by the ground truth,
indicating that the MI-based registration performed well
under different noise levels.
The reasons for failures with surface- and MI-based
registrations were different. In many cases, the surface-
based method failed simply because the error slightly
exceeded the predefined thresholds, because of distortion
of the delineated surface. However, most MI-based regis-
tration failures came from trapping in a local extreme,
which may be distant from the ground truth. The existence
of many local extremes also resulted in large distribution
errors in the MI-based results (Fig. 8b, c), which produced
the largest interquarter ranges of RMSEs in c1 and c2
image series (see Table 1). The properties of these two
similarity measures, estimated with the evaluation protocol
proposed by S
ˇkerl et al. [36], further supported our
observations. The risk of nonconvergence of the MI-based
cost function is larger than that of the surface-based one,
either in the vicinity of, or distant from, the ground truth
(see Fig. 1), showing that the MI-based method has more
difficulty converging to the ground truth for accurate reg-
istrations. In contrast, a larger value of the capture range
for the surface-based cost function was obtained because of
its monotonically increasing behavior with distance from
the ground truth.
The distinct limitations of the surface- and MI-based
methods can be compensated for with the hybrid-based
method, which produced the best results—with success rates
greater than 98%. The significant increase of the success rate
from the MI-based method to the hybrid-based method indi-
cates that most of the failed MI-based registrations recovered
after adding the surface-based method as a precursor. This
demonstrates that the surface-based method could guide the
estimations toward the vicinity of ground truth, escaping the
local extremes in the MI-based optimization process. Once the
estimations were close to the solution, MI-based registration
converges to the correct position with better precision. As
demonstrated in Fig. 8c, the errors resulting from the surface-
based method, which were slightly larger than those from the
MI-based method, were further reduced after the MI stage in
the hybrid-based method.
In comparing the speed of the surface- and MI-based
methods, the number of iterations was similar; however,
the MI-based computations consumed much more time.
We determined that only two to three iterations of the MI
calculation were required after the surface-based registra-
tion because this brought the parameters close to the true
solution. Therefore, the more the rotation deviates from the
true solution, the greater the computational cost savings
from the hybrid-based method—the hybrid-based method
saved 12–53% of the computations, compared with the
MI-based method when the rotational deviation ranged
from 0°to 30°, as shown in Fig. 9c. Our experimental
results, based on clinical data, suggest that, when the ori-
entational differences between two images are less than
10°, the computational time required by the MI-based
method was comparable to that of the surface-based
method, since the latter requires additional effort for sur-
face extraction. Otherwise, the use of the hybrid-based
method is recommended.
Fig. 9 The time required to co-register the as1, bc1, and cc2 image
series for the surface-, MI-, and hybrid-based methods at different
degrees of deviations. Note that the time unit is normalized according
to the observation that one MI-based iteration takes about 60 and 72
times more than the surface-based one at low and high resolution,
respectively
Med Biol Eng Comput
123
In the cases of brain tumors, previous studies had
demonstrated satisfactory results of co-registering SPECT
and MR images using either the surface-based method [9]
or the MI-based method [38]. Itti et al. used the surface-
based method to register the MR with SPECT images with
brain tumors, and the results were promising although a
semiautomatic segmentation by means of manual threshold
adjustments may be required [9]. Moreover, Soma et al.’s
study has reported that the MI-based method was effective
to register the MR images and SPECT images with brain
tumor [38]. These studies indicated that, as long as the
segmented surfaces from SPECT and MR images were
consistent no matter if there were tumors, the surface-based
method can result in an approximate initialization for the
subsequent MI-based method that can further achieve a
successful registration. In the future study, the hybrid-
based method is planned to be applied to the clinical
pathological data, including tumor, strokes, and other
macroscopic brain disorders.
In conclusion, we compared the performances of the
surface-based, MI-based, and the hybrid-based methods for
co-registration of SPECT and MR images. Experiments
using synthetic and real data demonstrated that the hybrid-
based scheme, surface-based registration, followed by MI-
based registration, reduced the computational load by at
least 12% compared with the MI-based method and
maintained the subvoxel accuracy (the averaged RMSE
t
and RMSE
r
values were no more than 1 mm and 2°,
respectively) and high success rates (greater than 98%). In
general, the use of the hybrid-based method is suggested
especially when the orientational differences between the
floating and reference images are greater than 10°.
Acknowledgments The authors express gratitude to Dr. Berengere
Aubert-Broche for providing a series of synthetic images, and Shih-
Pei Chen for his help on data acquisition and comments. Our gratitude
also goes to Jaya Ramchandani for his assistance in English language
editing. This study was partially supported by the Taipei Veterans
General Hospital (V96 ER1-005), and the National Science Council
of Taiwan (NSC 95-2752-B-075-001-PAE, NSC 95-2752-B-010-006-
PAE, NSC 96-2752-B-075-001-PAE, NSC 96-2752-B-010-006-PAE,
NSC 97-2752-B-075-001-PAE, NSC 97-2752-B-010-001-PAE, NSC
98-2752-B-075-001-PAE, and NSC 98-2752-B-010-001-PAE).
References
1. Aubert-Broche B, Grova C, Reilhac A, Evans, AC Collins DL
(2006) Realistic simulated MRI and SPECT databases. Applica-
tion to SPECT/MRI registration evaluation. In: Larsen R, Nielsen
M, Sporring J (eds) 9th International conference on medical
image computing and computer assisted intervention (MICCAI),
LNCS 4190. Springer, Berlin, pp 330–337
2. Barnden L, Kwiatek R, Lau Y, Hutton B, Thurfjell L, Pile K,
Rowe C (2000) Validation of fully automatic brain SPET to MR
co-registration. Eur J Nucl Med 27:147–154
3. Borgefors G (1986) Distance transformations in digital images.
Comput Vis Graph Image Process 34:344–371
4. Chen H-M, Varshney PK (2003) Mutual information-based
CT-MR brain image registration using generalized partial volume
joint histogram estimation. IEEE Trans Med Imaging 22:
1111–1119
5. Cover TM, Thomas JA (1991) Elements of information theory.
Wiley, New York
6. de Munck JC, Verster FC, Dubois EA, Habraken JBA, Boltjes B,
Claus JJ, van Herk M (1998) Registration of MR and SPECT
without using external fiducial markers. Phys Med Biol 43:1255–
1269
7. Eldeib AM, Yamany SM, Farag AA (1999) Multi-modal medical
volumes fusion by surface matching. In: Taylor CJ, Colchester
ACF (eds) 2nd International conference on medical image com-
puting and computer assisted intervention (MICCAI), LNCS
1679. Springer, Berlin, pp 672–679
8. Hajnal JV, Saeed N, Soar EJ, Oatridge A, Young IR, Bydder GM
(1995) A registration and interpolation procedure for subvoxel
matching of serially acquired MR images. J Comput Assist
Tomogr 19:289–296
9. Itti L, Chang L, Mangin JF, Darcourt J, Ernst T (1997) Robust
multimodality registration for brain mapping. Hum Brain Mapp
5:3–17
10. Kadrmas DJ, Frey EC, Tsui BM (1997) Analysis of the recon-
structibility and noise properties of scattered photons in
99m
Tc
SPECT. Phys Med Biol 42:2493–2516
11. Kagadis GC, Delibasis KK, Matsopoulos GK, Mouravliansky NA,
Asvestas PA, Nikiforidis GC (2002) A comparative study of sur-
face- and volume-based techniques for the automatic registration
between CT and SPECT brain images. Med Phys 29:201–213
12. Kittler J, Illingworth J (1986) Minimum error thresholding. Pat-
tern Recognit 19:41–47
13. Koole M, D’Asseler Y, Van Laere K, Van de Walle R, Van de
Wiele C, Lemahieu I, Dierckx RA (1999) MRI-SPET and SPET-
SPET brain co-registration: evaluation of the performance of
eight different algorithms. Nucl Med Commun 20:659–669
14. Lee J-D, Huang C-H, Weng Y-H, Lin K-J, Chen C-T (2007) An
automatic MRI/SPECT registration algorithm using image
intensity and anatomical feature as matching characters: appli-
cation on the evaluation of Parkinson’s disease. Nucl Med Biol
34:447–457
15. Lee H, Lee J, Kim N, Lyoo IK, Shin YG (2009) Robust and fast
shell registration in PET and MR/CT brain images. Comput Biol
Med 39:961–977
16. Liao Y-L, Sun Y-N, Guo W-Y, Chou Y-H, Hsieh J-C, Wu Y-T
(2007) A comparison between the surface-based and mutual-
information-based methods for co-registering SPECT and MR
images. In: Proceedings of the 29th IEEE annual international
conference of the IEEE Engineering in Medicine and Biology
Society, IEEE, New York, pp 868–871
17. Lohmann G (1998) Volumetric image analysis. Wiley, New York
18. Lu X, Zhang S, Su H, Chen Y (2008) Mutual information-based
multimodal image registration using a novel joint histogram
estimation. Comput Med Imaging Graph 32:202–209
19. Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P
(1997) Multimodality image registration by maximization of
mutual information. IEEE Trans Med Imaging 16:187–198
20. Maes F, Vandermeulen D, Suetens P (2003) Medical image
registration using mutual information. Proc IEEE 91:1699–1722
21. Malison RT, Miller EG, Greene R, McCarthy G, Charney DS,
Innis RB (1993) Computer-assisted coregistration of multislice
SPECT and MR brain images by fixed external fiducials.
J Comput Assist Tomogr 17:952–960
22. Otsu N (1979) Threshold selection method from gray-level his-
tograms. IEEE Trans Syst Man Cybern 9:62–66
Med Biol Eng Comput
123
23. Pfluger T, Vollmar C, Wismuller A, Dresel S, Berger F, Suntheim
P, Leinsinger G, Hahn K (2000) Quantitative comparison of
automatic and interactive methods for MRI-SPECT image reg-
istration of the brain based on 3-dimensional calculation of error.
J Nucl Med 41:1823–1829
24. Pluim JPW, Maintz JBA, Viergever MA (2000) Interpolation
artefacts in mutual information-based image registration. Comput
Vis Image Underst 77:211–232
25. Pluim JPW, Maintz JBA, Viergever MA (2003) Mutual-infor-
mation-based registration of medical images: a survey. IEEE
Trans Med Imaging 22:986–1004
26. Pohjonen H, Nikkinen P, Sipila
¨O, Launes J, Salli E, Salonen O,
Karp P, Yla
¨-Ja
¨a
¨ski J, Katila T, Liewendahl K (1996) Registration
and display of brain SPECT and MRI using external markers.
Neuroradiology 38:108–114
27. Press WH (1992) Numerical recipes in C: the art of scientific
computing. Cambridge University Press, Cambridge
28. Ridler TW, Calvard S (1978) Picture thresholding using an iter-
ative selection method. IEEE Trans Syst Man Cybern 8:630–632
29. Rohde GK, Aldroubi A, Healy DM (2009) Interpolation artifacts in
sub-pixel image registration. IEEE Trans Image Process
18:333–345
30. Salvado O, Wilson DL (2007) Removal of local and biased global
maximain intensity-basedregistration.Med Image Anal 11:183–196
31. Seppa M (2008) Continuous sampling in mutual-information
registration. IEEE Trans Image Process 17:823–826
32. Sezgin M, Sankur B (2004) Survey over image thresholding
techniques and quantitative performance evaluation. J Electron
Imaging 13:146–165
33. Shukla SS, Honeyman JC, Crosson B, Williams CM, Nadeau SE
(1992) Method for registering brain SPECT and MR images.
J Comput Assist Tomogr 16:966–970
34. Sinha TK, Cash DM, Weil RJ, Galloway RL, Miga MI (2002)
Cortical surface registration using texture mapped point clouds
and mutual information. In: Dohi T, Kikinis R (eds) 5th Inter-
national conference on medical image computing and computer
assisted intervention (MICCAI), LNCS 2489. Springer, Berlin,
pp 533–540
35. Sipila O, Nikkinen P, Pohjonen H, Poutanen VP, Visa A, Savo-
lainen S, Katila T, Liewendahl K (1997) Accuracy of a regis-
tration procedure for brain SPET and MRI: phantom and
simulation studies. Nucl Med Commun 18:517–526
36. S
ˇkerl D, Likar B, Pernus F (2006) A protocol for evaluation of
similarity measures for rigid registration. IEEE Trans Med
Imaging 25:779–791
37. Slomka PJ, Baum RP (2009) Multimodality image registration
with software: state-of-the-art. Eur J Nucl Med Mol Imaging
36:S44–S55
38. Soma T, Takaki A, Teraoka S, Ishikawa Y, Murase K, Koizumi K
(2008) Behaviors of cost functions in image registration between
201
Tl brain tumor single-photon emission computed tomography
and magnetic resonance images. Ann Nucl Med 22:825–832
39. Thacker N, Vokurka E, Jackson A (1999) Re-normalised sinc
interpolation for rapid reslicing of MRI data. In: ISMRM seventh
meeting proceedings, p 181
40. Thurfjell L, Lau YH, Andersson JL, Hutton BF (2000) Improved
efficiency for MRI-SPET registration based on mutual informa-
tion. Eur J Nucl Med 27:847–856
41. Townsend DW (2008) Multimodality imaging of structure and
function. Phys Med Biol 53:R1–R39
42. Tsao J (2003) Interpolation artifacts in multimodality image
registration based on maximization of mutual information. IEEE
Trans Med Imaging 22:854–864
43. Vajda I (1989) Theory of statistical inference and information.
Kluwer Academic Publishers, Dordrecht
44. Wang YL, Chiang MC, Thompson PM (2005) Mutual informa-
tion-based 3D surface matching with applications to face recog-
nition and brain mapping. In: Proceedings of the 10th IEEE
international conference on computer vision, IEEE Computer
Society Press, Los Alamitos, pp 527–534
45. West J, Fitzpatrick JM, Wang MY, Dawant BM, Maurer CR,
Kessler RM, Maciunas RJ (1999) Retrospective intermodality
registration techniques for images of the head: surface-based
versus volume-based. IEEE Trans Med Imaging 18:144–150
46. Wong JC, Studholme C, Hawkes DJ, Maisey MN (1997) Eval-
uation of the limits of visual detection of image misregistration in
a brain fluorine-18 fluorodeoxyglucose PET-MRI study. Eur J
Nucl Med 24:642–650
47. Xie XL, Beni G (1991) A validity measure for fuzzy clustering.
IEEE Trans Pattern Anal Mach Intell 13:841–847
48. Yokoi T, Soma T, Shinohara H, Matsuda H (2004) Accuracy and
reproducibility of co-registration techniques based on mutual
information and normalized mutual information for MRI and
SPECT brain images. Ann Nucl Med 18:659–667
49. Zaidi H, Montandon M-L, Alavi A (2008) The clinical role of
fusion imaging using PET, CT, and MR imaging. PET Clin
3:275–291
50. Zhu YM, Cochoff SM (2002) Influence of implementation
parameters on registration of MR and SPECT brain images by
maximization of mutual information. J Nucl Med 43:160–166
Med Biol Eng Comput
123
... Various authors have combined two or more registration methodologies/strategies in their algorithms (Andreetto et al., 2004;Auer et al., 2005;Chen et al., 2010;Christensen et al., 1997;Davatzikos et al., 1996;Kim et al., 2003;Liao et al., 2011). Some use feature and intensity based registration methodologies concurrently. ...
... A common solution is the use of a feature based algorithm for a coarse registration and then the use of an intensity based methodology for a fine registration as described in (Chen et al., 2010;Liao et al., 2011;Postelnicu et al., 2009). For example, in (Postelnicu et al., 2009), to optimally register volumetric brain images, relevant geometrical information is initially extracted from the segmented surfaces of cortical and subcortical structures, and afterwards the surfaces are registered and the deformation found is applied to the rest of the volume data. ...
Article
Full-text available
This paper presents a review of automated image registration methodologies that have been used in the medical field. The aim of this paper is to be an introduction to the field, provide knowledge on the work that has been developed and to be a suitable reference for those who are looking for registration methods for a specific application. The registration methodologies under review are classified into intensity or feature based. The main steps of these methodologies, the common geometric transformations, the similarity measures and accuracy assessment techniques are introduced and described.
... Hence we selected related work close to our problem which involves rigid transformation and cochlea images. However there is some interesting work which can be used for problems involving non-rigid image registration and different datasets as in [31][32][33][34][35]. Deep learning methods are increasing in popularity in medical image registration e.g. as in [36,37]. ...
Article
Full-text available
Background: The postoperative imaging assessment of Cochlear Implant (CI) patients is imperative. The main obstacle is that Magnetic Resonance imaging (MR) is contraindicated or hindered by significant artefacts in most cases with CIs. This study describes an automatic cochlear image registration and fusion method that aims to help radiologists and surgeons to process pre-and postoperative 3D multimodal imaging studies in cochlear implant (CI) patients. Methods and findings: We propose a new registration method, Automatic Cochlea Image Registration (ACIR-v3), which uses a stochastic quasi-Newton optimiser with a mutual information metric to find 3D rigid transform parameters for registration of preoperative and postoperative CI imaging. The method was tested against a clinical cochlear imaging dataset that contains 131 multimodal 3D imaging studies of 41 CI patients with preoperative and postoperative images. The preoperative images were MR, Multidetector Computed Tomography (MDCT) or Cone Beam Computed Tomography (CBCT) while the postoperative were CBCT. The average root mean squared error of ACIR-v3 method was 0.41 mm with a standard deviation of 0.39 mm. The results were evaluated quantitatively using the mean squared error of two 3D landmarks located manually by two neuroradiology experts in each image and compared to other previously known registration methods, e.g. Fast Preconditioner Stochastic Gradient Descent, in terms of accuracy and speed. Conclusions: Our method, ACIR-v3, produces high resolution images in the postoperative stage and allows for visualisation of the accurate anatomical details of the MRI with the absence of significant metallic artefacts. The method is implemented as an open-source plugin for 3D Slicer tool.
... Below there are some examples of image registration for dierent organs of the human body. The most popular in this respect are: brain [14,17], heart [12], lung [16,20], retina [15], breast [13], bones [9,10] and knee [22,25]. ...
Chapter
Full-text available
The main aim of this research is presenting an automated image matching methodology being used in the field of medicine for inter- and intraobjectional image matching. This paper shows a different approach avoiding the standard procedures associated with performing four main steps of the registration process: feature detection, feature matching, mapping function design and image transformation with resampling, and replacing them with the fuzzy image concept combined with the use of similarity measures. This methodology has been implemented in MATLAB and tested on clinical T1- and T2-weighted magnetic resonance imaging (MRI) slices of the knee joint in coronal and sagittal plane.
... To this end, Mutual Information (MI) based algorithms [21][22][23] are widely used to find the best co-registration transformation. In particular, MI was demonstrated to be optimal for multimodal data analysis where images recorded from different devices or with different contrasts, such as fMRI and MRI, must be aligned in a common space, whereas many other mathematical cost functions can be misleading when handling images with different SNR and sensitivity [24]. Of note, this approach was tested for co-registering fMRI and MRI images with high SNR and contrast on the whole brain volume. ...
Article
Full-text available
The prototypes of ultra-low-field (ULF) MRI scanners developed in recent years represent new, innovative, cost-effective and safer systems, which are suitable to be integrated in multi-modal (Magnetoencephalography and MRI) devices. Integrated ULF-MRI and MEG scanners could represent an ideal solution to obtain functional (MEG) and anatomical (ULF MRI) information in the same environment, without errors that may limit source reconstruction accuracy. However, the low resolution and signal-to-noise ratio (SNR) of ULF images, as well as their limited coverage, do not generally allow for the construction of an accurate individual volume conductor model suitable for MEG localization. Thus, for practical usage, a high-field (HF) MRI image is also acquired, and the HF-MRI images are co-registered to the ULF-MRI ones. We address here this issue through an optimized pipeline (SWIM—Sliding WIndow grouping supporting Mutual information). The co-registration is performed by an affine transformation, the parameters of which are estimated using Normalized Mutual Information as the cost function, and Adaptive Simulated Annealing as the minimization algorithm. The sub-voxel resolution of the ULF images is handled by a sliding-window approach applying multiple grouping strategies to down-sample HF MRI to the ULF-MRI resolution. The pipeline has been tested on phantom and real data from different ULF-MRI devices, and comparison with well-known toolboxes for fMRI analysis has been performed. Our pipeline always outperformed the fMRI toolboxes (FSL and SPM). The HF–ULF MRI co-registration obtained by means of our pipeline could lead to an effective integration of ULF MRI with MEG, with the aim of improving localization accuracy, but also to help exploit ULF MRI in tumor imaging.
Article
We propose a new two-dimensional structural representation method for registration of multimodal images by using the local structural symmetry of images, which is similar at different modalities. The symmetry is measured in various orientations and the best is mapped and used for the representation image. The optimum performance is obtained when using only two different orientations, which is called binary dominant symmetry representation (BDSR). This representation is highly robust to noise and intensity non-uniformity. We also propose a new objective function based on L2 distance with low sensitivity to the overlapping region. Then, five different meta-heuristic algorithms are comparatively applied. Two of them have been used for the first time on image registration. BDSR remarkably outperforms the previous successful representations, such as entropy images, self-similarity context, and modality-independent local binary pattern, as well as mutual information-based registration, in terms of success rate, runtime, convergence error, and representation construction.
Thesis
Ziel dieser Arbeit ist die Untersuchung und Weiterentwicklung von Ähnlichkeitsmaßen zur Registrierung von multimodalen medizinischen Bilddaten wie sie zum Beispiel in der Planung und Diagnose von klinischen Eingriffen zum Einsatz kommen. Hierbei wird zunächst eine Methode mit einem anisotropen Regularisierungsterm zur elastischen Registrierung von EPI-Daten vorgestellt, der speziell für das zu Grunde liegende Problem entwickelt wurde und dadurch genauere Registrierungsergebnisse ermöglicht. Außerdem werden anhand von rigiden Problemstellungen zwei Konzepte zur Registrierung entwickelt, bei denen zwei Maße zur Messung der Ähnlichkeit kombiniert werden. Durch die vorgestellte Linearkombination zweier Maße, die auf verschiedenen Bildmerkmalen beruhen, wird ein Ähnlichkeitsmaß erzeugt, das vielseitiger einsetzbar ist als die einzelnen Maße. Durch das in dieser Arbeit entwickelte Konzept einer Homotopie von Ähnlichkeitsmaßen, bei dem die Maße durch eine dynamische Gewichtungsfunktion verknüpft werden, lassen sich dagegen die Stärken zweier Maße kombinieren, die diese in unterschiedlichen Phasen der Optimierung aufweisen. Im hier präsentierten Fall kann somit eine kürzere Laufzeit und eine erhöhte Robustheit gegenüber Fehlregistrierungen bei gleicher Genauigkeit erzielt werden. Für die Evaluation der elastischen Registrierungsmethoden wird in dieser Arbeit zusätzlich ein neues Maß entwickelt und diskutiert, das durch eine automatische und damit objektive Auswertung der Registrierungsergebnisse einen Vergleich der Ergebnisse verschiedener Methoden ermöglicht.
Article
PET/CT and SPECT/CT, as multi-modality imaging techniques, have been widely used in routine clinical works. The application of commercial PET/MRI imaging scanner in clinical has also been approved. SPECT is characterized with high sensitivity and wide application, together with the extensive application and easy accessibility of its radiolabeled tracers. SPECT/MRI, which combines SPECT with high resolution MRI, will be widely used in the future. The research progresses in these fields mainly focused on systematic structure design, image co-registration and molecular nanoparticles used in animal experiments were reviewed in this paper.
Article
In cranioplasty, neurosurgeons use bone grafts to repair skull defects. To ensure the protection of intracranial tissues and recover the original head shape for aesthetic purposes, a custom-made pre-fabricated prosthesis must match the cranial incision as closely as possible. In our previous study (Liao et al. in Med Biol Eng Comput 49:203-211, 2011), we proposed an algorithm consisting of the 2D snake and image registration using the patient's own diagnostic low-resolution and defective high-resolution computed tomography (CT) images to repair the impaired skull. In this study, we developed a 3D multigrid snake and employed multiresolution image registration to improve the computational efficiency. After extracting the defect portion images, we designed an image-trimming process to remove the bumped inner margin that can facilitate the placement of skull implants without manual trimming during surgery. To evaluate the performance of the proposed algorithm, a set of skull phantoms were manufactured to simulate six different conditions of cranial defects, namely, unilateral, bilateral, and cross-midline defects with 20 or 40 % skull defects. The overall image processing time in reconstructing the defect portion images can be reduced from 3 h to 20 min, as compared with our previous method. Furthermore, the reconstruction accuracies using the 3D multigrid snake were superior to those using the 2D snake.
Article
Full-text available
One of the future-oriented areas of medical image processing is to develop fast and exact algorithms for image registration. By joining multi-modal images we are able to compensate the disadvantages of one imaging modality with the advantages of another modality. For instance, a Computed Tomography (CT) image containing the anatomy can be combined with metabolic information of a Positron Emission Tomography (PET) image. It is quite conceivable that a patient will not have the same position in both imaging systems. Furthermore some regions for instance in the abdomen can vary in shape and position due to different filling of the rectum. So a multi-modal image registration is needed to calculate a deformation field for one image in order to maximize the similarity between the two images, described by a so-called distance measure. In this work, we present a method to adapt a multi-modal distance measure, here mutual information (MI), with weighting masks. These masks are used to enhance relevant image structures and suppress image regions which otherwise would disturb the registration process. The performance of our method is tested on phantom data and real medical images.
Conference Paper
Full-text available
This paper describes the construction of simulated SPECT and MRI databases that account for realistic anatomical and functional variability. The data is used as a gold-standard to evaluate four SPECT/MRI similarity-based registration methods. Simulation realism was accounted for using accurate physical models of data generation and acquisition. MRI and SPECT simulations were generated from three subjects to take into account inter-subject anatomical variability. Functional SPECT data were computed from six functional models of brain perfusion. Previous models of normal perfusion and ictal perfusion observed in Mesial Temporal Lobe Epilepsy (MTLE) were considered to generate functional variability. We studied the impact noise and intensity non-uniformity in MRI simulations and SPECT scatter correction may have on registration accuracy. We quantified the amount of registration error caused by anatomical and functional variability. Registration involving ictal data was less accurate than registration involving normal data. MR intensity non-uniformity was the main factor decreasing registration accuracy. The proposed simulated database is promising to evaluate many functional neuroimaging methods, involving MRI and SPECT data.
Article
Multimodality image registration and fusion have a key role in routine diagnosis, staging, restaging, and the assessment of response to treatment, surgery, and radiotherapy planning of malignant disease. The complementarity between anatomic (CT and MR imaging) and molecular (SPECT and PET) imaging modalities is well established and the role of fusion imaging widely recognized as a central piece of the general tree of clinical decision making. Moreover, dual modality imaging technologies including SPECT/CT, PET/CT, and, in the future, PET/MR imaging, now represent the leading component of contemporary health care institutions. This article discusses recent advances in clinical multimodality imaging, the role of correlative fusion imaging in a clinical setting, and future opportunities and challenges facing the adoption of multimodality imaging.
Article
An object may be extracted from its background in a picture by theshold selection. Ideally, if the object has a different average gray level from that of its surrounding, the effect of thresholding will produce a white object with a black background or vice versa. In practice, it is often difficult, however, to select an appropriate threshold, and a technique is described whereby an optimum threshold may be chosen automatically as a result of an iterative process, successive iterations providing increasingly cleaner extractions of the object region. An application to low contrast images of handwritten text is discussed.
Article
Objective: Methods for automatically registering and reslicing MR images using an interpolation function that matches the structure of the image data are described. Materials and Methods: Phantom and human brain images were matched by rigid body rotations and translations in two and three dimensions using a least-squares optimization procedure. Subvoxel image shifts were produced with linear or sine interpolation. Results: The use of sine interpolation ensured that the repositioned images were faithful to the original data and enabled quantitative intensity comparisons to be made. In humans, image segmentation was vital to avoid extraneous soft tissue changes producing systematic errors in registration. Conclusion: The sinc-based interpolation technique enabled serially acquired MR images to be positionally matched to subvoxel accuracy so that small changes in the brain could be distinguished from effects due to misregistration. Index Terms: Image registration-Brain-Phantom-Magnetic resonance imaging. (C) Lippincott-Raven Publishers.