ArticlePDF Available

Coupled parametric active contours

Authors:

Abstract and Figures

We propose an extension of parametric active contours designed to track nonoccluding objects transiently touching each other, a task where both parametric and single level set-based methods usually fail. Our technique minimizes a cost functional that depends on all contours simultaneously and includes a penalty for contour overlaps. This scheme allows us to take advantage of known constraints on object topology, namely, that objects cannot merge. The coupled contours preserve the identity of previously isolated objects during and after a contact event, thus allowing segmentation and tracking to proceed as desired.
Content may be subject to copyright.
Coupled Parametric Active Contours
Christophe Zimmer, Member,IEEE,and
J.-C. Olivo-Marin, Senior Member,IEEE
Abstract—We propose an extension of parametric active contours designed to
track nonoccluding objects transiently touching each other, a task where both
parametric and single level set-based methods usually fail. Our technique
minimizes a cost functional that depends on all contours simultaneously and
includes a penalty for contour overlaps. This scheme allows us to take advantage
of known constraints on object topology, namely, that objects cannot merge. The
coupled contours preserve the identity of previously isolated objects during and
after a contact event, thus allowing segmentation and tracking to proceed as
desired.
Index Terms—Segmentation, tracking, active contours, topology, 2D video.
æ
1INTRODUCTION
ACTIVE contours provide a very effective and versatile framework
for image segmentation and object tracking [1], [2], [3], [4]. These
methods are based on the minimization of a cost functional, often
called energy, that contains data attachment terms related to the
position of the contour with respect to image features and
regularizing terms, such as penalties on the contour length or
area. This minimization proceeds iteratively, by letting the contour
move away from an initial guess according to an evolution
equation derived from the energy functional. The equilibrium
position of the contour then provides a segmentation of the image
which can be used as initial guess for the segmentation of the
following frame when tracking objects in image sequences.
Two main types of active contours exist according to whether
the contours are represented explicitly or implicitly. In the explicit
representation used by the original active contours (also called
snakes) [1], contours are parametric curves CðpÞ¼ðxðpÞ;yðpÞÞ, i.e.,
functions from a scalar interval ½a; bIR into the image domain
IR 2[1], [2], [3], [5], [6]. In the more recently popularized
implicit approach, contours are obtained as the zero level set
1ð0Þ¼fðx; yÞjðx; yÞ¼0gof a scalar function ðx; yÞdefined
over [7], [8], [4], [9]. These two types of active contours differ
considerably in their ability to handle object topology or multiple
objects, computational efficiency, ease of implementation, and user
interaction, as will be discussed in Section 2 below.
The applicative goal that motivated the present work is to track
mobile biological cells from videomicroscopy image sequences in
order to quantify their motion under varying experimental
conditions (mutants, drugs, chemotactic gradients, etc.). This task
requires both computational efficiency and the ability to control
object topology. Computational efficiency is needed because of the
large quantity of data to be analyzed (typically, several hundreds
to thousands of images per experiment with many experiments
needed to assess the effect of one condition or molecule). A control
of object topology is needed because moving cells frequently touch
each other in typical experiments. Since the intensities and textures
of touching cells are often (though not necessarily) similar, it is
useful, and often indispensable, to use the prior knowledge that
cells do not actually merge in order to obtain an acceptable
segmentation. Without this external constraint, most segmentation
methods will also merge the associated objects, even if the cells
were spatially isolated and segmented as distinct objects earlier in
the sequence. As will be argued in Section 2 below, existing
parametric or implicit active contour methods cannot address all of
these requirements simultaneously, mainly because level set
methods are slow, while existing parametric active contours
generally fail on touching objects.
Here, we present an extension of parametric active contours
that combines the ability to handle transiently touching objects and
exert topological control with computational efficiency and
simplicity. Section 2 discusses related work; Section 3 describes a
parametric active contour model for single (uncoupled) contours
adapted from [1] and [4]. In Section 4, we extend this model to
multiple coupled contours. Section 5 shows results on some
example data and Section 6 concludes the paper.
2RELATED METHODS
In this section, we discuss how existing active contours methods
fare with respect to the two applicative requirements mentioned
above, namely, computational speed and the ability to handle
multiple objects undergoing frequent contacts.
A prominent difference between explicit and implicit active
contours is the topological flexibility of level sets that stands in
contrast with the topological rigidity of parametric contours. A
standard parametric active contour [1], [2], [3], [10], [5], [6] consists
of a single (usually closed) connected curve, which cannot readily
split or merge with other contours unless relatively intricate
“surgical” procedures are used (e.g., [11]). If several parametric
contours are initialized in an image, these curves may evolve
independently and cross each other, leading to multiply labeled
regions and incorrect segmentations [12]. In contrast, the level set
approach [7], [8], [13], [4], [9] can represent an arbitrary number of
disjoint contours by a single function . The contour evolution
equation is replaced by an evolution equation for .Asevolves,
the contours 1ð0Þcan split and merge automatically, while
intersecting contours and multiple labels are ruled out by
construction. Level set methods are often considered superior to
parametric active contours, to a large extent because of this
topological flexibility. Indeed, since an arbitrary initial contour (or
a set of contours) can, in principle, turn into any number of object
contours, initialization can be entirely automatic. For the same
reason, objects not present on the first frame of a sequence can be
detected at later stages without user assisted reinitialization.
Finally, the method naturally adapts to objects that change
topology over time, such as dividing cells or merging vesicles.
However, this flexibility can turn into a weakness in situations
where a priori information on topology is needed to overcome
ambiguities in interpreting the image. This was recently recog-
nized [14] in the context of static brain image segmentation, where
standard, topologically unconstrained level set techniques tend to
produce spurious connected components at object boundaries
where image information is weak. To produce segmentations
consistent with a previously known topology, a topology preser-
ving extension of the level set methodology was proposed and
shown to improve the segmentation of either single objects such as
cortical white matter or separate objects of similar image
characteristics such as touching bone cells [14].
Another limitation concerns all implicit methods that employ
only one level set function [8], [4], [14]. As we pointed out in [15],
these methods are suboptimal for segmenting objects with distinct
image characteristics, for instance, fluorescent cells with different
intensity levels, because the image-dependent energy terms cannot
1838 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 27, NO. 11, NOVEMBER 2005
.The authors are with the Quantitative Image Analysis Group, Institut
Pasteur, 25 rue du Docteur Roux, 75015 Paris, France.
E-mail: {czimmer, jcolivo}@pasteur.fr.
Manuscript received 4 Aug. 2004; revised 5 Mar. 2005; accepted 9 Mar. 2005;
published online 14 Sept. 2005.
Recommended for acceptance by P. Torr.
For information on obtaining reprints of this article, please send e-mail to:
tpami@computer.org, and reference IEEECS Log Number TPAMI-0405-0804.
0162-8828/05/$20.00 ß2005 IEEE Published by the IEEE Computer Society
be tuned to individual objects (unless the contours are extracted at
each evolution time step to allow identification of connected
components, as in [16]). This is in contrast with parametric
contours, which can be used straightforwardly to minimize
independently defined energy functionals specific to individual
objects. This limitation can also be overcome within the implicit
framework by using multiple level set functions i;i ¼1...N[17],
[18], [19], [20], [15]. Multiple level set functions allow improved
segmentation results by taking into account specific characteristics
of individual objects [19], [15] or object classes [17], [18], [20].
However, if the different ievolve independently of each other,
their zero level sets are likely to intersect and create multiply
labeled regions as in the case of parametric active contours. This is
clearly undesired when segmenting nonoccluding objects. Vese
and Chan [18] have proposed a method using vector-valued labels
which allows Nphases to be represented efficiently by only
log2ðNÞlevel set functions and guarantees a perfect partition of the
image. However, these phases correspond to classes of similar
intensities or textures, thus, objects sharing similar image
characteristics (e.g., cells with similar levels of fluorescence) are
treated as a single phase and topological constraints between them
cannot be enforced. Note that the same limitation concerns other
labeling techniques quite generally, even those in which the
number of classes is fixed by the user.
Most other multiple level set techniques [17], [19], [15] represent
Nclasses by Nlevel set functions and introduce additional
constraints in order to ensure, at least approximately, a partition of
the image into nonoverlapping domains. These coupled level set
methods are appropriate to address the difficulty posed by
touching nonoccluding objects, whether these have similar image
characteristics or not [15]. However, the use of multiple functions
ifurther exacerbates the already large computational complexity
inherent to level set techniques. The most basic implementations
require updating over the whole image domain at every time
step of the discretized evolution equations. This is a huge
computational disadvantage compared to the parametric methods,
which mostly require updating the position of the contour control
points only. Several numerical schemes have been proposed to
reduce the computation time of level set methods, including
narrow band and fast marching [21], [22] or additive operator
splitting [23]. Despite substantial improvements in efficiency,
however, implicit active contours are generally still many times
if not orders of magnitude slower than corresponding parametric
active contours (see, e.g., [11]).
In addition, the explicit representation makes parametric
contours very amenable to user interaction, which is very helpful
to correct or guide the segmentation process when needed [1]. This
is less readily done with implicit methods since the contours must
first be extracted from the level set function. Finally, level set
methods require substantially more complex implementations for
2D images. These reasons justify the use of parametric active
contours, especially in applications where very large 2D image
sequences must be processed in reasonable time.
1
In the following,
we therefore propose a scheme that couples multiple parametric
active contours, thus combining the good segmentation properties
of the coupled level sets techniques with the simplicity and
efficiency of parametric methods.
3UNCOUPLED PARAMETRIC ACTIVE CONTOURS
In this section, we first consider single (uncoupled) parametric
active contours. Coupling will then be introduced in Section 4.
3.1 Gradient-Based Parametric Active Contours
We first briefly recall the original active contour model as
introduced by [1]. It is based on the following energy functional:
E0ðCÞ¼Z1
0
1
2@C
@p

2
þ1
2@2C
@p2

2
fðCðpÞÞ
"#
dp; ð1Þ
where and are constant positive parameters and fis an edge
map of the image, such as the gradient magnitude of a smoothed
version of the image intensity I. The last term in (1) is the data
attachment term, whose minimization tends to push the contour
toward pixels of high image gradient. The first two terms in (1) are
regularization terms, whose minimization makes the contour act as
a membrane that resists stretching (term) and a thin plate that
resists bending (term) [1]. The minimization of (1) is achieved by
letting the contour evolve from an initialization C0¼Cðt¼0Þ
according to the evolution equation [1]:
@C
@t ¼@2C
@p2@4C
@p4þrf: ð2Þ
When the contour attains a steady state ð@C
@t ¼0Þ, it satisfies the
Euler-Lagrange equation associated to the minimization of (1) and
achieves a segmentation of the image.
3.2 Region-Based Parametric Active Contours
Despite its success in many applications, the original snake [1]
suffers from well-known limitations, which include the need to
initialize the contour close to the actual boundary of the object and
the sensitivity of the segmentation to spurious edges generated by
noise. These two shortcomings stem partly from the fact that the
data attachment term in (1) only depends on the image gradient
along the contour and is thus “blind” to the rest of the image.
Sensitivity to initialization and noise can be greatly reduced by
using a data attachment term that depends on image quantities
integrated over the whole image domain , rather than on
quantities integrated only along the contour length. A particularly
appealing method of this type is the model of “active contours
without edges” proposed by Chan and Vese [4], based on the well-
known (piece-wise constant) Mumford-Shah functional [24] for
image segmentation.
In order to make the active contours robust to noise and
initialization, we therefore replace the gradient based data
attachment term of (1) by the two-region-based data attachment
terms of [4]. The energy functional then becomes:
E1ðC;c
in;c
outÞ¼1
2Z1
0
@C
@p

2
þ@2C
@p2

2
"#
dp
þin ZinsideðCÞ
Icin
ðÞ
2d
þout ZoutsideðCÞ
Icout
ðÞ
2d;
ð3Þ
where in and out are constant positive parameters, Iis the local
image intensity, d is the elementary surface, and cin ,cout are
unknown scalars. Using the Euler-Lagrange equations associated
to the minimization of E1, the following evolution equation can be
obtained:
@C
@t ¼@2C
@p2@4C
@p4in Icin
ðÞ
2out Icout
ðÞ
2
hi
@C
@p
n;ð4Þ
where nis the normal unit vector of Cpointing outward, assuming
that Cis parameterized counter-clockwise. The variables cin and
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 27, NO. 11, NOVEMBER 2005 1839
1. For 3D images, we prefer level set-based methods, as described in [26].
cout are given by the average intensity inside and outside,
respectively, of the current contour C:
cinðtÞ¼<I>
insideðCÞ;coutðtÞ¼<I>
outsideðCÞ;ð5Þ
where we use the notation <I>
A¼RAId

=RAd

for averages
of the intensity over a region A.
Thanks to the region-based data attachment terms in (3) and (5),
this model is able to segment objects with ill-defined boundaries,
such as fluorescent cells in noisy microscopy images, requiring
only a difference in mean intensities between the objects and the
background (see Section 5). A welcome property of the model is
that these mean intensities need not be specified a priori, but are
self-consistently computed (5) from the minimization of (3).
The surface integrals in (5) apparently imply a significant
computational cost compared to the gradient oriented active
contours of Section 3.1, making the efficiency advantage over
implicit methods less obvious. In practice, however, this additional
cost need not be paid. First, although the optimal values for cin and
cout are unknown at the onset, their final values after segmentation
of the first frame provide an excellent guess for the following
frames because object and background intensities change little
across consecutive frames. These coefficients can thus be held
constant during processing of all but the first frame or be updated
only after contour convergence, once per frame, to accommodate
slow changes in background or object intensity. Second, one can
replace by a smaller domain !around the initial contour
Cðt¼0Þthat must only be large enough to contain the object
entirely. This further reduces the computation time of surface
integrals and has the additional welcome consequence that cout
better reflects the average background intensity in the vicinity of
the object—and, therefore, yields a better segmentation—if the
background intensity varies across the image. Finally, note that the
residual cost of processing the first frame could be further reduced
by converting the region integrals into contour integrals using
Green’s theorem, as done, e.g., in [10], [6]. This way, only
one region integration is required before segmentation rather than
at each evolution time step.
4COUPLED PARAMETRIC ACTIVE CONTOURS
Parametric active contour methods were originally used to
segment and track single objects [1], [3], [2]. The most straightfor-
ward extension to Nobjects is to initialize Nactive contours
C1;...;CNon each object and let each Ciminimize an energy
functional that depends on Cionly, e.g., by using (3) and (4) with
C¼C1;...;C¼CN. However, as illustrated in Section 5, these
independently evolving contours lead to undesired results when
previously distinct objects touch each other. To prevent contours
from crossing each other and absorbing touching objects, we define
a scheme where contours are coupled through the use of a single
energy function that depends on all Ncontours simultaneously:
E2ðC1;...;CN;c
in;1;...;c
in;N ;c
outÞ¼
1
2X
N
i¼1Z1
0
@Ci
@p

2
þ@2Ci
@p2

2
"#
dp
þin X
N
i¼1ZinsideðCiÞ
Icin;i

2d
þout ZoutsideðC1Þ\...\outsideðCNÞ
Icout
ðÞ
2d
þX
N
i¼1X
N
j¼iþ1ZinsideðCiÞ\insideðCjÞ
d:
ð6Þ
For ¼0, this functional is simply a generalization of (3) to
multiple contours. Except for the term weighted by out, it then
coincides with the sum of (3) for all individual contours,
i¼1...NE1ðCi;c
in;i;c
outÞ. For N¼1and ¼0, (6) reduces exactly
to (3). An advantage of using multiple active contours to represent
different objects, rather than a single level set function, is apparent
from the fact that the values cin;1;...;c
in;N are not necessarily equal.
As mentioned in Section 2 and [15], this allows better segmentation
of objects that have different average intensity levels. The term
weighted by out accounts for the fact that there is a single
background, which is now defined by the region exterior to all
Ncontours. The new term, weighted by >0, is equal to the
summed areas of pair-wise intersecting contour interiors and, thus,
penalizes contour overlaps. While this overlap penalty is similar to
that of some of the multiple level set methods discussed in Section
2 [17], [19], [15], its incorporation into the framework of parametric
active contours is new.
The Nevolution equations associated to the minimization of (6)
are as follows ði¼1::NÞ:
@Ci
@t ¼@2Ci
@p2@4Ci
@p4
in Icin;i

2out Icout
ðÞ
2þX
j¼1...N;ji
jðCiÞ
"#
@Ci
@p
ni;
ð7Þ
where niis the outward normal to contour Ciand jðxÞis
the set indicator function of the interior of Cj, that is,
jðxÞ¼1if x2insideðCjÞ, otherwise jðxÞ¼0. The coefficients
cin;1;...;c
in;N ;c
out evolve according to the following Nþ1equations:
cin;iðtÞ¼<I>
insideðCiÞ;i¼1...N
coutðtÞ¼<I>
outsideðC1Þ\...\outsideðCNÞ:ð8Þ
The interpretation of the new evolution equation (7) is straightfor-
ward when compared to its single contour equivalent (4): The term
associated to the overlap penalty (term) is zero along portions of
Cithat are not enclosed by any of the other contours; along
portions of Cilocated inside other contours, this term is a vector
that points inwards, causing the contour to recede from regions
occupied by others. The speed of this recession is proportional to
the number of contours that overlap each other, thus, multiple
overlaps will be reduced faster than simple overlaps.
The coupling between contours requires additional computa-
tions to evaluate j. In our implementation, jðCiðpÞÞ is determined
simply by counting the intersections of a ray emanating from each
control point CiðpÞwith the polygons formed by the control points
of the other contours Cj;j i[25]. This counting is performed only
for control points CiðpÞinside the bounding box of Cj, otherwise,
jðCiðpÞÞ is set to zero—this leads to a dramatic reduction of the
computational cost due to coupling. If needed, efficiency could be
further improved by computing jfrom binary masks associated to
each contour, provided that these masks are updated only at pixels
crossed by the contours during each evolution time step [10].
5EXPERIMENTS
In this section, we illustrate the application of the coupled active
contours on some examples. The first example (Fig. 1) shows the
evolution of three coupled contours in the absence of image data
and regularization, i.e., for ¼¼in ¼out ¼0; > 0. In this
case, the energy reduces to the overlap penalty. The contours do
not move except where they overlap each other; in the overlapping
regions, the contours recede from each other until they are disjoint,
as expected.
Fig. 2 illustrates how coupled active contours improve the
segmentation and tracking of two transiently touching objects. This
1840 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 27, NO. 11, NOVEMBER 2005
example sequence shows two objects that are isolated on the first
frame, touch each other in the next two frames, and are again
isolated in the fourth frame. In this simple situation, two
uncoupled parametric active contours lead to blatantly incorrect
segmentations (Fig. 2, rows 2 and 3): As soon as the objects touch
(time points t2and t3), the two contours overlap and encompass
both objects. This is expected, since the intensity difference
between the two objects is smaller than the intensity difference
between each object and the background. When the two objects
move apart (t4), each contour remains attached to both objects
simultaneously. Even worse, at the gap between the objects, each
contour intersects itself, fails to converge, and “blows apart,” as
apparent from Fig. 2, rows 2 and 3 at time t4(only intermediate
stages of contour evolution are shown for this time point). As a
result, tracking breaks down completely. In contrast, coupling the
active contours as described in Section 4 inhibits overlaps. As a
result, each contour remains attached to its legitimate object,
leading to much improved segmentations and tracking can
proceed as desired (Fig. 2, rows 4 and 5).
Fig. 3 shows the same data corrupted by noise. Again, the
uncoupled active contours fail (Fig. 3, rows 2-3). It is apparent
(rows 4-5) that the coupled active contour method produces almost
equally good results than on the noise-free data of Fig. 2. The only
noticeable degradation is a slight misplacement of the boundary
between the touching objects (compare Figs. 2 and 3 at time t3,
rows 4-5, and see Section 6).
An example on real data is given in Fig. 4. The images are taken
from a sequence obtained with a fluorescence microscope and
show two migrating cells that are initially distant, then touch, and
later move away from each other. Again, uncoupled contours
produce incorrect segmentations as soon as the cells touch (Fig. 4,
row 2, time t2) and tracking subsequently breaks down (time t3).
With coupled active contours, however, a satisfying segmentation
is obtained and cells are accurately tracked through the contact
event (Fig. 4, rows 4-5). With our implementation and for the
experiments reported here, processing times per frame (first frame
excepted) range from fractions of a second to 1 second on a Sun
Java workstation with two AMD Opteron 248 processors.
6CONCLUSION AND FUTURE WORK
We have presented an extension of parametric active contours to
track multiple objects that frequently touch each other as they
move. Our method couples multiple active contours in a scheme
that inhibits contour overlaps and maintains contours disjoint even
though the tracked objects enter in contact. The coupling constraint
is introduced directly as an overlap penalty into a single energy
function dependent on all contours (in contrast to earlier and less
general parametric coupling schemes [11], [12]). Although this can
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 27, NO. 11, NOVEMBER 2005 1841
Fig. 2. Segmentation of two colliding objects by coupled versus uncoupled active
contours. Row 1: Test image sequence (from left to right). Rows 2 and 3:
Segmentation by two uncoupled active contours. Rows 4 and 5: Segmentation by
two coupled active contours (¼0:1). Rows 2 and 4 show active contour a,
initialized on the bottom object. Rows 3 and 5 show active contour b, initialized on
the top object. Note how coupled active contours provide a satisfying segmenta-
tion, whereas uncoupled contours fail. (Intermediate stages of contour evolution
are shown for t4in rows 2-3 because no convergence is attained.) Other
parameters: ¼0,¼0:1,in ¼out ¼0:1. Image size: 250 250. Average
computation time per frame: 0.2s.
Fig. 3. Same as Fig. 2, but with Gaussian noise added to the images. Average
computation time per frame: 0.4s.
Fig. 1. Three contours evolving under the effect of the overlap penalty alone.
(a) Initial contours. (b) Intermediate stage. (c) Final contours. Image size:
400 400. Computation time: 0.6s.
also be achieved using multiple coupled level sets, the parametric
approach is preferable for processing large image sequences
thanks to its lower complexity. It is also more amenable to
allowing user interaction and easier to implement for 2D data. A
level set method such as [15] would, however, be useful as a
complementary procedure to automatically initialize the contours
and detect incoming objects at any time during the sequence.
Although our model maintains contours of touching objects
disjoint, it does not guarantee that these contours accurately trace
the interface between objects, as mentioned in the discussion of
Fig. 3 above. In cases of very unfavorable initialization, it can even
occur that one contour collapses while the other grows to enclose
both objects. In practice, this extreme situation is easily prevented
with the help of an additional penalty against excessive changes of
the surface enclosed by each contour. Accurate localization of the
interface for objects that can share very similar image character-
istics, however, will require more specific data attachment terms or
inclusion of stronger a priori constraints, for instance, on object
shape (e.g., [5], [9]). Such constraints can be easily integrated with
the coupled parametric active contours by adding new penalty
terms to the cost functional (6). Thus, the presented method
provides a basis for future work dedicated to segmentation and
tracking of multiple nonoccluding objects undergoing frequent
contacts. Although the primary applicative motivation of this work
is biological cell imaging, the approach could be of interest to other
applications where objects move in 2D images without over-
lapping each other, such as pedestrians or cars observed by a
camera looking to the ground.
ACKNOWLEDGMENTS
The authors would like to thank the three anonymous referees for
helpful comments, acknowledge useful discussions with Bo
Zhang, and thank Nancy Guille
´n, Evelyne Coudrier, and Franc¸ois
Amblard for the cell images in Fig. 4. This work was funded by
Institut Pasteur.
REFERENCES
[1] M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active Contour Models,”
Int’l J. Computer Vision, vol. 1, pp. 321-331, 1988.
[2] F. Leymarie and M.D. Levine, “Tracking Deformable Objects in the Plane
Using an Active Contour Model,” IEEE Trans. Pattern Analysis and Machine
Intelligence, vol. 15, no. 6, pp. 617-634, June 1993.
[3] A. Blake and M. Isard, Active Contours. Springer, 1998.
[4] T. Chan and L. Vese, “Active Contours without Edges,” IEEE Trans. Image
Processing, vol. 10, no. 2, pp. 266-277, 2001.
[5] N. Ray, S. Acton, and K. Ley, “Tracking Leukocytes in Vivo with Shape and
Size Constrained Active Contours,” IEEE Trans. Medical Imaging, vol. 21,
no. 10, pp. 1222-1235, Oct. 2002.
[6] M. Jacob, T. Blu, and M. Unser, “Efficient Energies and Algorithms for
Parametric Snakes,” IEEE Trans. Image Processing, vol. 13, no. 9, pp. 1231-
1244, Sept. 2004.
[7] S. Osher and J. Sethian, “Fronts Propagating with Curvature-Dependent
Speed: Algorithms Based on Hamilton-Jacobi Formulations,” J. Computa-
tional Physics, vol. 79, pp. 12-49, 1988.
[8] R. Malladi and J. Sethian, “A Unified Approach to Noise Removal, Image
Enhancement, and Shape Recovery,” IEEE Trans. Image Processing, vol. 11,
no. 5, pp. 1554-1568, 1996.
[9] N. Paragios, “A Level Set Approach for Shape-Driven Segmentation and
Tracking of the Left Ventricle,” IEEE Trans. Medical Imaging, vol. 22, no. 6,
pp. 773-776, June 2003.
[10] C. Chesnaud, P. Refregier, and V. Boulet, “Statistical Region Snake-Based
Segmentation Adapted to Different Physical Noise Models,” IEEE Trans.
Pattern Analysis and Machine Intelligence, vol. 21, no. 11, pp. 1145-1157, Nov.
1999.
[11] H. Delingette and J. Montagnat, “Shape and Topology Constraints on
Parametric Active Contours,” Computer Vision and Image Understanding,
vol. 83, pp. 140-171, 2001.
[12] C. Zimmer, E. Labruye
`re, V. Meas-Yedid, N. Guille
´n, and J.-C. Olivo-
Marin, “Segmentation and Tracking of Migrating Cells in Videomicroscopy
with Parametric Active Contours,” IEEE Trans. Medical Imaging, vol. 21,
pp. 1212-1221, 2002.
[13] N. Paragios and R. Deriche, “Geodesic Active Contours and Level Sets for
the Detection and Tracking of Moving Objects,” IEEE. Trans. Pattern
Analysis and Machine Intelligence, vol. 22, no. 3, pp. 266-280, Mar. 2000.
[14] X. Han, C. Xu, and J. Prince, “A Topology Preserving Level Set Method for
Geometric Deformable Models,” IEEE Trans. Pattern Analysis and Machine
Intelligence, vol. 25, no. 6, pp. 755-768, June 2003.
[15] B. Zhang, C. Zimmer, and J.-C. Olivo-Marin, “Tracking Fluorescent Cells
with Coupled Geometric Active Contours,” Proc. IEEE Int’l Symp.
Biomedical Imaging, pp. 476-479, 2004.
[16] T.B. Sebastian, H. Tek, J. Crisco, and B. Kimia, “Segmentation of Carpal
Bones from CT Images Using Skeletally Coupled Deformable Models,”
Medical Image Analysis, vol. 7, no. 1, pp. 21-45, Mar. 2003.
[17] C. Samson, L. Blanc-Fe
´raud, G. Aubert, and J. Zerubia, “A Level Set Model
for Image Classification,” Int’l J. Computer Vision, vol. 40, no. 3, pp. 187-198,
2000.
[18] L. Vese and T. Chan, “A Multiphase Level Set Framework for Image
Segmentation Using the Mumford and Shah Model,” Int’l J. Computer
Vision, vol. 50, no. 3, pp. 271-293, 2002.
[19] N. Paragios, “Coupled Geodesic Active Regions for Image Segmentation: A
Level Set Approach,” European Conf. Computer Vision, pp. 22–240, 2001.
[20] A. Yezzi, A. Tsai, and A. Willsky, “A Fully Global Approach to Image
Segmentation via Coupled Curve Evolution Equations,” J. Visual Comm. and
Image Representation, vol. 13, nos. 1/2, pp. 195-216, Mar./June 2002.
[21] D. Adalsteinsson and J. Sethian, “A Fast Level Set Method for Propagating
Interfaces,” J. Computational Physics, vol. 118, no. 2, pp. 269-277, 1995.
[22] J. Sethian, Level Set Methods and Fast Marching Methods. New York:
Cambridge Univ. Press, 1999.
[23] J. Weickert and G. Kuehne, “Fast Methods for Implicit Active Contour
Models,” Geometric Level Set Methods in Imaging, Vision, and Graphics,
S. Osher and N. Paragios, eds., chapter 3, pp. 43-57, Springer, 2003.
[24] D. Mumford and J. Shah, “Optimal Approximation by Piecewise Smooth
Functions and Associated Variational Problems,” Comm. Pure and Applied
Math., vol. 42, pp. 577-685, 1989.
[25] J. O’Rourke, Computational Geometry in C. Cambridge Univ. Press, 1993.
[26] A. Dufour, V. Shinin, S. Tajkhbaksh, N. Guille
´n-Aghion, J.-C. Olivo-Marin,
and C. Zimmer, “Segmenting and Tracking Fluorescent Cells in Dynamic
3-D Microscopy with Coupled Active Surfaces,” IEEE Trans. Image
Processing, vol. 14, no. 9, pp. 1396-1410, Sept. 2005.
1842 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 27, NO. 11, NOVEMBER 2005
Fig. 4. Fluorescent cells in transient contact tracked by coupled (rows 4-5) versus
uncoupled (rows 2-3) active contours. Intermediate contour evolution stages are
shown in rows 2 and 3 at time t3. Parameters: ¼0,¼0:1,in ¼out ¼1,
¼f0;0:1g. Image size: 255 256. Average computation time per frame: 0.9s.
... Traditional segmentation techniques based on deformable models [5]- [7] generalize poorly to brightfield images due to low contrast and blurred cell boundaries. The more recent techniques which are capable of handling intensity inhomogeneity [7], [8], are better suited for segmenting isolated objects, but are less effective for clustered cells. ...
... The final cell separation and smooth contours are obtained by appropriately combining the cell region and boundary predictions embedded in an active contour model. Coupled active contours [5], [23], which are specifically designed to prevent merging of adjacent parametric curves, is applied to obtain smooth boundaries of individual cells. ...
... The force function, further, compensates the false positives and false negatives in region prediction using the edge prediction and vice versa. Additionally, to prevent contour leakage due to inconsistent edge predictions, the model is implemented using the coupled active contour [5] available as a plugin in the open source bioimage analysis toolkit, Icy [23]. Convergence is achieved when the speed function restricts further curve propagation, or when the disjoint curves meet. ...
... This is mainly due to the fact that there are other organs adjacent and close to the liver which makes segmentation more difficult. In addition, the liver itself may also contain tumors such as hepatoma and haemagioma that can obscure the boundary [4]. In order to address these problems, we present an automatic CT liver image classification system which can be used to detect four types of liver tumor: hepatoma, haemagioma, cyst and cirrhosis. ...
Article
Full-text available
The detection and diagnose of liver tumors from CT images by using digital image processing, is a modern technique depends on using computer in addition to textural analysis to obtain an accurate liver diagnosis, despite the method's difficulty that came from liver's position in the abdomen among the other organs. This method will make the surgeon able to detect the tumor and then easing treatment also, it helps physicians and radiologists to identify the affected parts of the liver in order to protect the normal parts as much as possible from exposure to radiation. This study describes a new 2D liver segmentation method for purpose of transplantation surgery as a treatment for liver tumors. Liver segmentation is not only the key process for volume computation but also fundamental for further processing to get more anatomy information for individual patient. Due to the low contrast, blurred edges, large variability in shape and complex context with clutter features surrounding the liver that characterize the CT liver images. In this paper, the CT images are taken, and then the 2D segmentation process which is based on the hybrid method which is the combination of modified k-Mean (which depend on the distance and color), the statistical structure which are the first order statistical feature and the geometrical features are applied to the liver image to extract the CT liver boundary and further classify liver diseases.
... Early research in cell segmentation is dominated by classical methods such as active contours [5], [30], [31], mathematical morphology [32], and graph-based methods [33]. These unsupervised methods are flexible, largely interpretable, and are reasonably robust to domain shifts between two imaging modalities. ...
Article
Full-text available
The method proposed in this paper is a robust combination of multi-task learning and unsupervised domain adaptation for segmenting amoeboid cells in microscopy. A highlight of this work is the manner in which the model’s hyperparameters are estimated. The detriments of ad-hoc parameter estimation are well known, but this issue remains largely unaddressed in the context of CNN-based segmentation. Using a novel min-max formulation of the segmentation cost function our proposed method analytically estimates the model’s hyperparameters, while simultaneously learning the CNN weights during training. This end-to-end framework provides a consolidated mechanism to harness the potential of multi-task learning to isolate and segment clustered cells from low contrast brightfield images, and it simultaneously leverages deep domain adaptation to segment fluorescent cells without explicit pixel-level re-annotation of the data. Experimental validations on multi-cellular images strongly suggest the effectiveness of the proposed technique, and our quantitative results show at least 15% and 10% improvement in cell segmentation on brightfield and fluorescence images respectively compared to contemporary supervised segmentation methods.
... Another popular class of segmentation methods are model-based approaches, where a model of an object is fitted to the image data via energy minimization. Often, object contours are represented by control points (e.g., [18], [19], [20]) or level sets of functions in variational frameworks (e.g., [21], [22], [23]). However, the initialization of the control points or the level set functions is crucial due to local minima of the energies. ...
Article
Cell nuclei segmentation is challenging due to shape variation and closely clustered or partially overlapping objects. Most previous methods are not globally optimal, limited to elliptical models, or are computationally expensive. In this work, we introduce a globally optimal approach based on deformable shape models and global energy minimization for cell nuclei segmentation and cluster splitting. We propose an implicit parameterization of deformable shape models and show that it leads to a convex energy. Convex energy minimization yields the global solution independently of the initialization, is fast, and robust. To jointly perform cell nuclei segmentation and cluster splitting, we developed a novel iterative global energy minimization method, which leverages the inherent property of superadditivity of the convex energy. This property exploits the lower bound of the energy of the union of the models and improves the computational efficiency. Our method provably determines a solution close to global optimality. In addition, we derive a closed/form solution of the proposed global minimization based on the superadditivity property for non/clustered cell nuclei. We evaluated our method using fluorescence microscopy images of five different cell types comprising various challenges, and performed a quantitative comparison with previous methods. Our method achieved state/of/the/art or improved performance.
... Extension of parametric AC model is also done by Zimmer et al. [52], to track multiple objects. The authors combined multiple contours in a single framework by taking the idea of the Chan-Vese (CV) model [53]. ...
Article
Image segmentation is a fundamental and tedious task of computer vision. Because of inherent noise and intensity inhomogeneity in real-world images, it remains a difficult problem in practical applications such as image analysis, scene understanding, object detection, and many others. Several mathematical models proposed for image segmentation in the past few decades with an effective policy. Among these models, deformable models earned more attention and are widely used techniques due to their accuracy, efficiency, and robust effectiveness. This study reviews and compares various deformable models for the segmentation of digital images available in the literature. First, for a comprehensive study, these deformable models are classified into two classes such as direct partial differential equation (PDE) based approaches and variational based approaches. Beside this, variational based approaches are further classified into parametric and geometric models. Their advantages as well as shortcomings are discussed in detail from an objective viewpoint. Then, to check the robustness of various discussed classes of deformable models, a set of synthetic, natural, and real medical images are considered along with inhomogeneity and noise. Also, to measure the segmentation accuracy, different quantitative metrics based on contour and region are utilized. Numerical experiments with different classes of images, reveal that both direct PDE based and variation based models perform well in clean and noisy images. Whereas, the variational based approaches are superior for images having intensity inhomogeneity with high computational time. The qualitative and quantitative investigations confirm that the subtle change in model assumptions can have a significant impact on segmentation.
... It becomes worst when the cell overlap with a group. Zimmer and Olivo-Marin in 2005 [10] or based on contours that prevents the developing contours of developments of nuclei that makes contact from become merged. Energy ranged functional was developed with an overlap that has penalty between contours at the paired range. ...
Article
Full-text available
RBC called Erythrocytes is one of the important element in blood composition which is main responsible in all living cells for its gaseous exchanges with the environment externally. In general, at the physiological maintained conditions, RBC in view provides circular in the front and also looks bi-concave at side. One of serious disease with reference to blood cells is Cancer where the healthy RBC are affected. This reduces the body's immunity factors. To identify the cancer cell various methods are employed but it does not provide the proper detection of blood cells. In this method, proper identification of the cancer cells from the unaffected RBCs was identified in which are presented in blood samples using various imaging tools and also with the techniques. The proposed novel method called Online Region Based Segmentation (ORBS) method is done which is used to discover the areas of the boundary of the unaffected corpuscles. By using properties of region, a suitable metric is formulated to determine the shape which is abnormal in the blood cells. Overall accuracy of 96.9% is obtained using proposed ORBS methods and deep learning classification (DLC) method is accurate as 97.1% that helps to diagnose cancer cell using the feature extraction process which is done automatically. The computation time was found to be less when related to the other existing method which is 22 seconds. Closeness of Proposed method in relative to True Positive values at ROC curves indicates the performance which is higher than other methods. Experimental results prove proposed systems effectiveness when compared by means of other detection methods.
... Watershed methods look for the boundaries between adjacent cells in the ridgelines of the brightness mountains that cells feature over the darker background (Table 1). 72,73 Alternatively, variational methods allow readily combining information from the image (similarly to the methods above, e.g., uniform/non-uniform 74 brightness separation or edge detection) with a priori knowledge (which can include constraints on cell shape and contour intersections) 75 by customizing a so-called energy that is to be minimized. This flexibility comes at the price of additional burden: they require a ''seed'' (an initial prospective contour for each cell), as well as more complex numerical techniques. ...
Article
Full-text available
Bioimage analysis (BIA) has historically helped study how and why cells move; biological experiments evolved in intimate feedback with the most classical image processing techniques because they contribute objectivity and reproducibility to an eminently qualitative science. Cell segmentation, tracking, and morphology descriptors are all discussed here. Using ameboid motility as a case study, these methods help us illustrate how proper quantification can augment biological data, for example, by choosing mathematical representations that amplify initially subtle differences, by statistically uncovering general laws or by integrating physical insight. More recently, the non-invasive nature of quantitative imaging is fertilizing two blooming fields: mechanobiology, where many biophysical measurements remain inaccessible, and microenvironments, where the quest for physiological relevance has exploded data size. From relief to remedy, this trend indicates that BIA is to become a main vector of biological discovery as human visual analysis struggles against ever more complex data.
... Later in [191], the active contour method for the binary case was extended to k values (for k > 2 the problem remains NP-hard). Further works on active contours are [70,219,58]. For an overview on level set segmentation, we refer to [58]. ...
Thesis
In this work, we consider Mumford-Shah and Potts models and their higher order generalizations. Mumford-Shah and Potts models are among the most well-known variational approaches to edge-preserving smoothing and partitioning of images. Though their formulations are intuitive, their application is not straightforward as it corresponds to solving challenging, particularly non-convex, minimization problems. The main focus of this thesis is the development of new algorithmic approaches to Mumford-Shah and Potts models, which is to this day an active field of research. We start by considering the situation for univariate data. We find that switching to higher order models can overcome known shortcomings of the classical first order models when applied to data with steep slopes. Though the existing approaches to the first order models could be applied in principle, they are slow or become numerically unstable for higher orders. Therefore, we develop a new algorithm for univariate Mumford-Shah and Potts models of any order and show that it solves the models in a stable way in O(n^2). Furthermore, we develop algorithms for the inverse Potts model. The inverse Potts model can be seen as an approach to jointly reconstructing and partitioning images that are only available indirectly on the basis of measured data. Further, we give a convergence analysis for the proposed algorithms. In particular, we prove the convergence to a local minimum of the underlying NP-hard minimization problem. We apply the proposed algorithms to numerical data to illustrate their benefits. Next, we apply the multi-channel Potts prior to the reconstruction problem in multi-spectral computed tomography (CT). To this end, we propose a new superiorization approach, which perturbs the iterates of the conjugate gradient method towards better results with respect to the Potts prior. In numerical experiments, we illustrate the benefits of the proposed approach by comparing it to the existing Potts model approach from the literature as well as to the existing total variation type methods. Hereafter, we consider the second order Mumford-Shah model for edge-preserving smoothing of images which –similarly to the univariate case– improves upon the classical Mumford-Shah model for images with linear color gradients. Based on reformulations in terms of Taylor jets, i.e. specific fields of polynomials, we derive discrete second order Mumford-Shah models for which we develop an efficient algorithm using an ADMM scheme. We illustrate the potential of the proposed method by comparing it with existing methods for the second order Mumford-Shah model. Further, we illustrate its benefits in connection with edge detection. Finally, we consider the affine-linear Potts model for the image partitioning problem. As many images possess linear trends within homogeneous regions, the classical Potts model frequently leads to oversegmentation. The affine-linear Potts model accounts for that problem by allowing for linear trends within segments. We lift the corresponding minimization problem to the jet space and develop again an ADMM approach. In numerical experiments, we show that the proposed algorithm achieves lower energy values as well as faster runtimes than the method of comparison, which is based on the iterative application of the graph cut algorithm (with α-expansion moves).
... Traditionally different image processing algorithms were proposed for segmentation of cells in microscopy images. Some of these include, intensity thresholding techniques [4], watershed techniques [5], morphological filtering techniques [6] and active contours [7]. ...
Chapter
Full-text available
Separating cells from the background in microscopy images is the critical step in image processing pipeline for the study of single cell life cycle. Live cell imaging experiments involve thousands of cells and images taken for a few days, which results in huge data generation. Automatic analysis of such images is essential rather than performing analysis manually. The challenges involved are non-uniform illumination of the image, different types of cell lines to be studied, large curation time required and analysis of large data to name a few. In this work we present a image processing pipeline using a convolutional neural network (CNN) model followed by thresholding and morphological operations for segmenting the NIH 3T3 cells in microscopic images. The segmentation results are evaluated by comparing them with the ground truth images. The proposed methodology gave a Dice index of 0.93 on a stack of 238 phase contrast images. Further, we show that CNN based approach performs superior to conventional image processing segmentation methods on phase contrast images of NIH 3T3 cells.
Chapter
Full-text available
Implicit active contour models belong to the most popular level set methods in computer vision. Typical implementations, however, suffer from poor efficiency. In this chapter we survey an efficient algorithm that is based on an additive operator splitting (AOS). It is suitable for geometric and geodesic active contour models as well as for mean curvature motion. It uses harmonic averaging and does not require to compute the distance function in each iteration step. We prove that the scheme satisfies a discrete maximumminimum principle which implies unconditional stability if no balloon forces are present. Moreover, it possesses all typical advantages of AOS schemes: simple implementation, equal treatment of all axes, suitability for parallel computing, and straightforward generalization to higher dimensions. Experiments show that one can gain a speed up by one order of magnitude compared to the widely used explicit time discretization.
Conference Paper
This paper presents a method based on geometric active contours to segment and track biological cells in fluorescence videomicroscopy. To achieve robust segmentation of fluorescent cells with fuzzy boundaries, we use the region-based model introduced by Chan and Vese. The level set approach allows fully automatic initialization, detection of dividing cells and of cells entering the field of view during the sequence. However, the standard approach using only one level set function leads to undesired merging of touching cells and failure of tracking. To prevent this, we represent each cell by its own level set function and introduce a coupling constraint that minimizes overlaps of active contours. This technique maintains the separation of contours and allows tracking to proceed even when the boundary between cells is weak or invisible.
Article
We propose a new multiphase level set framework for image segmentation using the Mumford and Shah model, for piecewise constant and piecewise smooth optimal approximations. The proposed method is also a generalization of an active contour model without edges based 2-phase segmentation, developed by the authors earlier in T. Chan and L. Vese (1999. In Scale-Space'99, M. Nilsen et al. (Eds.), LNCS, vol. 1682, pp. 141–151) and T. Chan and L. Vese (2001. IEEE-IP, 10(2):266–277). The multiphase level set formulation is new and of interest on its own: by construction, it automatically avoids the problems of vacuum and overlap; it needs only log n level set functions for n phases in the piecewise constant case; it can represent boundaries with complex topologies, including triple junctions; in the piecewise smooth case, only two level set functions formally suffice to represent any partition, based on The Four-Color Theorem. Finally, we validate the proposed models by numerical results for signal and image denoising and segmentation, implemented using the Osher and Sethian level set method.
Article
We present a supervised classification model based on a variational approach. This model is devoted to find an optimal partition composed of homogeneous classes with regular interfaces. The originality of the proposed approach concerns the definition of a partition by the use of level sets. Each set of regions and boundaries associated to a class is defined by a unique level set function. We use as many level sets as different classes and all these level sets are moving together thanks to forces which interact in order to get an optimal partition. We show how these forces can be defined through the minimization of a unique fonctional. The coupled Partial Differential Equations (PDE) related to the minimization of the functional are considered through a dynamical scheme. Given an initial interface set (zero level set), the different terms of the PDE's are governing the motion of interfaces such that, at convergence, we get an optimal partition as defined above. Each interface is guided by internal forces (regularity of the interface), and external ones (data term, no vacuum, no regions overlapping). Several experiments were conducted on both synthetic and real images.
Conference Paper
We present a supervised classification model based on a vari- ational approach. This model is devoted to find an optimal partition com- pound of homogeneous classes with regular interfaces. We represent the regions of the image defined by the classes and their interfaces by level set functions, and we define a functional whose minimum is an optimal partition. The coupled Partial Differential Equations (PDE) related to the minimization of the functional are considered through a dynamical scheme. Given an initial interface set (zero level set), the different terms of the PDE’s are governing the motion of interfaces such that, at con- vergence, we get an optimal partition as defined above. Each interface is guided by internal forces (regularity of the interface), and external ones (data term, no vacuum, no regions overlapping). Several experiments were conducted on both synthetic an real images.
Article
In recent years, the field of active contour-based image segmentation has seen the emergence of two competing approaches. The first and oldest approach represents active contours in an explicit (or parametric) manner corresponding to the Lagrangian formulation. The second approach represents active contours in an implicit manner corresponding to the Eulerian framework. After comparing these two approaches, we describe several new topological and physical constraints applied to parametric active contours in order to combine the advantages of these two contour representations. More precisely, we introduce three algorithms related to the control of the contour topology, geometry, and deformation. The first algorithm controls both vertex spacing and contour smoothness in an independent and intrinsic manner. The second algorithm controls the contour resolution (number of vertices) while the third algorithm automatically creates or fuses connected components on closed or opened contours. The efficiency of these algorithms is demonstrated on several images including medical images and a comparison with the level-sets method is also provided.