ArticlePDF Available

Abstract and Figures

This work introduces a residual operator called ultimate grain filter which is a powerful image operator based on numerical residues. With a multi-scale approach, the ultimate grain filter analyzes an image under a series of grain filters of increasing grain sizes. Thus, contrasted objects can be detected if a relevant residue is generated when they are filtered out by one of these grain filters. We also present an efficient algorithm for ultimate grain filter computation by using a structure called tree of shapes. In fact, since the result of a given grain filter can be obtained by pruning the corresponding tree and reconstructing it, we show that the result of the ultimate grain filter (which is based on numerical residues from a family of grain filters) can be obtained by the computation of the difference (remaining nodes) of the corresponding pruned trees. Finally, we propose the use of ultimate grain filter to extract contrasted objects using a priori knowledge of an application.
Content may be subject to copyright.
ULTIMATE GRAIN FILTER
Wonder Alexandre Luz Alves and Ronaldo Fumio Hashimoto
Department of Computer Science, Institute of Mathematics and Statistics,
University of S˜
ao Paulo, S˜
ao Paulo, Brazil
ABSTRACT
This work introduces a residual operator called ultimate grain filter
which is a powerful image operator based on numerical residues.
With a multi-scale approach, the ultimate grain filter analyzes an
image under a series of grain filters of increasing grain sizes. Thus,
contrasted objects can be detected if a relevant residue is generated
when they are filtered out by one of these grain filters. We also
present an efficient algorithm for ultimate grain filter computation
by using a structure called tree of shapes. In fact, since the result
of a given grain filter can be obtained by pruning the corresponding
tree and reconstructing it, we show that the result of the ultimate
grain filter (which is based on numerical residues from a family of
grain filters) can be obtained by the computation of the difference
(remaining nodes) of the corresponding pruned trees. Finally, we
propose the use of ultimate grain filter to extract contrasted objects
using a priori knowledge of an application.
Index Termsresidual morphological operator; ultimate grain
filter; grain filter; tree of shapes; connected operator.
1. INTRODUCTION
An operator in Mathematical Morphology (MM) can be seen as a
mapping between complete lattices [1]. In particular, mappings on
the set of all gray level images F(D)defined on domain D Z2
are of special interest in MM and they are called image operators,
i.e., ψ:F(D) F(D). In this study we focus on a special type
of operator known as residual operator and is defined as the differ-
ence (pixel-by-pixel) between two operators, say ψand ϕ, applied
to a given image f F(D), i.e., r(f) = ψ(f)ϕ(f). In the
literature, it is possible to find many applications of residual oper-
ators in Image Processing: morphological gradients (which are the
residue of a dilation δand an erosion ε, i.e., r(f) = δ(f)ε(f)),
white top-hats (which are the residue of the identity operator id
and an opening γ, i.e., r(f) = id(f)γ(f)); and black top-hats
(which are the residue of a closing φand the identity operator, i.e.,
r(f) = φ(f)id(f)) are examples of residual operators. Residual
operators have been extended to extract residues of two families (in-
dexed by a set I) of operators, applied to an image, {ψi(f) : iI}
and {ϕi(f) : iI}as the supremum of residual operators, i.e.,
r(f) = supiI{ri(f) : ri(f) = ψi(f)ϕi(f)}[2]. For ex-
ample, the ultimate opening operator is defined as the supremum of
residual operators when ψiand ϕiare consecutive openings, i.e.,
ri(f) = γi(f)γi+1(f)[2].
These residual operators have been extended to extract residues
of families of connected operators [3, 4, 5, 6, 7]. One of the ad-
vantages is that the connected operators do not create new contours.
Thus, contrasted objects are detected when they are filtered out by a
connected operator, generating an important connected residue. At
first, Retornaz and Marcotegui [3] proposed the ultimate attribute
opening (UAO) and by duality ultimate attribute closing (UAC) for
scene text localization. Then, Fabrizio and Marcotegui [8] devel-
oped an efficient algorithm for computing UAO using component
tree. Later, Hern´
andez and Marcotegui [5] added shape information
to extract residues from UAO. Differently from these works [3, 5, 8],
Meyer [4] proposed to extract residues from a family of levelings
obtained by using erosions and dilations as markers. The obtained
levelings produce positive and negative residues (which are not the
case for the UAO and UAC). In this way, Meyer [4], in contrast to
Beucher [2], treats separately the positive and negative residues.
Given the above considerations, in this work, we propose a new
residual operator called ultimate grain filter (UGF). A powerful
residual operator constructed through a multi-scale approach. It pro-
vides the same treatment to both poles of the image, which allows
us, differently from UAO (resp. UAC), to assume no hypotheses
about the contrast of objects and this is the main advantage of UGF
in compasison to the UAO (resp. UAC). We also show that the
family of grain filters is represented by a family of trees that satisfies
a scale-space based on levelings. Lastly, we use these trees to build
an efficient algorithm for computing UGF.
The remainder of this paper is structured as follows. For the sake
of completeness, Sections 2 and 3 briefly recall some definitions and
properties of image representation by tree structures and grain fil-
ters. In particular, Section 3 provides the first result of this work.
The other contributions of this paper are given in Section 4, where
we introduce the UGF and present an efficient algorithm for its com-
putation as well experimental results. Finally, Section 5 concludes
this work and presents some future research direction.
2. PRELIMINARIES
For any µZwe define Xµ(f) = {p D :f(p)< µ}and
Xµ(f) = {p D :f(p)µ}as the lower and upper level sets at
value µ, respectively. From these sets we deduce two other sets L(f)
and U(f)composed of the connected components (CCs) of the lower
and upper level sets of f, i.e., L(f) = {C CC4(Xµ(f)) : µZ}
and U(f) = {C CC8(Xµ(f)) : µZ}, where CC4(X)and
CC8(X)are sets of 4and 8connected CCs of X, respectively. Then,
the pairs ordered consisting of the families of CCs of lower and upper
level sets and the usual set inclusion relation, i.e., (L(f),)and
(U(f),), induce two dual trees [9]. They can be represented by
non-redundant data structures known as min-tree and max-tree [10].
Combining this pair of dual component trees, min-tree and max-
tree, into a single tree, we have the tree of shapes [9, 11]. Note that,
if C L(f)then the CCs of D\Care in U(f)(or vice versa, C
U(f) CC4(D\C) L(f)). Then, let P(D)denote the powerset
of Dand let sat :P(D) P(D)be the operator of saturation [12]
(or filling holes) and SAT (f) = {sat(C) : C L(f) U (f)}
be the family of CCs of the upper and lower level sets with holes
filled. The elements of SAT (f), called shapes, are nested by an
978-1-4799-5751-4/14/$31.00 ©2014 IEEE ICIP 20142953
inclusion relation and thus the pair (SAT (f),), induces the tree
of shapes [9, 12].
Tree of shapes, and also component trees, are complete repre-
sentation of images which can be represented by a compact and non-
redundant data structure [12, 13] so that a pixel p D which is asso-
ciated to the smallest shape of the tree containing it, by parenthood
relationship, is also associated to all its ancestors shapes. Then, we
denote by SC (Tf, p)the smallest shape containing pin tree Tf.
3. GRAIN FILTERS AND LEVELINGS
Grain filters are powerful connected filters that have as characteristic
the simplification of the image by regional extrema. The first kind of
grain filters has been introduced by L. Vicent [14] in order to remove
small CCs of level sets called grain, i.e.,
x D,[γκ
λ(f)](x) = sup
C∈L0:xC
{inf
yC{f(y)}},
x D,[φκ
λ(f)](x) = inf
C∈U0:xC{sup
yC
{f(y)}},
where L0and U0are subsets of L(f)and U(f), respectively, such
that its elements have area κgreater than λ. These operators (that
do not commute, i.e., γκ
λ(φκ
λ(f)) 6=φκ
λ(γκ
λ(f))), are opening (and
closing respectively), the grains are the CCs of level sets. A second
kind of grain filters was introduced by Masnou [15] and their prop-
erties were studied by V. Caselles and P. Monasse [12, 16]. In this
filter, the grains are “shapes” and defined as: x D,
[νκ
λ(f)](x) = sup
B∈Bλ
{inf
yx+B{f(y)}},
where Bλis family of sets such that B Bλsatisfies: (i)Bis
connected and closed; (ii) 0 sat(B);(iii)κ(sat(B)) λ;(iv)
0/Bκ(CC(D\B , 0)) λ[17].
If we store the information of the area of CCs or “shapes” in
each node of the tree, then we can efficiently obtain these operators
by pruning. For example, if the chosen tree is the max-tree (resp.
min-tree) then the reconstruction of the pruned tree is an area open-
ing (resp. closing) whereas if it is the tree of shapes, the resulting
operator is a grain filter [12]. In fact, many different kinds of in-
formation can be stored in the nodes of the trees, and we call them
attributes. Formally, an attribute is a function κ:P(D)Rand it
can represent node information such as area, volume, height, width,
circularity. Some attributes can be increasing, i.e., if A, B P(D)
and AB, then κ(A)κ(B). If the attributes are increasing then
the attribute values also follow a well-defined ordering in these trees
due to the hierarchy of the level sets. Thanks to this property, we can
perform connected filtering (extensive, anti-extensive and self-dual)
simply pruning the tree nodes whose attribute value κis lower than
a given threshold λfollowed by reconstruction of the pruned tree.
Remembering that the connected filters represent a wide class
of filters in which F. Meyer [18, 19, 20, 21] extensively studied their
specializations. One of these specializations, known as levelings, are
powerful simplifying filters that preserves order and do not create
new structures (regional extrema and contours).
Definition 1. An operator ψ:F(D) F (D)is said leveling, if for
any f F(D)the following relation is valid for all pairs of neigh-
boring pixels, i.e., (p, q)neighbors: [ψ(f)](p)>[ψ(f)](q)
f(p)[ψ(f)](p)and [ψ(f)](q)f(q).
Levelings are increasing operators and constrain the values
of neighboring pixels, since f(p)[ψ(f)](p)>[ψ(f)](q)
f(q)f(p)> f(q). Moreover, it is possible to show that
if ψ(f)is the result of filtering using increasing attributes from
the max-tree, min-tree or tree of shapes built from an image f,
then ψ(f)is a leveling of f. In other words, attribute openings
(resp. closings) and grain filters are levelings. In the case of the
max-tree (resp. min-tree) the resulting operator is anti-extensive
(resp. extensive), that is, ψ(f)f(resp. ψ(f)f). Further-
more, ψ(f)is a leveling of fsince ψ(f)f(resp. ψ(f)f)
and for any pair of neighboring pixels (p, q)the following condi-
tion holds: [ψ(f)](p)>[ψ(f)](q)[ψ(f)](q) = f(q)(resp.
[ψ(f)](p)>[ψ(f)](q)[ψ(f)](p) = f(p)).
Now, in the case of tree of shapes, it is needed to know how
are the relations between neighboring pixels in the nodes of the tree.
Note that, in the tree there are two types of connectivity. In this
sense, Proposition 1 and 2 help us understand how the neighboring
pixels are related in the tree.
Proposition 1. Let Tfbe the tree of shapes of an image f. Then,
(p, q)neighbors such that f(p)6=f(q), we have: either SC(Tf, p)
and SC (Tf, q)are comparable; or SC(Tf, p) SC(Tf, q ) = .
Proposition 2. If A, B SAT (f)such that BAand Aand B
are of different types (lower or upper). Then, for any pixel in B, its
neighborhood pixels are in a node rooted in the subtree of node A.
Theorem 1. Let fbe an image and consider the tree Tf(max-tree,
min-tree or tree of shapes) that represents f. Consider that the tree
was filtered by an increasing attribute and this pruned tree was re-
constructed into an image fλ. Then, fλis a leveling of f.
Hint of the proof. The proof for the component tree is trivial. Now,
to prove for the tree of shapes, we need just to check if for all (p, q)
neighbors, the definition of leveling is valid. Let us consider two
cases, where f(p) = f(q)and where f(p)6=f(q). In the first case,
fλmeets the definition of leveling by vacuity. In the second case, we
have SC(Tf, p)6=SC(Tf, q)and the pruning in Tfcan: either (1)
preserve both nodes; or (2) eliminate both nodes; or (3) eliminate
one of the two nodes. In such cases, Propositions 1 and 2 help to
prove that the definition of leveling is satisfied.
Levelings can be nested to create a scale-space decomposition of
an image [20]. Suppose Tfis the tree that represents an image fand
fλis the result of a filtering by increasing attribute in Tfby a thresh-
old λ(i.e., a pruning in Tf). Denote by Tfλthis pruned version of
Tf. Likewise, consider the tree Tfλ+1 pruned obtained by a filtering
of an increasing attribute in Tfλby a threshold λ+ 1. Hence, by
Theo. 1, we have that fλ+1 is a leveling of fλand fλis a leveling of
f. Then, from Def. 1, we have that, for all pairs of neighboring pix-
els (p, q),fλ+1 (p)> fλ+1(q)fλ(p)fλ+1 (p)> fλ+1(q)
fλ(q)f(p)fλ(p)fλ+1(p)and fλ+1 (q)fλ(q)f(q),
showing that fλ+1 is also a leveling of f. This shows that the tree
generates a family of levelings that further simplifies the image f,
thus constituting a morphological scale-space [4, 12, 20]. Fig. 1
shows some examples of images presented in a scale-space of level-
ings obtained by pruning the min-tree, max-tree and tree of shapes.
4. ULTIMATE GRAIN FILTER
Remembering, a residual operator is defined as the difference be-
tween two operators, say ψand φ, applied to a given image f
F(D), i.e., p D,[r(f)](p) = [ψ(f)](p)[φ(f)](p). So we
say that if r(p)>0then r(p)is a positive residue; if r(p)<0
978-1-4799-5751-4/14/$31.00 ©2014 IEEE ICIP 20142954
min-tree
area = 0 area = 491 area = 1966 area = 9953
max-tree
area = 0 area = 491 area = 1966 area = 9953
tree of shapes
area = 0 area = 491 area = 1966 area = 9953
Fig. 1. Examples of images presented in a scale-space of levelings
obtained by pruning the min-tree, max-tree and tree of shapes, re-
spectively.
then r(p)is negative residue; otherwise, r(p)is a null residue. Dif-
ferently from S. Beucher [2] that defined the extended residual op-
erator as supremum of either positive residual operators or negative
residual operators, in this work, we will follow Meyer [4] and con-
sider the extended residual operator as supremum of positive and
negative residual operators, i.e., r(f) = supλI{rλ(f) : rλ(f) =
r+
λ(f)r
λ(f)}where r+
λ(f) = [ψλ(f)φλ(f)]0and r
λ(f) =
[φλ(f)ψλ(f)] 0.
As original contribution of this work, we present in this section a
residual operator, called ultimate grain filter (UGF), where the prim-
itives are consecutive grain filters, say ψλand ψλ+1, of an indexed
family {ψλ:λI}of grain filters of increasing sizes, i.e.,
R±(f) = R+(f) R(f),
R+(f) = sup
λI
{r+
λ(f) : r+
λ(f) = [ψλ(f)ψλ+1(f)] 0}and
R(f) = sup
λI
{r
λ(f) : r
λ(f) = [ψλ+1(f)ψλ(f)] 0}.
Note that, if ψλ=γκ
λ(resp. ψλ=φκ
λ) then R±is the UAO
(resp. UAC). We call the operator R±of UGF only when ψλ=νκ
λ.
The UGF is a powerful operator based on numerical residues. With
a multi-scale approach, the UGF analyzes an image under a series of
grain filters of increasing sizes. Contrasted objects are detected when
they are filtered out by a grain filter, generating important residues.
Fig. 2 shows some examples of applications of UAO, UAC and UGF
for extraction of contrasts in a given image. The UGFs, as grain
input image fUAO of fUAC of fUGF of f
Fig. 2. Examples of extraction of contrast.
filters, are operators that provide the same treatment to both poles
of the image (see Fig. 2 how UGF extracted both back and white
contrasts), which allow us, differently from UAOs (resp. UACs),
not to assume any hypotheses about the contrast of objects and this
is the main advantage of UGFs in comparison to the UAOs (resp.
UACs). Moreover, it is possible to show that the UGFs are connected
operators, i.e., preserve contours of the input image.
4.1. Associated information
It can be associated to each pixel x D a set of extracted attributes
of a node C, containing x, that generated the maximum non-null
residue in rλfor x. In this regard, a set of associated information
can be extracted at the time of computation of the UGF. For example,
a function that associates to each pixel the index that produced the
maximum non-null residue [2], i.e.,
q±(x) = q+(x), if R+(x)>R(x),
q(x), otherwise,
where q+(x) = max{λ+ 1 : [r+
λ(f)](x) = [R+(f)](x)>0}and
q(x) = max{λ+ 1 : [r
λ(f)](x)=[R(f)](x)>0}. Thus,
SC (Tfq(x), x)is the node containing xthat produced the maximum
non-null residue. This information may be very useful to perform an
analysis of the extraction of residues.
4.2. Algorithm
As the family of grain filters of a image fcan be obtained by prun-
ing its corresponding tree Tf, then two consecutive grain filters, say
fλand fλ+1, can be obtained by pruning Tfby thresholds I(λ)and
I(λ+ 1), respectively. Note that, I:IRis a monotone function
that associated the family of indices to the values of indices. There-
fore, we can use the nodes of Tfλthat are not in Tfλ+1 to compute
the residue between fλand fλ+1. Then, let Nr(λ)the set of nodes
removed from Tfλto build the pruned tree Tfλ+1. Therefore, we
can calculate the positive (resp. negative) residues of the pixels of a
given node C N r(λ)by
r+
λ(C) =
[level(C)level(parent(C)) 0]
if parent(C)/ N r(λ),
[level(C)level(parent(C)) 0]
+r+
λ(parent(C)),
otherwise,
where level(C)is the gray level associated to node Cand
parent(C)is the parent node of C. Therefore, given an index
λI={0,1, ..., IMAX }we need to know what are the nodes that
are in Nr(λ)and this leads us to think about the family of primitives
of the UGF. For example, the set of all the possible primitives with
the attribute κ, is given by Nr(λ) = {C Tfλ:κ(C) I(λ)}.
The UGF often underestimates the contrast of blurred ob-
jects in the image, since contours of objects are composed of
small gradual transitions [6] of gray levels. Thus, the contrast
of an object is divided into several transitions and the greatest
value of the residue obtained from UGF is inevitably smaller
than the real contrast of object. One way to reduce this discom-
fort is to simplify the family of primitives such that the transi-
tions of the gray levels between two consecutive primitives are
more abrupt. So, suppose that C N r(λ)and parent(C)/
Nr(λ). Then, we have κ(parent(C)) κ(C)>0since
κ(C) I(λ)and κ(parent(C)) >I(λ). Therefore, if
κ(parent(C)) κ(C)> , then we can add nodes in Nr(λ)
978-1-4799-5751-4/14/$31.00 ©2014 IEEE ICIP 20142955
corresponding small gradual transitions of node C. Thus, we have
Nr(λ) = Nr(λ) {C[A, B] : A N r(λ),parent(A)/
Nr(λ), B is the first ancestor of A such that κ(parent(C))
κ(C)> }. An adaptive way to find is to use extinction val-
ues [22]. We also can use shape recognition techniques to choose
nodes of Nr(λ)based on a criterion of similarity [5]. This al-
lows us to extract only the residues of nodes that satisfy the criterion
, i.e., Nr,(λ) = {C N r(λ) : Csatisfies the criterion }.
Now, we present Alg. 1 to compute the UGF. This algorithm is
based on the algorithm presented by Fabrizio and Marcotegui [8] to
compute the UAO using max-tree and the concept of gradual tran-
sition shown in [6]. This algorithm uses a top-down approach: the
residue of the parent node is calculated; it is then propagated to the
child nodes. Every child node compares its residue with the residue
of the parent node and keeps the maximum of these two values in
R. At the same time, it can also store the index that generated the
maximum residue. After this step, each child node becomes a parent
node and the process is repeated.
Algorithm 1: Compute the ultimate grain filter.
1void UGF(node C)begin
2if C N r,(κ(C)) then
3r+[C] = level(C)parent(level(C)) 0;
r[C] = parent(level(C)) level(C)0;
4if parent(C) N r,(κ(C)) then
5r+[C] = r+[C] + r+[parent(C)];
r[C] = r[C] + r[parent(C)];
6R+[C] = R+[parent(C)] r+[C];
7R[C] = R[parent(C)] r[C];
// Here: compute q+and q
8foreach node Schild of Cdo
9UGF (S);
Alg. 1 needs as input a tree (max-tree, min-tree or tree of
shapes). These trees can be built by simple algorithms based union-
find proposed in [13, 23] with complexity O(|D| log(|D|)). Once
the tree is built (see algorithms in [10, 13]), we execute the UGF
function, for all child nodes of the root node. Since the test in
line 4can be done in constant time and UGF function visits each
node of the tree exactly once and the number of nodes of the tree
is at most the number of pixels in the image, UGF is executed with
time complexity O(|D|). Then, we can reconstruct the output of
UGF (by p D,R±[p] = R+[SC(T, p)] R[S C(T, p)])
and associated information in O(|D|)time complexity. There-
fore, to extract the UGF, we build the tree with time complexity
O(|D| log(|D|)) [10, 13] and compute the residues with time com-
plexity O(|D|). Thus, the time complexity of the proposed algorithm
is O(|D| log(|D|)).
4.3. Example of application
Contrast extraction, through numerical residues, has been used suc-
cessfully in various applications in a preprocessing step such as text
location [3, 24], segmentation of building facades [5], and restora-
tion of historical documents [4]. In this sense, we present a simple
example of application (in problem of extraction of objects) that evi-
dences its potential for future applications. It is worth noting that for
the success of the application is very important to select the primi-
tives based on the knowledge of the application domain. So, let us
consider the problem of text location and cell segmentation, where
objects are characters and cells respectively. Thus, we added shape
information, by using , based on heuristics of text location [7, 24]
and cell segmentation, respectively. The results are shown in Fig. 3,
where (3a)and (3d)are the input images; (3b)and (3e)are the re-
sults of UGF; and (3c)and (3f)are labeled images of the associated
function q±. In Fig. 4, (4a)is the input image and (4b)and (4c)are
labeled images of the associated function q±. Watch a demo of UGF
at http://score.ime.usp.br/wonder/icip2014.
(a) (b) (c)
(d) (e) (f)
Fig. 3. Example of extraction of residues applied to text location
using height (bounding box) attribute and heuristics for the problem
of text location to select the family of primitives.
(a) (b) (c)
Fig. 4. Application of the UGF with circular shape information.
5. CONCLUSION
In this work, we introduced a new residual operator called ultimate
grain filter (UGF) based on numerical residues. A powerful residual
operator constructed through a multi-scale approach. It provides the
same treatment to both poles of the image, which allows us, differ-
ently from the UAO (resp. UAC), not to assume hypotheses about
the contrast of objects and this is the main advantage of UGF in
comparison to the UAO (resp. UAC). We also showed that the fam-
ily of grain filters is represented by a family of trees that satisfies a
scale-space based on levelings. These trees are used to build an effi-
cient algorithm for computing UGF. Extraction of contrast by means
of numerical residues has been used successfully in various appli-
cations, such as text location, segmentation of building facades and
binarization of historical documents. Experimental results applied
to text location show the potentiality of using UGF to real image
processing problems.
Acknowledgements
We would like to thank the financial support from CAPES, CNPq,
FAPESP (grant #2011/50761-2) and NAP eScience-PRP-USP.
978-1-4799-5751-4/14/$31.00 ©2014 IEEE ICIP 20142956
6. REFERENCES
[1] Jean Serra, Image Analysis and Mathematical Morphology,
vol. 2: Theoretical Advances, Academic Press, London, 1988.
[2] Serge Beucher, “Numerical residues,” Image Vision Comput.,
vol. 25, no. 4, pp. 405–415, Apr. 2007.
[3] Thomas Retornaz and Beatriz Marcotegui, “Scene text lo-
calization based on the ultimate opening,” in Proceedings of
the 8th International Symposium on Mathematical Morphol-
ogy and its Applications to Image and Signal Processing, 2007,
vol. 1 of ISMM ’08, pp. 177–188.
[4] Fernand Meyer, “Levelings and flat zone morphology, Pattern
Recognition, International Conference on, vol. 0, pp. 1570–
1573, 2010.
[5] Jorge Hernandez and Beatriz Marcotegui, “Shape ultimate at-
tribute opening, Image and Vision Computing, vol. 29, no. 8,
pp. 533 545, 2011.
[6] Beatriz Marcotegui, Jorge Hernandez, and Thomas Retornaz,
“Ultimate opening and gradual transitions,” in Mathematical
Morphology and Its Applications to Image and Signal Process-
ing, vol. 6671, pp. 166–177. 2011.
[7] W.A.L. Alves, A. Morimitsu, J.S. Castro, and R.F. Hashimoto,
“Extraction of numerical residues in families of levelings, in
Graphics, Patterns and Images (SIBGRAPI), 2013 26th SIB-
GRAPI - Conference on, 2013, pp. 349–356.
[8] Jonathan Fabrizio and Beatriz Marcotegui, “Fast implemen-
tation of the ultimate opening,” in Proceedings of the 9th In-
ternational Symposium on Mathematical Morphology, 2009,
ISMM ’09, pp. 272–281.
[9] Vicent Caselles, Enric Meinhardt, and Pascal Monasse, “Con-
structing the tree of shapes of an image by fusion of the trees
of connected components of upper and lower level sets, Posi-
tivity, vol. 12, no. 1, pp. 55–73, 2008.
[10] Edwin Carlinet and Thierry Graud, “A comparison of many
max-tree computation algorithms,” in Mathematical Mor-
phology and Its Applications to Signal and Image Processing,
vol. 7883 of Lecture Notes in Computer Science, pp. 73–85.
Springer Berlin Heidelberg, 2013.
[11] Pascal Monasse and Frederic Guichard, “Fast computation of
a contrast-invariant image representation, IEEE Transactions
on Image Processing, vol. 9, no. 5, pp. 860–872, 2000.
[12] Vicent Caselles and Pascal Monasse, “Geometric description
of images as topographic maps,” Lecture Notes in Mathemat-
ics, 2010.
[13] Thierry Graud, Edwin Carlinet, Sbastien Crozet, and Laurent
Najman, “A quasi-linear algorithm to compute the tree of
shapes of nd images,” in Mathematical Morphology and Its
Applications to Signal and Image Processing, vol. 7883 of Lec-
ture Notes in Computer Science, pp. 98–110. Springer Berlin
Heidelberg, 2013.
[14] Luc Vincent, “Morphological area openings and closings for
grey-scale images, in Proc. of the Workshop Shape in Picture.
1992, pp. 197–208, Springer.
[15] Simon Masnou and Jean-Michel Morel, “Image restoration
involving connectedness, in In Proceedings of the 6 th Inter-
national Workshop on Digital Image Processing and Computer
Graphics. 1998, pp. 334–6, SPIE.
[16] Vicent Caselles and Pascal Monasse, “Grain filters,” Journal of
Mathematical Imaging and Vision, vol. 17, no. 3, pp. 249–270,
2002.
[17] Pascal Monasse, Morphological representation of digital im-
ages and application to registration, Ph.D. thesis, Universit´
e
Paris IX-Dauphine, Paris, France, June 2000.
[18] Fernand Meyer, “From connected operators to levelings, in
Proceedings of the 4th International Symposium on Mathemat-
ical Morphology and its Applications to Image and Signal Pro-
cessing, 1998, pp. 191–198.
[19] Fernand Meyer, “The levelings, in Proceedings of the 4th In-
ternational Symposium on Mathematical Morphology and its
Applications to Image and Signal Processing, 1998, ISMM
’98, pp. 199–206.
[20] Fernand Meyer and Petros Maragos, “Nonlinear scale-space
representation with morphological levelings, Journal of Vi-
sual Communication and Image Representation, vol. 11, no. 2,
pp. 245 265, 2000.
[21] Fernand Meyer, “Levelings, image simplification filters for
segmentation, Journal of Mathematical Imaging and Vision,
vol. 20, no. 1-2, pp. 59–72, 2004.
[22] Alexandre Gonalves Silva and Roberto de Alencar Lotufo,
“Efficient computation of new extinction values from extended
component tree,” Pattern Recognition Letters, vol. 32, no. 1,
pp. 79 90, 2011.
[23] Ch Berger, T. Geraud, R. Levillain, N. Widynski, A. Baillard,
and E. Bertin, “Effective component tree computation with ap-
plication to pattern recognition in astronomical imaging,” in
Image Processing, 2007. ICIP 2007. IEEE International Con-
ference on, 2007, vol. 4, pp. IV 41–IV 44.
[24] W.A.L. Alves and R.F. Hashimoto, “Text regions extracted
from scene images by ultimate attribute opening and decision
tree classification,” in Proceedings of the 23rd SIBGRAPI Con-
ference on Graphics, Patterns and Images, 2010, pp. 360–367.
978-1-4799-5751-4/14/$31.00 ©2014 IEEE ICIP 20142957
... These undesirable residual regions often include desirable residual regions (which should be preserved) due to the design of the ultimate levelings which consider maximum residues. In this sense, several researches have been proposed in recent years, introducing strategies to filter undesirable residues during the residual extraction process [6,7,8,9,10]. ...
... Ultimate levelings constitute a wider class of residual operators defined from a scale-space of levelings {ψ i : i ∈ I} [6,8,9]. An ultimate leveling analyzes the evolution of residual values from a family of consecutive primitives, i.e. r + ...
... Those facts lead to efficient algorithms for computing ultimate levelings and its variations [8,14]. Ultimate levelings are operators that extract residual information from primitive families. ...
Chapter
Full-text available
Ultimate levelings are operators that extract important image contrast information from a scale space based on levelings. During the residual extraction process, it is very common that some residues are extracted from undesirable regions, but they should be filtered out. In order to attend this problem it can be used some strategies to filter residues extracted by ultimate levelings. In this paper, we selected desirable regions from the residual extraction process through a binary classifier. The selected regions are used later in a solution to the bounding box detection problem applied in a plant images dataset.
... Another application is reported in Ref. 11, where the UO is combined with the notion of area stability to analyze urban scenes. Derived from the UAO, in Ref. 12, the ultimate grain filter (UGF) was introduced as an operator based on numerical residues. The UGF analyzes an image under a series of grain filters of increasing grain sizes. ...
... E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 1 2 ; 6 3 ; 5 6 5 UEðXÞ ¼ Y 1 fd X ðxÞ − Rðd X ðxÞ; d X ðxÞ − 1Þg; (12) where Y t ðfÞ ¼ fx∶fðxÞ ≤ tg is the threshold at level t. Other recent residues take into account a family of openings fγ λ g and they are interesting for the gray-level case. ...
Article
Full-text available
This work is focused on the robustness of the ultimate opening (UO) and its associated function (AFUO) related to illumination changes and marker detection. It is well known that morphological filters commute to anamorphoses. This property ensures the robustness of the morphological filters to the lighting changes. However, the supremum of residues of these filters requires some conditions to commute to anamorphoses. Therefore, initially, a study of the robustness to anamorphoses is carried out. We demonstrate that the UO is invariant to linear anamorphoses and it is robust to power anamorphoses. On the other hand, related to marker detection, a study of the UO and AFUO is carried out. It is detected that small pieces of discs corrupt the UO, complicating use in marker detection. This behavior occurs because opening residues are not idempotent, so a modified UO is introduced to attenuate this inconvenience enabling a better selection of markers. Likewise, an example of a better selection of markers is carried out from images previously filtered and enhanced. Finally, the use of these transformations in image segmentation is illustrated.
... These structures are able to fully reconstruct images using theirs hierarchies and information stored in each node of the tree, which varies according to the chosen tree. A major advantage of using trees to represent images is that they can be used to perform connected filtering efficiently [1,4,5,6]. In particular, extensive and anti-extensive connected filtering can be performed efficiently using component trees through some strategies defined from information stored in tree nodes. ...
Article
Full-text available
This paper presents a machine vision system (MVS) for visual quality inspection of beans which is composed by a set of software and hardware. The software was built from proposed approaches for segmentation, classification, and defect detection, and the hardware consists of equipment developed with low-cost electromechanical materials. Experiments were conducted in two modes: offline and online. For offline experiments, aimed at evaluating the proposed approaches, we composed a database containing 270 images of samples of beans with different mixtures of skin colors and defects. In the online mode, the beans contained in a batch, for example, a bag of 1 kg, are spilled continuously on the conveyor belt for the MVS to perform the inspection, similar to what occurs in an automated industrial visual inspection process. In the offline experiments, our approaches for segmentation, classification, and defect detection achieved, respectively, the average success rates of 99.6%, 99.6%, and 90.0%. In addition, the results obtained in the online mode demonstrated the robustness and viability of the proposed MVS, since it is capable to analyze an image of 1280 × 720 pixels, spending only 1.5 s, with average successes rates of 98.5%, 97.8%, and 85.0%, respectively, to segment, classify, and detect defects in the grains contained in each analyzed image.
Article
Ultimate levelings are operators that extract important image contrast information from a scale-space based on levelings. Along with the residual extraction process, some residues usually come from undesirable regions, and they should be filtered out. For this, some strategies can be applied to filter these undesirable residues. In this paper, we introduce a new approach to detect desirable regions from ultimate levelings through a new hierarchical structure called residual tree. From this structure, we extract attribute vectors to build a machine learning model which gives a matching value between ground truth regions and residual tree nodes. Thus, from the selected residual tree nodes, we present a new approach to choose the best disjoint residual nodes which gives the regions of the ultimate levelings. Finally, from the ultimate levelings, we use its partition associated function to solve the segmentation problem. In order to evaluate our new approach, some experiments were carried out with a plant dataset and results report the state-of-the-art performance in plant segmentation.
Chapter
Ultimate levelings are operators that extract important image contrast information from a scale-space based on levelings. During the residual extraction process, it is very common that some residues are selected from undesirable regions, but they should be filtered out. In order to avoid this problem some strategies can be used to filter residues extracted by ultimate levelings. In this paper, we introduce a novel strategy to filter undesirable residues from ultimate levelings based on a regression model that predicts the correspondence between objects of interest and residual regions. In order to evaluate our new approach, some experiments were carried out with a plant dataset and the results show the robustness of our method.
Chapter
Bit-quads are \(2\times 2\) binary patterns which are counted within a binary image and can be used to compute attributes. Based on previous works which proposed an efficient algorithm to count bit-quads in component trees, in this paper, we discuss how we can count these patterns in tree of shapes by presenting two approaches. In the first one, we show how counting quads in component trees can be used to count them in tree of shapes by using the depth of the node as the value of pixels in a larger and interpolated image representation (used in an algorithm for constructing tree of shapes). In the second approach, we propose a novel algorithm which uses this larger image representation, but, the resulting quad counts are for the input image. In this way, our approach gives exactly the counts for the original image. We also provide experimental results showing that our algorithm is much faster than the non-incremental naive approach.
Conference Paper
Full-text available
In this work it is proposed an intelligent vision-based system for automatic classification of beans most consumed in Brazil. The system is able to classify the grains contained in a sample according to their skin colors, and is composed by three modules: image acquisition and pre-processing; segmentation of grains and classification of grains. In the conducted experiments, we used an apparatus controlled by a PC that includes a conveyor belt, an image acquisition chamber and a camera, to simulate an industrial line of production. The results obtained in the performed experiments indicate that the proposed system could be applied to visual quality inspection of beans produced in Brazil, since one of the steps in this process is the measurement of the mixture contained in a sample, taking into account the skin color of grains, for determining the predominant class of product and, consequently, its market price.
Conference Paper
Full-text available
To compute the morphological self-dual representation of images, namely the tree of shapes, the state-of-the-art algorithms do not have a satisfactory time complexity. Furthermore the proposed algorithms are only effective for 2D images and they are far from being simple to implement. That is really penalizing since a self-dual representation of images is a structure that gives rise to many powerful operators and applications, and that could be very useful for 3D images. In this paper we propose a simple-to-write algorithm to compute the tree of shapes; it works for nD images and has a quasi-linear complexity when data quantization is low, typically 12 bits or less. To get that result, this paper introduces a novel representation of images that has some amazing properties of continuity, while remaining discrete.
Article
Full-text available
Motivated by operators simplifying the topographic map of a function, we study the theoretical properties of two kinds of “grain” filters. The first category, discovered by L. Vincent, defines grains as connected components of level sets and removes those of small area. This category is composed of two filters, the maxima filter and the minima filter. However, they do not commute. The second kind of filter, introduced by Masnou, works on “shapes”, which are based on connected components of level sets. This filter has the additional property that it acts in the same manner on upper and lower level sets, that is, it commutes with an inversion of contrast. We discuss the relations of Masnou's filter with other classes of connected operators introduced in the literature. We display some experiments to show the main properties of the filters discussed above and compare them.
Article
Full-text available
Database indexing and retrieval tools can enormously benefit from the automatic detection and processing of textual data in images. We present a new connected-component based method using mor-phological numerical residues for automatic localization of text in general image database. This technique tends to be robust to font, scale and slant changes and detects superimposed as well as scene text. The robustness of our approach is proven by the results in ImagEval evaluation campaign, which database included old post-cards, graphic schemes, stamps, indoor and outdoor scene images and also images without any textual data. In spite of the wide va-riety of texts and images our approach obtains interesting results without parameter tuning for each image class.
Book
The tree of shapes of an image, as a unified mix of the component trees, was presented and analyzed in the present notes. Its existence was proven for a semicontinuous image. It relies on the simple model according to which shapes have mostly uniform reflectance properties and are distinguishable from the background. This model has no parameters and is well-posed, though it cannot really account for occlusions and subjective contours notably. The classical area openings of mathematical morphology have their counterpart in the grain filter, which has the advantage of being self-dual on continuous images. If small grains, or small shapes, are filtered out, the tree has a structure essentially finite, and a weak Morse theory was developed under these assumptions: several notions of singularity or criticality were defined and shown to be equivalent. To close the theoretical part, the procedure merging the component trees to build the tree of shapes was detailed, providing an algorithm valid in any dimension. In the most frequent two-dimensional case, a more efficient algorithm, the Fast Level Set Transform, was described. Though not as fast as an optimized component tree algorithm, the FLST is quite usable. A variant based on a bilinear interpolation of the image was also presented, which has the advantage of exhibiting more regular level lines. However, it is slightly slower than the FLST and needs as parameters the levels of quantization to represent the image in a finite structure. Finally, a wide range of concrete applications was exposed and briefly explained.
Conference Paper
This work introduces a residual operator called ultimate attribute leveling. We also present an efficient algorithm for ultimate attribute leveling computation by using a structure called tree of shapes. Our algorithm for computating ultimate attribute leveling is based on the fact (proved in this work) that levelings can be obtained by pruning nodes from the tree of shapes. This is a novel result, since so far it is known that levelings can be obtained from component trees. Finally, we propose the use of ultimate attribute leveling with shape information to extract contrast using a priori knowledge of an application. Experimental results applied to text location show the potentiality of using ultimate attribute leveling with shape information for solving problems in image processing area.
Article
The aim ofMorse theory is to describe the topological changes of the (iso)level sets of a real valued function in terms of its critical points. Our purpose in this chapter is to describe several different notions of critical values (one of them will be called critical value, the other singular value) and prove that they are equivalent. One of those notions, the one of singular value, is intuitively related to the classical notion of critical value for a smooth function (points where the gradient vanishes), the other notion called critical value is based on a computational approach. When N = 2, this algorithm computes the Morse structure of the image from its upper and lower level sets. When N ≥ 3, our notion of critical value is of topological nature and not equivalent to the notion of critical value used in classical Morse theory. Indeed, a function which embeds in its level sets the transformation of a sphere into a torus by pinching a hole into it may have no critical values (in our sense), besides a maximum. Due to that, we shall refer to it as a weak version of Morse theory of the topographic map. The results of this Chapter will be used in Chapter 5 in order to justify, in any dimension N ≥ 2, the construction of the tree of shapes by merging of the trees of connected components of upper and lower level sets. Let us mention that when N = 3 the topological type of level surfaces could be later computed on the tree. Since this is not essential for our purposes, we shall not pursue it in the present text.
Article
The filter that removes from a binary image the components with area smaller than a parameter λ is called area opening. Together with its dual, the area closing, it is first extended to grey-scale images. It is then proved to be equivalent to a maximum of morphological openings with all the connected structuring elements of area greater than or equal to λ. The study of the relationships between these filters and image extrema leads to a very efficient area opening/closing algorithm. Grey-scale area openings and closings can be seen as transformations with a structuring element which locally adapts its shape to the image structures, and therefore have very nice filtering capabilities. Their effect is compared to that of more standard morphological filters. Some applications in image segmentation and hierarchical decomposition are also briefly described.
Conference Paper
With the development of connected filters for the last decade, many algorithms have been proposed to compute the max-tree. Max-tree allows to compute the most advanced connected operators in a simple way. However, no fair comparison of algorithms has been proposed yet and the choice of an algorithm over an other depends on many parameters. Since the need of fast algorithms is obvious for production code, we present an in depth comparison of five algorithms and some variations of them in a unique framework. Finally, a decision tree will be proposed to help user in choosing the right algorithm with respect to their data.