Content uploaded by Diego Oliva
Author content
All content in this area was uploaded by Diego Oliva on Jan 26, 2023
Content may be subject to copyright.
1
Manta Ray Foraging Optimizer-based Image Segmentation with a
Two-Strategy Enhancement
Benedict Jun Ma a, João Luiz Junho Pereira b, Diego Oliva c, Shuai Liu d, Yong-Hong Kuo a e *
a Department of Industrial and Manufacturing Systems Engineering, The University of Hong Kong, Hong Kong.
b Computer Science Division, Aeronautics Institute of Technology, São José dos Campos, Brazil.
c Departamento de Ciencias Computacionales, Universidad de Guadalajara, Guadalajara, Mexico.
d Department of Logistics and Maritime Studies, Hong Kong Polytechnic University, Hong Kong.
e HKU Musketeers Foundation Institute of Data Science, The University of Hong Kong, Hong Kong.
* Corresponding author (yhkuo@hku.hk)
ABSTRACT
Image processing is an evolving field that calls for more powerful techniques to extract useful information from
images. In particular, image segmentation is a preprocessing step that helps separate objects in a digital image.
This article introduces an enhanced manta ray foraging optimizer (MRFO) based on two strategies – oppositional
learning (OL) and vertical crossover (VC) search – for color image segmentation. This combination technique
focuses on the enhancement of the explorative and exploitative cores, without compromising the computational
speed. The proposed algorithm, termed OL-MRFO-VC, is integrated with Kapur entropy to identify the best
threshold configuration in each image component (RGB). The technique is tested over three datasets consisting
of different scenes. The threshold vector consists of both lower and higher levels in the experiments. In addition,
OL-MRFO-VC is compared with fourteen competitive metaheuristics, and eleven measures are used to evaluate
their performance quantitatively and qualitatively. According to the computational results, our proposed method
outperforms state-of-the-art techniques, especially in the higher threshold levels. Furthermore, the p values in the
Wilcoxon signed-rank test confirm a significant improvement brought by our proposed method, suggesting a
superior capability of OL-MRFO-VC for solving image segmentation problems.
Keywords: Manta ray foraging optimization, Metaheuristic, Multilevel thresholding, Image processing,
Oppositional learning, Vertical crossover search
1. Introduction
1.1 Background and motivation
The image segmentation problem (ISP), an essential subject in image processing, refers to dividing a digital image
into distinct regions according to homogeneous characteristics [1–4], including pixel intensity, texture structure,
etc. [5–7]. Among the various image segmentation methods, thresholding occupies a pivotal position in terms of
simplicity, accuracy, and robustness [8–11]. Basically, thresholding can be categorized into two classes: bilevel
thresholding (binarization) and multilevel thresholding (MLT) [12]. Binarization classifies pixel intensity into
vectors of binary numbers based on the image histogram information, transforming the grayscale from a 0-255
spectrum to a 0-1 spectrum [13]. With this definition, bilevel is capable of tackling the segmentation of an image
that contains one object out of the background. However, it is more challenging to process complex color images.
Compared to the gray image segmentation, color images consist of three components (red, green, and blue color
values) that should be segmented respectively due to the dissimilar color constituents in each pixel. As a result,
MLT plays a vital role in color image segmentation, with broader applications.
2
MLT leverages at least two thresholds to separate an image into multiple classes. There are two categories of
MLT techniques in the literature: parametric and nonparametric approaches. Most parameter-based methods use
mathematical models and assume data follow certain distributions with parameters to estimate, and nonparametric
methods are commonly based on distribution-free methods or distributions with parameters unspecified [13]. One
of the well-known parametric methods is the Gaussian mixture model (GMM), in which it is indispensable to
identify a mixture of Gaussian density functions that approximate the histogram before calculating the thresholds
[14,15]. However, the segmentation performance in parametric techniques is highly dependent on the distribution
of the image histogram and the accuracy of the model parameter estimates [16]. Because of this, nonparametric
techniques in MLT are developed by which threshold values are usually calculated by criteria such as between-
class variance (Otsu) [17], Kapur entropy [18], and the variants based on Shannon entropy theory [4,19,20].
The primary challenge in MLT is the computation time, as the computational complexity increases exponentially
with the increasing number of thresholds [21]. Since MLT can be regarded as an NP-hard problem, metaheuristics
(MHs) have thereby been applied to effectively optimize the objective functions of the nonparametric thresholding
(e.g., Otsu and Kapur) [4]. MH is a high-level procedure designed to develop and identify a heuristic that may
yield reasonable solutions to optimization problems. MH utilizes the principles of swarm intelligence (SI) [22].
SI is part of evolutionary computation, which is one of the three main pillars of computational intelligence (CI),
in parallel with neural networks and fuzzy systems [23]. With the development of nature-inspired computing
paradigms over the past few years, this field of study has attracted much attention. MHs commonly originate and
are inspired by natural phenomena, providing the optimization framework with stochastic features that can
iteratively improve the current solutions [24–27]. Although a considerable number of MHs have been proposed,
none of them can guarantee global optimum in all optimization problems according to No-Free-Lunch theorem
[28]. For this reason, scholars are motivated to develop new algorithms or enhance existing methods, and apply
them to a variety of complex problems. As a result, in this paper, an enhanced version of the manta ray foraging
optimization (MRFO) is proposed for MLT-based color image segmentation. To the best of our knowledge, our
work is the first that applies it for color image segmentation, where its previous applications were on grayscale
image segmentation, for example in CT images [29], brain MR images [30], and benchmark images [31].
1.2 Related works
Some MH-based thresholding methods have been proposed over the past few years, where their effectiveness has
been demonstrated. Houssein et al. [13] used black widow optimization (BWO) in MLT image segmentation with
Otsu and Kapur as objective functions. Their results revealed that the BWO-based method outperforms other
algorithms, such as gray wolf optimization (GWO) and whale optimization algorithm (WOA), in terms of fitness
values and three other measures. Sharma et al. [32] combined the butterfly optimization algorithm (BOA) and
symbiosis organisms search (SOS), using the mutualism and parasitism phases from SOS to enhance the search
behavior in BOA. Three experiments, including benchmark functions, real-world problems, and the MLT problem,
were conducted to verify the overall performance of the newly proposed MPBOA regarding search behaviors and
convergence time. Wunnava et al. [33] applied Harris hawks optimization (HHO) with a mutation to restrict the
escape energy and an adaptive chance to decide whether the Harris hawk perches along with family members or
moves to a random tall tree. Then an improved 2D gray gradient (I2DGG) MLT method was investigated with
AHHO. The AHHO-based I2DGG performed better than 2D- and 1D-Tsallis entropy as evidenced from the
experimental results. Ren et al. [34] improved the slime mould algorithm with a new movement strategy and the
Gaussian kernel probability strategy, and the MGSMA was designed especially for MLT image segmentation.
Their experimental results reveal that it is a high-performing optimizer and can provide high-quality segmentation
results. Singh et al. [35] used a simple-to-implement and computationally inexpensive algorithm called learning
enthusiasm-based teaching-learning-based optimization (LebTLBO) for MLT. This optimizer was compared with
3
several state-of-the-art methods on both Otsu and Kapur entropy, verifying it was highly efficient in performance
metrics and segmentation quality.
The success of deep learning (DL) has prompted the development of image segmentation, and a comprehensive
review can be found in [36]. However, other non-DL-based methods, such as k-means clustering, thresholding,
and region-growing, are still being widely adopted in practice [36]. Since image segmentation is commonly
regarded as the preprocessing or postprocessing step in practical applications, the comparison of different image
segmentation methods (e.g., DL versus non-DL) makes no sense to some extent, as they are usually integrated to
make up for the deficiencies of both sides. There has been substantial research which combines the MLT and DL
techniques to solve image-related problems. For example, Kumar et al. [37] used modified deep learning and
multilevel thresholding to detect brain tumor sizes. In their study, MLT is considered the preprocessing step, and
then the images are classified by the DL models. The purpose of our work is to improve the performance (e.g.,
accuracy, robustness, and computation time) of MLT-based methods. Although diversified MHs have been
employed in MLT, most of them only focus on gray images and lower threshold levels, which are less realistic in
actual implementations. Therefore, in this paper, we mainly research multilevel thresholding of color images
rather than grayscale images. Besides, higher threshold levels are another challenge considered in this study.
On the other hand, the MRFO algorithm [38] is an effective bioinspired optimizer for engineering applications
that imitates the three foraging behaviors of manta rays, including chain, cyclone, and somersault. Even though
MRFO is a newly proposed optimizer, scholars in different research fields have validated its performance. For
example, it has been applied to effectively solve the economic emission dispatch (EED) problems [39], feature
selection (FS) problems [40], maximum power point tracker (MPPT) [41], etc.
The MRFO has proven to be an acceptable alternative for solving complex optimization problems. Nevertheless,
as with other metaheuristics, MRFO suffers from some drawbacks. For example, the exploitation operators do
not follow a standard way to identify the optimum [42]. Additionally, premature convergence can sometimes be
an issue in MRFO [43]. These facts affect the diversity of the population along the iterative process and produce
stagnation in suboptimal solutions, whereas this can be mitigated and partly overcome by improving its search
diversification. Note that in MHs, it is always expected to have a good balance between the exploration of the
search space and the exploitation of most of its prominent regions [44].
1.3 Contribution statement
Based on the information provided in Subsection 1.2, this paper presents an enhanced version of the MRFO that
incorporates two important operators to balance exploration and exploitation in the search process. The modified
algorithm, OL-MRFO-VC, combines the MRFO with oppositional learning (OL) [45] and vertical crossover (VC)
search [46]. The former ensures the improvement of MRFO diversification, and the latter aims not to significantly
impair the primal convergence speed. The motivation for this improvement is to generate an efficient optimization
tool that facilitates a proper analysis of the search space and the best solutions for the problem. OL-MRFO-VC is
tested using benchmark functions with multi-dimensional search spaces and complex landscapes [47]. In addition,
experiments and comparisons with comparable approaches were conducted to validate the effectiveness of the
enhancement on MRFO in the MLT segmentation of color images. The novelty of this article is two-fold. First,
we enhance the MRFO capabilities by applying OL and VC. Second, our proposed approach effectively segments
digital images from different sources. The main highlights of this paper are outlined as follows:
1) Two strategies are embedded in MRFO to propose a global optimizer named OL-MRFO-VC.
2) The OL-MRFO-VC is tested over complex optimization problems with high dimensionality.
3) The enhanced version of the MRFO is applied to solve the image segmentation problem based on multilevel
thresholding and Kapur entropy.
4
4) The performance of the proposed algorithm is validated on both the lower and higher threshold levels in color
image segmentation.
5) OL-MRFO-VC is compared to a total of fourteen state-of-the-art metaheuristics.
6) The evaluation of a total of fourteen algorithms is measured by eleven metrics.
The rest of the paper is organized as follows. Section 2 briefly introduces MRFO. Section 3 illustrates an enhanced
MRFO based on a two-strategy framework. Section 4 describes the multilevel thresholding based on the proposed
algorithm and Kapur entropy. Section 5 shows the experimental settings. Section 6 shows the experimental results
and analyses. Section 7 concludes our work and suggests future research directions. In addition, supplementary
figures and tables are presented in Appendices A-D.
2. Manta ray foraging optimizer
Manta ray foraging optimizer (MRFO) simulates the foraging behaviors of manta rays in oceans, preying on
plankton with their mouths that have horn-shaped cephalic lobes [38]. As a gigantic marine creature, manta rays
have evolved various intriguing foraging strategies to meet their daily food demand, including chain foraging,
cyclone foraging and somersault foraging. The details and relevant mathematical models of these three behaviors
are given in the following subsections.
2.1 Chain foraging
Manta rays are generally observed to prey on plankton in groups, in which they commonly form an orderly chain
in succession. Since the missed plankton from the previous manta rays will be eradicated by those behind them,
a group of manta rays in a line can funnel the most amount of food into their gills. In MRFO, the best solution
denotes plankton with the highest concentration that manta rays want to approach and eat. Considering the chain
foraging behavior, each individual updates its position iteratively by the best solution obtained thus far and the
solution in front of it. Then, the mathematical model in this phase is shown as follows:
(2.1)
(2.2)
where indicates the position of the manta ray (an agent) at the iteration; is a random vector within
;
indicates the position of the plankton with the highest concentration (global best solution); refers
to a weight coefficient relying on ; is the total number of agents.
As represented in Eq. (2.1), at each iteration in the search space, the agent updates its position () according
to the position of the agent (
) and the position of the food (
).
2.2 Cyclone foraging
When plankton are concentrated regularly in deep oceans, tens of manta rays line up head-to-tail to generate a
spiral vortex and swim spirally toward the food. This cyclone foraging strategy makes each manta ray move
toward plankton in a spiral path; simultaneously, it keeps following the previous individual to update its position.
In fact, the whale optimization algorithm (WOA) [48] is inspired by a similar spiral movement of whales; however,
the difference between WOA and MRFO is that a head-to-tail chain is considered in the cyclone foraging behavior
as well. The mathematical model in this strategy is represented as follows:
(2.3)
(2.4)
5
where indicates the maximum iteration; both and are random vectors ranging in ; and is a weighting
factor deciding the spiral shape.
It can be seen in Eq. (2.3) that each agent updates its position () with respect to the best solution (
),
suggesting the good exploitative ability. To improve the balance between the exploration and exploitation stages,
it is necessary to broaden the region of agents' activities. Therefore, in MRFO, a random position is generated
within the search space to force each manta ray to swim as far away from the current best solution as possible.
This mechanism is modeled mathematically as follows:
(2.5)
(2.6)
where
indicates a new position yielded randomly, and and are the lower and upper boundaries of
the search space, respectively.
2.3 Somersault foraging
The somersault behavior of manta rays is an unusual foraging scenario in which they swim to and do backward
somersaults around the plankton randomly and circularly. In MRFO, plankton are regarded as a pivot, and manta
rays always update their current positions around the best solution found thus far to optimize food intake. The
mathematical formula regarding somersault foraging is presented below.
(2.7)
where indicates the somersault weight deciding the range of somersault behaviors, and here ; and
are two random vectors within .
The above three subsections describe the main iterative updating mechanisms of manta rays in MRFO, in which
the chain foraging forces each agent to update its position regarding the current best agent and the agent in front
of it. The cyclone foraging allows each agent to update its positions with respect to the one in front of it and the
best solution obtained thus far, or the agent generated randomly. The somersault foraging makes each agent
adaptively move in a dynamic search range. The pseudocode of MRFO is shown in Figure A.1.
3. Enhanced MRFO
In general, the iterative process of a metaheuristic can be split into three phases: exploration, hybrid exploration
and exploitation, and exploitation. The explorative stage reflects the diversity of the agents in the global search
space, while the exploitative stage focuses on the search ability in a local area. The middle stage, consisting of
both exploration and exploitation, is another essential component in deciding the overall performance of an
optimizer. During the exploration process, agents are required to explore the whole search space randomly and
sufficiently to avoid local stagnation, and individuals gather the information iteratively from the explorative
aspect to find the global optima in the exploitative aspect. It should be noted that excessive exploration will
degrade the convergence rate, and excessive exploitation may lead to the loss of potential global optima.
Consequently, an appropriate balance between exploration and exploitation makes an optimization algorithm
more efficient and less time-consuming, which is the prerequisite to being considered a state-of-the-art optimizer.
In this paper, the basic MRFO is enhanced by two effective strategies based on the above theoretical illustration
to improve its performance and applied to multilevel thresholding problems.
3.1 Oppositional learning
6
Based on the analysis and conclusion derived from [31], the exploration ability of MRFO is inferior compared to
its exploitation ability. Consequently, a technique named oppositional learning (OL) [45], a well-demonstrated
strategy by many studies, is used in the initial stage to improve MRFO diversification.
The OL approach diversifies the population by generating a set of opposite positions in accordance with the
current positions of the agents in the search space. Suppose is defined as a real number over the interval
, and the opposite number denoted by is calculated by Eq. (3.1).
(3.1)
The above definition can be generalized to a -dimensional space with the -agent in a matrix Eq. (3.2), and the
relevant opposite matrix is presented in Eq. (3.3), in which each element is calculated by Eq. (3.4).
(3.2)
(3.3)
(3.4)
Regarding as the fitness function, if
is superior to in a particular problem, then
;
otherwise, .
3.2 Vertical crossover
Focusing on the task of multilevel thresholding in this study, the vertical crossover operator from the crisscross
optimization algorithm (CSO) [46] is combined into MRFO. In CSO, horizontal crossover and vertical crossover
are two different search strategies. The horizontal crossover operates on all dimensions between two agents, while
the vertical crossover operates on all agents between two dimensions. Many studies adopted either horizontal
crossover or vertical crossover or both search behaviors in CSO to mitigate some shortcomings in other
optimization algorithms. For example, Liu et al. [49] combined horizontal and vertical crossover mechanisms and
Nelder-Mead simplex with the Harris hawks optimizer (HHO) to estimate the parameters of photovoltaic (PV)
models. The experimental results revealed that the modified algorithm was competitive in extracting the unknown
parameters of different PV models and complex outdoor environments.
The vertical crossover is an arithmetic crossover operated on two different dimensions in each search agent, which
prevents individuals from local stagnation to some extent without significantly impairing the convergence speed.
One of the significant advantages of combining the vertical crossover mechanism in the multilevel thresholding
problem is that high threshold levels will be selected to complete the image segmentation, which is equal to the
dimension of the search space in programming. Moreover, the main effect of the vertical crossover from previous
studies was the avoidance of falling into the local optimum and improvement of solution accuracy.
Assuming that and are two dimensions in the individual, then the moderation solution () calculated
by a vertical crossover is defined as:
(3.5)
where is a random number distributed uniformly within ; and refer to the value of the
individual in the -dimension and -dimension, respectively;
indicates the offspring of and
in the vertical crossover.
3.3 The proposed OL-MRFO-VC
7
In this paper, both the OL strategy and the VC mechanism are combined with the standard MRFO to construct
the enhanced MRFO algorithm, named OL-MRFO-VC. On the one hand, OL allows for a high-quality initial
population distributed more uniformly in each dimension of the search space to improve its exploratory search
capability. On the other hand, vertical crossover is able to upgrade the exploitative ability in a local area to mitigate
local stagnation. At the same time, the convergence speed of MRFO would not be significantly impaired in OL-
MRFO-VC according to the previous analysis. The pseudocode of the proposed algorithm is shown in Figure 1.
Input: (population); (dimension); (maximum iteration); (upper bound); (lower bound).
%% Initialize randomly %%
for
for
.
. // OL from Eq. (3.4)
end for
end for
Fitness function .
for
if then
.
end if
end for %% Update the position of each agent %%
Initialize the iteration count .
while
for
if then
if then
Update using Eq. (2.5). // Cyclone foraging with random
else if
Update using Eq. (2.3). // Cyclone foraging
end if
else if
Update using Eq. (2.1). // Chain foraging
end if
end for
for
Update using Eq. (2.7). // Somersault foraging
end for
for
Generate a random number .
if then
.
.
for
. // Vertical crossover: Eq. (3.5)
end for
end if
end for
end while
Output:
Fig. 1 Pseudocode of OL-MRFO-VC.
4. Multilevel thresholding
The between-class variance (Otsu) and Kapur entropy are two popular and effective techniques in multilevel
thresholding. From the experiments and conclusion in [50], the authors proposed a novel multilevel thresholding
technique named SAMFO-TH consisting of two modifications. Their experiments were performed on ten color
images, six benchmark images and four satellite images, with both the Otsu and Kapur entropy approaches. In
addition, the authors compared the performance of these two thresholding techniques with structural similarity
(SSIM) and feature similarity (FSIM) metrics. The relevant experimental results revealed that Kapur outperforms
Otsu most of the time in the selected algorithms. Therefore, in this paper, we focus on evaluating the segmentation
performance of OL-MRFO-VC with Kapur entropy.
8
4.1 Kapur entropy
In an RGB digital image, each color component consists of pixels and gray levels associated with the
histogram information. The image histogram is constructed, in which refers to the frequency
of the gray level and its probability can be represented as:
(4.1)
Suppose there are thresholds used to segment the image into classes and yield a threshold vector
, hence the entropy of each class is calculated by:
(4.2)
where indicates the entropy value of the class, and indicates the probability of class in the image.
Then the objective (fitness) function can be modeled as follows:
(4.3)
4.2 OL-MRFO-VC based thresholding
In Section 3, we propose a new algorithm named OL-MRFO-VC. In this study, Eq. (4.3) will be considered the
fitness function employed in MHs. The flowchart of OL-MRFO-VC in finding the optimal threshold vector with
Kapur entropy is shown in Figure 2.
Fig. 2 Flowchart of OL-MRFO-VC with Kapur entropy.
5. Experimental settings
In this section, extensive experiments have been conducted to validate the proposed algorithm's performance as
compared to the standard MRFO and some other state-of-the-art MHs. There are two series of experiments. The
first is to conduct an ablation experiment to show the effectiveness of each individual strategy, i.e., OL and VC.
9
In the second part (Section 5.3), OL-MRFO-VC is applied for the multilevel thresholding problem over different
digital images. All the experiments are performed with comparisons, validations using metrics, statistical analysis,
and a nonparametric test.
5.1 Ablation experiment
In this subsection, an ablation experiment is made to assess the effectiveness of the two strategies, OL and VC.
A total of four optimizers (MRFO, OL-MRFO, MRFO-VC and OL-MRFO-VC) are tested on 23 well-known
benchmark functions [47]. Different dimensions (D = 30 and D = 100) are examined to demonstrate the robustness
of the results. In this part, the number of iterations is set as 500, and the number of agents is set as 20. In addition,
each algorithm under each function runs 30 times. Table 1 presents the fitness values on benchmark functions.
Table 2 presents the computation time.
According to Table 1, when D = 30, we find that the proposed OL-MRFO-VC achieves the best performance (21
out of 23 test functions), followed by the OL-MRFO and MRFO, and finally, the MRFO-VC. More specifically,
it is surprising to find that MRFO-VC underperforms the basic model, while the combination of oppositional
learning and vertical crossover offers significant benefits in improving the explorative and exploitative cores of
MRFO, as OL-MRFO-VC outperforms MRFO. For the higher dimension (D = 100), the proposed approach also
performs better than the basic model, while MRFO is better than OL-MRFO and MRFO-VC. This result validates
an appropriate and effective choice of two strategies. Neither of them can significantly improve the performance
of MRFO, but their integration can make it. In addition, all the standard deviation (Std) values (Std = 0.000E+00)
suggest the stability of the MRFO algorithm.
Table 1 Fitness values on the benchmark functions.
D = 30
D = 100
MRFO
OL-MRFO
MRFO-VC
OL-MRFO-
VC
MRFO
OL-MRFO
MRFO-VC
OL-MRFO-
VC
F1
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
F2
3.500E-197
2.363E-204
5.477E-180
5.062E-205
2.336E-201
7.093E-197
9.922E-175
1.181E-202
F3
0.000E+00
0.000E+00
2.561E-286
0.000E+00
0.000E+00
0.000E+00
3.538E-275
0.000E+00
F4
2.402E-197
1.324E-198
6.621E-174
9.706E-203
4.117E-194
9.721E-191
4.270E-171
6.829E-202
F5
2.398E+01
2.412E+01
2.686E+01
2.348E+01
9.600E+01
9.564E+01
9.730E+01
9.491E+01
F6
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
F7
2.540E-04
4.219E-04
2.257E-04
5.181E-05
4.071E-05
9.244E-05
1.314E-04
5.536E-05
F8
-9.036E+03
-9.411E+03
-7.129E+03
-8.246E+03
-2.427E+04
-2.378E+04
-1.290E+04
-2.349E+04
F9
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
F10
8.882E-16
8.882E-16
8.882E-16
8.882E-16
8.882E-16
8.882E-16
8.882E-16
8.882E-16
F11
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
0.000E+00
F12
1.323E-10
4.088E-11
2.468E-01
3.152E-10
1.499E-02
1.708E-02
5.375E-01
1.672E-02
F13
2.966E+00
2.966E+00
2.974E+00
2.966E+00
9.896E+00
9.899E+00
9.947E+00
9.893E+00
F14
9.980E-01
9.980E-01
9.980E-01
9.980E-01
/
/
/
/
F15
3.075E-04
1.224E-03
3.201E-04
3.075E-04
/
/
/
/
F16
-1.032E+00
-1.032E+00
-1.032E+00
-1.032E+00
/
/
/
/
F17
3.979E-01
3.979E-01
3.979E-01
3.979E-01
/
/
/
/
F18
3.000E+00
3.000E+00
3.000E+00
3.000E+00
/
/
/
/
F19
-3.863E+00
-3.863E+00
-3.863E+00
-3.863E+00
/
/
/
/
F20
-3.322E+00
-3.203E+00
-3.318E+00
-3.322E+00
/
/
/
/
F21
-5.055E+00
-1.015E+01
-1.015E+01
-1.015E+01
/
/
/
/
F22
-5.088E+00
-5.088E+00
-5.088E+00
-1.040E+01
/
/
/
/
F23
-5.128E+00
-5.128E+00
-5.128E+00
-1.054E+01
/
/
/
/
Rank avg.
1.7826
1.8261
2.2174
1.0870
1.6667
2.000
2.9167
1.1667
Rank
3
2
4
1
2
3
4
1
According to Table 2, we clearly observe that the proposed OL-MRFO-VC does not impair the computation time
of MRFO, which is even faster than the basic version. In total, 0.1691s is derived by OL-MRFO-VC, compared
with the 0.1694s achieved by MRFO in Table 3. When D = 100, OL-MRFO-VC achieves the shortest computation
time compared to OL-MRFO, which is the best computation time when D = 30. In addition, the MRFO-VC has
a longer time than the rest of the three, but the vertical crossover does not significantly impair the computation
time in OL-MRFO-VC.
10
Overall, from the pretest on benchmark functions, OL can enhance the original performance of MRFO, while its
combination of the VC can further improve the accuracy without impacting the computation time. In this view,
OL-MRFO-VC has the potential to complete the MLT segmentation of color images.
Table 2 Computation time (in seconds) on the benchmark functions.
D = 30
D = 100
MRFO
OL-MRFO
MRFO-VC
OL-MRFO-VC
MRFO
OL-MRFO
MRFO-VC
OL-MRFO-VC
Mean
Std
Mean
Std
Mean
Std
Mean
Std
Mean
Std
Mean
Std
Mean
Std
Mean
Std
F1
0.0901
0.0121
0.0874
0.0054
0.1437
0.0098
0.0883
0.0082
0.1355
0.0033
0.1344
0.0031
0.3150
0.0080
0.1349
0.0050
F2
0.0976
0.0132
0.0939
0.0070
0.1484
0.0079
0.0946
0.0062
0.1436
0.0034
0.1437
0.0048
0.3309
0.0118
0.1427
0.0041
F3
0.4320
0.0124
0.4346
0.0125
0.4857
0.0123
0.4399
0.0174
1.6109
0.0761
1.6195
0.0947
1.7974
0.0941
1.6071
0.0781
F4
0.0852
0.0023
0.0853
0.0032
0.1406
0.0039
0.0857
0.0034
0.1356
0.0080
0.1330
0.0060
0.3199
0.0137
0.1313
0.0053
F5
0.1064
0.0037
0.1068
0.0047
0.1609
0.0055
0.1051
0.0031
0.1559
0.0078
0.1559
0.0070
0.3388
0.0124
0.1554
0.0050
F6
0.0892
0.0076
0.0887
0.0119
0.1424
0.0063
0.0863
0.0045
0.1346
0.0034
0.1327
0.0023
0.3143
0.0084
0.1330
0.0027
F7
0.1615
0.0038
0.1643
0.0116
0.2235
0.0125
0.1607
0.0037
0.3628
0.0061
0.3633
0.0072
0.5509
0.0172
0.3636
0.0077
F8
0.1089
0.0045
0.1074
0.0038
0.1641
0.0085
0.1088
0.0046
0.1843
0.0039
0.1835
0.0049
0.3668
0.0100
0.1848
0.0045
F9
0.0905
0.0036
0.0902
0.0034
0.1475
0.0072
0.0905
0.0038
0.1479
0.0073
0.1459
0.0047
0.3269
0.0086
0.1467
0.0068
F10
0.1005
0.0067
0.0989
0.0058
0.1540
0.0067
0.0982
0.0034
0.1577
0.0116
0.1562
0.0113
0.3378
0.0101
0.1545
0.0082
F11
0.1164
0.0069
0.1155
0.0061
0.1752
0.0142
0.1161
0.0086
0.1803
0.0058
0.1822
0.0050
0.3684
0.0132
0.1831
0.0132
F12
0.3078
0.0104
0.3068
0.0099
0.3635
0.0112
0.3086
0.0085
0.6999
0.0136
0.7005
0.0143
0.8822
0.0152
0.7001
0.0151
F13
0.3072
0.0130
0.3046
0.0098
0.3608
0.0076
0.3057
0.0121
0.6871
0.0252
0.6864
0.0230
0.8746
0.0353
0.6837
0.0160
F14
0.6535
0.0285
0.6611
0.0413
0.6592
0.0272
0.6580
0.0308
/
/
/
/
/
/
/
/
F15
0.0726
0.0027
0.0741
0.0048
0.0809
0.0035
0.0733
0.0034
/
/
/
/
/
/
/
/
F16
0.0662
0.0032
0.0676
0.0049
0.0711
0.0041
0.0671
0.0034
/
/
/
/
/
/
/
/
F17
0.0634
0.0037
0.0619
0.0027
0.0649
0.0025
0.0620
0.0043
/
/
/
/
/
/
/
/
F18
0.0588
0.0053
0.0587
0.0084
0.0619
0.0046
0.0584
0.0048
/
/
/
/
/
/
/
/
F19
0.0936
0.0043
0.0918
0.0028
0.0992
0.0149
0.0931
0.0045
/
/
/
/
/
/
/
/
F20
0.0942
0.0034
0.0929
0.0023
0.1050
0.0042
0.0936
0.0036
/
/
/
/
/
/
/
/
F21
0.1623
0.0146
0.1612
0.0060
0.1690
0.0064
0.1601
0.0051
/
/
/
/
/
/
/
/
F22
0.1976
0.0058
0.1966
0.0052
0.2030
0.0053
0.1996
0.0113
/
/
/
/
/
/
/
/
F23
0.2613
0.0216
0.2550
0.0089
0.2633
0.0105
0.2551
0.0064
/
/
/
/
/
/
/
/
Avg.
0.1694
0.0082
0.1690
0.0080
0.2020
0.0085
0.1691
0.0071
0.3834
0.0144
0.3836
0.0154
0.5674
0.0208
0.3822
0.0139
5.2 Evaluation metrics in MLT
To evaluate the segmentation performance of various algorithms quantitatively and qualitatively, eleven measures
are considered to make a detailed comparison.
(1) Fitness value
Equation (4.3) is regarded as the fitness function in all the selected MHs, and then the fitness values are calculated.
Since maximum solutions are required in Kapur entropy; thus, the higher fitness values yielded indicate the better
performance of an algorithm.
(2) Standard deviation (Std)
Each algorithm has 30 runs in each benchmark image to avoid the stochastic characteristics; the standard deviation
of fitness values among the 30 runs is calculated to assess the stability of different algorithms.
(3) Peak signal-to-noise ratio (PSNR)
Peak signal-to-noise ratio (PSNR) [51] measures the difference between the original and the segmented images,
and higher values suggest a better effect. The calculation of PSNR is as follows:
(5.1)
(5.2)
where and indicate the image's size; and refer to the original and segmented images, respectively.
(4) Structural similarity (SSIM)
Structural similarity (SSIM) [52] defines the similarity between the original and the segmented images within
, and the higher values suggest a better performance is obtained. The calculation of SSIM is as follows:
(5.3)
11
where and are the mean intensities of the original and segmented images; and are the standard
deviation of the original and segmented images; is the covariance of these two images; and are two
constants set as 6.52025 and 58.52252, respectively.
(5) Feature similarity (FSIM)
Feature similarity (FSIM) [53] measures the interdependent information between the original and the segmented
images within . A higher value suggests better performance. The calculation of FSIM is as follows:
(5.4)
(5.5)
(5.6)
(5.7)
(5.8)
(5.9)
(5.10)
where is the region of the image; is the gradient magnitude of the image; indicates the magnitude of the
response vector in on ; indicates the local amplitude of the scale ; is a small positive number.
(6) Probability rand index (PRI)
Probability rand index (PRI) [54] calculates the fraction of the pairs of pixels whose labels are consistent between
the ground truth and the segmented image within . The higher values of PRI suggest a better segmentation
performance, and the formulation of PRI is as follows:
(5.11)
(5.12)
(5.13)
where is the number of pixels; is the ground truth probability that .
(7) Variation of information (VoI)
Variation of information (VoI) [55] measures the amount of randomness in one segmentation that another one
cannot explain through the distance between two clusterings. The lower VoI values suggest a better segmentation
performance, and the calculation of VoI is as follows:
(5.14)
(5.15)
(5.16)
where and refer to the entropies of and the mutual information between the two clusterings, respectively.
(8) Computation time
Computation time is essential in comparing different algorithms in terms of computational complexity.
(9) Wilcoxon signed-rank test
This is a nonparametric statistical hypothesis test used to check whether there is a significant difference between
the two compared algorithms. The null hypothesis here is constructed as follows. There is no significant difference
between the two compared algorithms. When , it suggests that the null hypothesis can be rejected at a
5% significance level.
(10) Convergence curves
12
The convergence curves perform the average best-so-far progress over the iterative number, which serves as a
one-running result to compare various algorithms in convergence rate clearly.
(11) Segmented images
The segmented images from ten competitive algorithms at different threshold levels are presented and compared,
and some evident defects can be found in some low-performance algorithms.
5.3 MLT experiments
A set of experimental simulations are performed in the MATLAB R2017b environment with an Intel i7-8700
CPU @ 3.20 GHz (RAM 16.0 GB).
5.3.1 Experiment 1: BSD500 dataset
Ten color benchmark images (denoted as Test 1-Test 10) from the Berkeley Segmentation Dataset (BSD500) [56]
are used to test the performance of the different algorithms, shown in Figure A.2, with the corresponding ground
truth pictures and histogram information. Importantly, two benchmark images (Test 7: 198004 and Test 8: 157055)
involving human facial parts are hidden for privacy protection.
The threshold levels for image segmentation in this part consist of the lower (K=4, 6, 8, 10) and the higher (K=15,
20, 25, 30) to identify different algorithms' performance. It noted that with the increase in threshold levels in
thresholding, the computational complexity upgrades exponentially. Due to the stochastic characteristic of MHs,
each MH independently runs 30 times for fair comparisons.
In addition to OL-MRFO-VC and MRFO, eight other well-known algorithms are used for comparison, including
the sine cosine algorithm (SCA) [57], artificial bee colony (ABC) [58], particle swarm optimization (PSO) [59],
salp swarm algorithm (SSA) [60], flower pollination algorithm (FPA) [61], multiverse optimization (MVO) [62],
harmony search algorithm (HSA) [63] ant lion optimization (ALO) [64]. The values of some specific parameters
in the ten competitive algorithms are shown in Table A.1.
5.3.2 Experiment 2: Cityscapes dataset
In this subsection, nine images (denoted as E1–E9, urban street scenes) from the large-scale Cityscapes dataset
[65,66] are tested for color image segmentation. These benchmark images are shown in Figure A.3. This dataset
has been used widely to test the performance of models and algorithms. In this subsection, five recent (after 2019)
MHs are selected for comparison: slime mould algorithm (SMA) [67], hunger games search (HGS) [68], Runge-
Kutta optimizer (RUN) [69], seagull optimization algorithm (SOA) [70], and tunicate swarm algorithm (TSA)
[71].
Our proposed OL-MRFO-VC is compared with MRFO and the aforementioned optimizers. The threshold levels
are chosen in K = 10, 20, and 30. Also, the metrics (Std, PSNR, SSIM, FSIM, PRI, VoI, and computation time)
are computed for evaluation. The other experimental settings are consistent with those in Experiment 1.
5.3.3 Experiment 3: Intel image dataset
In this subsection, 50 images (denoted as R1–R50) from the Intel image dataset [72] are adopted for the
experiments related to image segmentation, as shown in Figure A.4. The proposed OL-MRFO-VC is compared
with all the optimizers mentioned in Experiment 1 and Experiment 2, while the number of thresholds is chosen
from K=10, 20, and 30. In addition, the metrics (PSNR, SSIM, FSIM, PRI, VoI, and computation time) are
evaluated for comparison. The other experimental settings are consistent with those in Experiment 1.
6. Experimental results and analyses
6.1 BSD500 dataset
13
The results of all eleven measures are presented in Tables B.1-B.21. Moreover, Figures 4-17 visualize the values
of metrics (2)-(8).
Fig. 3 Legend of Figures 4-17.
Fig. 4 Std values in lower threshold levels
(Tests 1-10).
Fig. 5 Std values in higher threshold levels
(Tests 1-10).
Fig. 6 PSNR values in lower threshold levels
(Tests 1-10).
Fig. 7 PSNR values in higher threshold levels
(Tests 1-10).
Fig. 8 SSIM values in lower threshold levels
(Tests 1-10).
Fig. 9 SSIM values in higher threshold levels
(Tests 1-10).
Fig. 10 FSIM values in lower threshold levels
(Tests 1-10).
Fig. 11 FSIM values in higher threshold levels
(Tests 1-10).
Fig. 12 PRI values in lower threshold levels
(Tests 1-10).
Fig. 13 PRI values in higher threshold levels
(Tests 1-10).
Fig. 14 VoI values in lower threshold levels
(Tests 1-10).
Fig. 15 VoI values in higher threshold levels
(Tests 1-10).
Fig. 16 Computation time in lower threshold
levels (Tests 1-10).
Fig. 17 Computation time in higher threshold
levels (Tests 1-10).
6.1.1 Analysis of fitness values
14
Tables B.1-B.4 present the fitness values of three color components obtained from all ten competitive algorithms
in ten benchmarks. It can be seen that the proposed OL-MRFO-VC achieves the highest values most of the time,
followed by MRFO, demonstrating the superiority of MRFO in multilevel thresholding and the effectiveness of
our modifications. Besides, we observe that SSA and ALO perform well in lower threshold levels (K = 4, 6, 8,
and 10) to some extent. However, there is a more apparent gap between OL-MRFO-VC with these two algorithms
when in higher threshold levels (K = 15, 20, 25, and 30), revealing their shortcomings in the more complicated
and higher dimensional cases.
6.1.2 Analysis of Std
Tables B.5-B.6 present the standard deviation of fitness values over 30 runs, and Figures 4-5 visualize the data.
It is clear to find that OL-MRFO-VC receives lower values of Std in both the lower and higher threshold levels,
suggesting the stability of MRFO in this subject. In addition, we observe that all the algorithms have higher Std
values with increasing threshold levels, showing the challenge of the thresholding techniques at high levels.
6.1.3 Analysis of PSNR
Tables B.7-B.8 present the PSNR values of all the competitive algorithms, and Figures 6-7 visualize the data.
We can draw a conclusion from the PSNR results that OL-MRFO-VC outperforms the other start-of-the-art MHs
selected in this paper since higher values of this metric indicate a better segmentation performance. Besides, SCA
shows its disadvantages in the multilevel thresholding task, especially in higher levels. Comparing SCA and FPA
in detail, it is counterintuitive to find that FPA performs better than SCA at higher levels, while SCA outperforms
FPA at lower levels.
6.1.4 Analysis of SSIM
Tables B.9-B.10 present the SSIM values of all the competitive algorithms, and Figures 8-9 visualize the data.
Performing a deep comparison of SSIM in the lower and higher threshold levels shows that the positive difference
between OL-MRFO-VC and the other competitors is more apparent in the higher levels than in the lower levels,
which suggests the superiority of the proposed algorithm in more complicated scenarios.
6.1.5 Analysis of FSIM
Tables B.11-B.12 present the FSIM values of all the competitive algorithms, and Figures 10-11 visualize the
data. It is intuitive to conclude that all the algorithms achieve better values with increasing threshold levels since
more regions with different grayscales are formed. In addition, the curves in Figure 11 are sparser than those in
Figure 10, validating the usefulness of the two strategies to make MRFO robust in complex conditions.
6.1.6 Analysis of PRI
Tables B.13-B.14 present the PRI values of all the competitive algorithms, and Figures 12-13 visualize the data.
It can be seen that the difference between the proposed algorithm and the other compared MHs is not noticeable
in lower levels from Figure 12, although OL-MRFO-VC receives the highest values in PRI most of the time from
Table B.13. However, it is clearer to note that OL-MRFO-VC outperforms the others in Figure 13. For example,
OL-MRFO-VC obtains a peak over the rest of the competitors in Test 5.
6.1.7 Analysis of VoI
Tables B.15-B.16 present the VoI values of all the competitive algorithms, and Figures 14-15 visualize the data.
It is intuitive to observe that OL-MRFO-VC outperforms the other algorithms due to the lowest VoI values
obtained most of the time. It is similar to the other metrics that the proposed algorithm has a more apparent
improvement in the higher threshold levels than in the lower levels compared to all the other MHs. It should be
noted that there is a peak in Test 2 with the lower levels and in Test 5 with the higher levels.
15
6.1.8 Analysis of computation time
Tables B.17-B.18 present the computation time of the competitive algorithms, and Figures 16-17 visualize the
data. It can be seen that ALO spends much more computation time than the others, and HSA requires minimal
running time. Besides, the computation time spent in OL-MRFO-VC and MRFO is close overall. According to
the illustration earlier, the computation time rises with increasing threshold levels. However, except for ALO, the
difference in computation time of all the other MHs is negligible in Figure 17 compared to Figure 16.
6.1.9 Analysis of Wilcoxon rank test
Table 3 presents the Wilcoxon signed-rank test result regarding seven quantitative metrics, in which the proposed
OL-MRFO-VC is compared to the remaining nine MHs across the lower and higher threshold levels in ten color
images. We observe a significant difference between the proposed algorithm and the others in terms of Std, PSNR,
SSIM, FSIM, PRI, and VoI. For the computation time, it is counterintuitive to find that OL-MRFO-VC is not
significantly different from MRFO at the lower threshold levels (), but a significant difference is yielded
from the statistical results at the higher levels. Besides, there is a significant difference between OL-MRFO-VC
and the rest of the algorithms in computation time.
Table 3 Wilcoxon's rank test between OL-MRFO-VC and competitive algorithms in lower and higher threshold levels.
OL-MRFO-VC
vs.
MRFO
MVO
PSO
SCA
SSA
HSA
ALO
FPA
ABC
Std
Lower
1.3506E-03
2.2803E-07
3.5694E-08
3.5694E-08
1.4420E-05
3.5694E-08
2.2001E-03
3.5694E-08
3.5694E-08
Higher
1.7254E-03
3.5681E-08
3.5694E-08
3.5694E-08
3.5694E-08
3.5694E-08
5.3404E-07
3.5694E-08
3.5694E-08
PSNR
Lower
7.1934E-06
4.5223E-04
3.0370E-07
9.1804E-06
2.4357E-04
7.6072E-06
6.1501E-04
6.1413E-07
1.9531E-06
Higher
8.1001E-03
3.5694E-08
3.5694E-08
3.5694E-08
5.5418E-06
3.5694E-08
2.1872E-04
3.5694E-08
5.6264E-08
SSIM
Lower
7.4656E-06
1.4376E-04
3.8442E-08
2.9243E-07
1.6657E-07
4.6578E-06
2.2209E-07
2.1648E-07
2.1539E-07
Higher
2.4241E-03
1.5666E-06
3.5255E-08
3.5480E-08
6.6170E-07
4.9168E-07
7.8117E-06
3.5117E-08
1.5279E-07
FSIM
Lower
2.2821E-04
4.8315E-07
3.5681E-08
3.5694E-08
3.4254E-07
2.5383E-07
1.2665E-05
3.5656E-08
1.2727E-07
Higher
9.8610E-04
2.9270E-07
3.5656E-08
3.5681E-08
2.1626E-07
3.5669E-08
3.3041E-06
3.5669E-08
3.5694E-08
PRI
Lower
1.3003E-05
3.1989E-06
3.8496E-08
6.6921E-06
1.3767E-07
7.0065E-07
1.5855E-07
3.5138E-07
6.0516E-08
Higher
6.8239E-03
7.0470E-07
5.2178E-08
2.9288E-07
2.5428E-06
6.7037E-06
7.5936E-04
7.3108E-08
7.5556E-07
VoI
Lower
1.0835E-04
2.2122E-05
9.4917E-08
8.1761E-08
5.7501E-06
9.2918E-07
5.5213E-05
3.0370E-07
1.0652E-06
Higher
8.3621E-03
7.5578E-07
6.5389E-08
3.5694E-08
4.9809E-07
1.4758E-07
9.1756E-05
4.8379E-08
2.1218E-07
Time
Lower
0.5058
3.5669E-08
3.5694E-08
3.5681E-08
3.5669E-08
3.5656E-08
3.5681E-08
3.5694E-08
3.5669E-08
Higher
1.0752E-02
3.5694E-08
3.5681E-08
3.5694E-08
3.5694E-08
3.5694E-08
3.5694E-08
3.5669E-08
3.5681E-08
6.1.10 Analysis of convergence curves
The one-running results of the convergence curves covering all competitive MHs are presented in Table 4, in
which six scenarios are chosen randomly due to page limitations. It can also be seen that the proposed OL-MRFO-
VC has a faster convergence rate than the other competitors, especially at higher threshold levels.
Table 4 Convergence curves of all competitive algorithms in selected conditions.
(a) Convergence curves of Test 2 with K = 4
(b) Convergence curves of Test 4 with K = 8
(c) Convergence curves of Test 5 with K = 10
16
(d) Convergence curves of Test 7 with K = 15
(e) Convergence curves of Test 8 with K = 25
(f) Convergence curves of Test 10 with K = 30
6.1.11 Analysis of segmented images
The segmented images of Test 1, Test 6, and Test 9 are presented in Tables B.19-B.21. It is clear to find that
some lousy results, such as PSO with K=4 and 6 in Test 1, SCA with K=10 in Test 6, and FPA with K=4 in Test
9, which are consistent with the quantitative results of various metrics.
6.2 Cityscapes dataset
All values of metrics (2)-(8) in E1-E9 are presented in Tables C.1-C.7. Figures 19-25 visualize the data.
Figure 18 Legend of Figures 24-30.
Fig. 19 Std values in K=10, 20, 30 (E1-
E9).
Fig. 20 PSNR values in K=10, 20, 30 (E1-
E9).
Fig. 21 SSIM values in K=10, 20, 30 (E1-
E9).
Fig. 22 FSIM values in K=10, 20, 30 (E1-
E9).
Fig. 23 PRI values in K=10, 20, 30 (E1-
E9).
Fig. 24 VoI values in K=10, 20, 30 (E1-E9).
Fig. 25 Computation time values in K=10, 20, 30 (E1-E9).
17
6.2.1 Analysis of Std
Table C.1 presents the standard deviation of fitness values over 30 runs, regarding E1-E9. Figure 19 visualizes
the corresponding data. It can be seen that OL-MRFO-VC outperforms the other modern algorithms in terms of
stability. Additionally, MRFO performs as well (ranked second), indicating that the basic framework of this
optimizer is superior. In the future, focusing on a specific engineering problem, MRFO can be studied and
enhanced particularly due to its robust features.
6.2.2 Analysis of PSNR
Table C.2 presents the average values of PSNR over 30 runs, regarding E1-E9, and Figure 20 visualizes the data.
SMA (in dark blue) and HGS (in yellow) perform poorly compared with the other five algorithms. Besides, when
comparing OL-MRFO-VC and MRFO in detail, it is found that their difference is slight in some images, such as
E1, E5, and E7, but there is a significant gap in the other images, especially in E2, E6, and E9. Nevertheless, the
average values of all the threshold levels in nine benchmarks show an improvement based on two strategies.
6.2.3 Analysis of SSIM
Table C.3 presents the SSIM values of the competitive algorithms in E1-E9, and Figure 21 visualizes the data.
Again, SMA and HGS show their inferiority in the MLT, which also demonstrates the existence of the no-free-
lunch theory. In Figure 21, it is clear that OL-MRFO-VC is much better than MRFO, as there is a significant gap.
However, the performance of TSA, SOA, and RUN is quite close to that of the proposed technique.
6.2.4 Analysis of FSIM
Table C.4 presents the FSIM values of the competitive algorithms in E1-E9, and Figure 22 visualizes the data.
Different from the approximate performance among OL-MRFO-VC, TSA, SOA, and RUN in SSIM values, FSIM
values here of the proposed algorithm show an apparent gap. Particularly, the improvement based on the basic
MRFO is also significant, indicating the effectiveness of our two-strategy framework.
6.2.5 Analysis of PRI
Table C.5 presents the PRI values of the competitive algorithms in E1-E9, and Figure 23 visualizes the data. A
total of seven algorithms can be distinguished into three groups according to their performance: HGS and SMA
are the worst performers, and then MRFO and RUN belong to the medium group; finally, OL-MRFO-VC, SOA,
and TSA are in the best group. In particular, in E6 and E3, the proposed OL-MRFO-VC is significantly superior
to the others, especially at the high threshold level (K=30).
6.2.6 Analysis of VoI
Table C.6 presents the VoI values of the competitive algorithms in E1-E9, and Figure 24 visualizes the data.
Recall that a lower value of VoI indicates a better outcome. It is still clear that OL-MRDO-VC outperforms the
others mostly; however, SOA is quite competitive and reaches better segmentation results than the proposed
technique in E4 and E7. The average value among all cases here shows that OL-MRFO-VC is better than SOA.
6.2.7 Analysis of computation time
Table C.7 presents the computation time values of the competitive algorithms in E1-E9, and Figure 25 visualizes
the data. This metric is essential in evaluating the practicability due to expensive computational facilities. Overall,
all the algorithms are able to obtain (near-)optimal results in a few seconds. When focusing on the comparison of
MRFO and OL-MRFO-VC, the proposed method even significantly accelerates the computation, such as in E5.
6.2.8 Analysis of Wilcoxon rank test
Table 5 presents the Wilcoxon signed-rank test result regarding seven quantitative metrics, in which the proposed
OL-MRFO-VC is compared to the other six modern MHs. We observe an apparent difference (p<0.05) between
the proposed algorithm and the others in terms of Std, PSNR, SSIM, FSIM, PRI, VoI, and computation time.
18
Table 5 Wilcoxon's rank test between OL-MRFO-VC and modern algorithms.
OL-MRFO-VC
vs.
MRFO
SMA
HGS
RUN
TSA
SOA
Std
3.79E-06
3.79E-06
3.79E-06
3.79E-06
3.79E-06
3.79E-06
PSNR
2.98E-02
3.90E-06
2.85E-06
1.32E-05
1.76E-04
7.94E-05
SSIM
4.57E-05
4.58E-05
4.58E-05
4.20E-03
1.28E-04
9.10E-03
FSIM
4.58E-05
4.58E-05
4.58E-05
4.58E-05
4.58E-05
4.99E-04
PRI
4.58E-05
4.58E-05
4.58E-05
4.58E-05
2.46E-04
8.96E-02
VoI
2.56E-06
2.56E-06
2.56E-06
2.56E-06
1.95E-05
3.47E-01
Time
2.91E-05
1.90E-05
5.60E-06
1.02E-02
5.61E-06
5.60E-06
6.3 Intel image dataset
The values of metrics (3)-(8) in 50 images (R1–R50) are shown in Figures D1-D6. Their means are presented in
Table 6.
Table 6 Evaluation results of competitive algorithms in R1-R50
PSNR
SSIM
FSIM
PRI
VoI
Time
TSA
29.4408
0.9974
0.9573
0.8957
3.7047
0.1514
SSA
26.8623
0.9948
0.9322
0.8744
3.8857
0.1413
SOA
28.8762
0.9973
0.9542
0.8944
3.7159
0.1535
SMA
21.9747
0.9803
0.8583
0.7950
4.4447
0.3424
SCA
26.0324
0.9947
0.9267
0.8642
4.0615
0.1996
RUN
30.7954
0.9980
0.9672
0.9060
3.6080
0.4036
PSO
27.2289
0.9951
0.9334
0.8756
3.8757
0.2159
MVO
30.5070
0.9979
0.9654
0.9034
3.6407
0.2178
MFO
30.7099
0.9979
0.9668
0.9041
3.6195
0.1964
HAS
29.5890
0.9974
0.9586
0.8970
3.7063
0.0082
HGS
21.8467
0.9814
0.8567
0.7951
4.4567
0.1488
FPA
27.5362
0.9955
0.9378
0.8787
3.8377
0.1884
ALO
31.0308
0.9980
0.9679
0.9056
3.6110
1.8161
ABC
29.4589
0.9972
0.9557
0.8953
3.7205
0.3454
MRFO
30.6644
0.9978
0.9659
0.9021
3.6509
0.4119
Ours
31.3960
0.9981
0.9697
0.9085
3.5806
0.4025
From the experiments, we observe the superiority of the proposed OL-MRFO-VC in color image segmentation.
Except for the computation time, the values obtained from OL-MRFO-VC in PSNR, SSIM, FSIM, PRI, and VoI
are the best, among K = 10, 20, and 30. As for computation time, when comparing OL-MRFO-VC with MRFO
particularly, we find that the computational speed has slight improvement. Our experimental results demonstrate
the effectiveness of the vertical crossover strategy which focuses on maintaining computational speed. To
summarize, our experimental evidence suggests that OL-MRFO-VC is a robust global optimizer for MLT-based
color image segmentation.
7. Conclusions and future directions
In this paper, a novel multilevel thresholding method based on the enhanced MRFO algorithm is proposed, in
which MRFO is enhanced by two effective strategies to develop a new optimizer, termed OL-MRFO-VC. On the
one hand, the OL strategy is initially employed in the random phase to increase the diversity of the population,
and therefore, to improve the exploration performance by focusing on the global space. On the other hand, the
vertical crossover mechanism from the search behaviors in CSO is embedded into the original MRFO, operating
on different dimensions in each agent to prevent local stagnation, without significantly impairing the convergence
19
speed compared to typical ones. We conduct extensive experiments using the images from BSD 500, Cityscapes,
and Intel images datasets to validate the performance of the proposed approach on multilevel thresholding tasks.
In addition, the quantitative and qualitative evaluations from eleven measures, such as PSNR, SSIM, FSIM, PRI,
and VoI, are realized to compare and analyze the differences and similarities between the competitive algorithms.
According to these experimental results, we conclude that the proposed OL-MRFO-VC has advantages in terms
of accuracy, robustness, and convergence rate, which can be valuable and powerful in multilevel thresholding.
For future work, more powerful strategies can be combined with MRFO to enhance the performance due to the
original ascendancy. In addition, these well-performing algorithms can be applied to more applications, such as
optimizing artificial neural networks (ANNs), mega infrastructure projects, and complex industrial engineering
problems.
Acknowledgment
The research of the last author was partially supported by the 2019 Guangdong Special Support Talent Program
– Innovation and Entrepreneurship Leading Team (China) (2019BT02S593).
References:
[1] W.A. Hussein, S. Sahran, S.N.H.S. Abdullah, A fast scheme for multilevel thresholding based on a modified bees algorithm,
Knowl Based Syst. 101 (2016) 114–134. https://doi.org/10.1016/J.KNOSYS.2016.03.010.
[2] S. Agrawal, R. Panda, P. Choudhury, A. Abraham, Dominant color component and adaptive whale optimization algorithm for
multilevel thresholding of color images, Knowl Based Syst. 240 (2022) 108172.
https://doi.org/10.1016/J.KNOSYS.2022.108172.
[3] S. Aja-Fernández, A.H. Curiale, G. Vegas-Sánchez-Ferrero, A local fuzzy thresholding methodology for multiregion image
segmentation, Knowl Based Syst. 83 (2015) 1–12. https://doi.org/10.1016/J.KNOSYS.2015.02.029.
[4] S. Song, H. Jia, J. Ma, A chaotic Electromagnetic Field Optimization algorithm based on fuzzy entropy for multilevel
thresholding color image segmentation, Entropy. 21 (2019). https://doi.org/10.3390/e21040398.
[5] B. Lei, J. Fan, Infrared pedestrian segmentation algorithm based on the two-dimensional Kaniadakis entropy thresholding,
Knowl Based Syst. 225 (2021) 107089. https://doi.org/10.1016/J.KNOSYS.2021.107089.
[6] R. Bandyopadhyay, R. Kundu, D. Oliva, R. Sarkar, Segmentation of brain MRI using an altruistic Harris Hawks' Optimization
algorithm, Knowl Based Syst. 232 (2021) 107468. https://doi.org/10.1016/J.KNOSYS.2021.107468.
[7] S. Dey, I. Saha, S. Bhattacharyya, U. Maulik, Multi-level thresholding using quantum inspired meta-heuristics, Knowl Based
Syst. 67 (2014) 373–400. https://doi.org/10.1016/J.KNOSYS.2014.04.006.
[8] M. Abdel-Basset, R. Mohamed, S. Mirjalili, A novel Whale Optimization Algorithm integrated with Nelder–Mead simplex for
multi-objective optimization problems, Knowl Based Syst. 212 (2021) 106619.
https://doi.org/10.1016/J.KNOSYS.2020.106619.
[9] S. Chakraborty, S. Nama, A.K. Saha, An improved symbiotic organisms search algorithm for higher dimensional optimization
problems, Knowl Based Syst. 236 (2022) 107779. https://doi.org/10.1016/J.KNOSYS.2021.107779.
[10] R. Salgotra, U. Singh, S. Singh, N. Mittal, A hybridized multi-algorithm strategy for engineering optimization problems, Knowl
Based Syst. 217 (2021) 106790. https://doi.org/10.1016/J.KNOSYS.2021.106790.
[11] M. Jahangoshai Rezaee, M. Eshkevari, M. Saberi, O. Hussain, GBK-means clustering algorithm: An improvement to the K-
means algorithm based on the bargaining game, Knowl Based Syst. 213 (2021) 106672.
https://doi.org/10.1016/J.KNOSYS.2020.106672.
[12] E.H. Houssein, K. Hussain, L. Abualigah, M.A. Elaziz, W. Alomoush, G. Dhiman, Y. Djenouri, E. Cuevas, An improved
opposition-based marine predators algorithm for global optimization and multilevel thresholding image segmentation, Knowl
Based Syst. 229 (2021) 107348. https://doi.org/10.1016/J.KNOSYS.2021.107348.
[13] E.H. Houssein, B.E. din Helmy, D. Oliva, A.A. Elngar, H. Shaban, A novel Black Widow Optimization algorithm for multilevel
thresholding image segmentation, Expert Syst Appl. 167 (2021) 114159. https://doi.org/10.1016/J.ESWA.2020.114159.
[14] H. Permuter, J. Francos, I. Jermyn, A study of Gaussian mixture models of color and texture features for image classification
and segmentation, Pattern Recognit. 39 (2006) 695–706. https://doi.org/10.1016/J.PATCOG.2005.10.028.
[15] L. Esmaeili, S.J. Mousavirad, A. Shahidinejad, An efficient method to minimize cross-entropy for selecting multi-level threshold
values using an improved human mental search algorithm, Expert Syst Appl. 182 (2021) 115106.
https://doi.org/10.1016/J.ESWA.2021.115106.
20
[16] J. Carreira, C. Sminchisescu, CPMC: Automatic object segmentation using constrained parametric min-cuts, IEEE Trans Pattern
Anal Mach Intell. 34 (2012) 1312–1328. https://doi.org/10.1109/TPAMI.2011.231.
[17] N. Otsu, A threshold selection method from gray-level histograms, IEEE Transactions on Systems, Man, and Cybernetics. 9
(1979) 62–66.
[18] J.N. Kapur, P.K. Sahoo, A.K.C. Wong, A new method for gray-level picture thresholding using the entropy of the histogram,
Comput Vis Graph Image Process. 29 (1985) 273–285. https://doi.org/10.1016/0734-189X(85)90125-2.
[19] B. Kosko, Fuzzy entropy and conditioning, Inf Sci (N Y). 40 (1986) 165–174. https://doi.org/10.1016/0020-0255(86)90006-X.
[20] A.R. Plastino, A. Plastino, Stellar polytropes and Tsallis' entropy, Phys Lett A. 174 (1993) 384–386.
https://doi.org/10.1016/0375-9601(93)90195-6.
[21] Z. Xing, An improved emperor penguin optimization based multilevel thresholding for color image segmentation, Knowl Based
Syst. 194 (2020) 105570. https://doi.org/10.1016/J.KNOSYS.2020.105570.
[22] A. Banerjee, D. Singh, S. Sahana, I. Nath, Impacts of metaheuristic and swarm intelligence approach in optimization, in:
Cognitive Big Data Intelligence with a Metaheuristic Approach, Academic Press, 2022: pp. 71–99.
https://doi.org/10.1016/B978-0-323-85117-6.00008-X.
[23] A.E. Eiben, J. Smith, From evolutionary computation to the evolution of things, Nature 2015 521:7553. 521 (2015) 476–482.
https://doi.org/10.1038/nature14544.
[24] B.J. Ma, S. Liu, A.A. Heidari, Multi-strategy ensemble binary hunger games search for feature selection, Knowl Based Syst.
248 (2022) 108787. https://doi.org/10.1016/J.KNOSYS.2022.108787.
[25] J.L.J. Pereira, M.B. Francisco, C.A. Diniz, G. Antônio Oliver, S.S. Cunha, G.F. Gomes, Lichtenberg algorithm: A novel hybrid
physics-based meta-heuristic for global optimization, Expert Syst Appl. 170 (2021) 114522.
https://doi.org/10.1016/J.ESWA.2020.114522.
[26] B.J. Ma, Hybrid Adaptive Moth-Flame Optimizer and Opposition-Based Learning for Training Multilayer Perceptrons, (2022)
273–319. https://doi.org/10.1007/978-3-030-99079-4_11.
[27] B.J. Ma, Correction to: Hybrid Adaptive Moth-Flame Optimizer and Opposition-Based Learning for Training Multilayer
Perceptrons, (2022) C1–C4. https://doi.org/10.1007/978-3-030-99079-4_20.
[28] D.H. Wolpert, W.G. Macready, No free lunch theorems for optimization, IEEE Transactions on Evolutionary Computation. 1
(1997) 67–82. https://doi.org/10.1109/4235.585893.
[29] E.H. Houssein, M.M. Emam, A.A. Ali, Improved manta ray foraging optimization for multi-level thresholding using COVID-
19 CT images, Neural Comput Appl. 33 (2021) 16899–16919. https://doi.org/10.1007/S00521-021-06273-3/TABLES/10.
[30] B. Jena, M.K. Naik, R. Panda, A. Abraham, Maximum 3D Tsallis entropy based multilevel thresholding of brain MR image
using attacking Manta Ray foraging optimization, Eng Appl Artif Intell. 103 (2021) 104293.
https://doi.org/10.1016/J.ENGAPPAI.2021.104293.
[31] M. Abd Elaziz, D. Yousri, M.A.A. Al-qaness, A.M. AbdelAty, A.G. Radwan, A.A. Ewees, A Grunwald–Letnikov based Manta
ray foraging optimizer for global optimization and image segmentation, Eng Appl Artif Intell. 98 (2021) 104105.
https://doi.org/10.1016/j.engappai.2020.104105.
[32] S. Sharma, A.K. Saha, A. Majumder, S. Nama, MPBOA - A novel hybrid butterfly optimization algorithm with symbiosis
organisms search for global optimization and image segmentation, Multimed Tools Appl. 80 (2021) 12035–12076.
https://doi.org/10.1007/s11042-020-10053-x.
[33] A. Wunnava, M.K. Naik, R. Panda, B. Jena, A. Abraham, An adaptive Harris hawks optimization technique for two dimensional
grey gradient based multilevel image thresholding, Appl Soft Comput. 95 (2020) 106526.
https://doi.org/10.1016/j.asoc.2020.106526.
[34] L. Ren, A.A. Heidari, Z. Cai, Q. Shao, G. Liang, H.L. Chen, Z. Pan, Gaussian kernel probability-driven slime mould algorithm
with new movement mechanism for multi-level image segmentation, Measurement. 192 (2022) 110884.
https://doi.org/10.1016/J.MEASUREMENT.2022.110884.
[35] S. Singh, N. Mittal, H. Singh, A multilevel thresholding algorithm using LebTLBO for image segmentation, Neural Comput
Appl. 32 (2020) 16681–16706. https://doi.org/10.1007/S00521-020-04989-2/TABLES/9.
[36] S. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, D. Terzopoulos, Image Segmentation Using Deep Learning: A
Survey, IEEE Trans Pattern Anal Mach Intell. 44 (2022) 3523–3542. https://doi.org/10.1109/TPAMI.2021.3059968.
[37] A. Ramesh Kumar, H. Kuttiappan, Detection of brain tumor size using modified deep learning and multilevel thresholding
utilizing modified dragonfly optimization algorithm, Concurr Comput. 34 (2022) e7016. https://doi.org/10.1002/CPE.7016.
[38] W. Zhao, Z. Zhang, L. Wang, Manta ray foraging optimization: An effective bio-inspired optimizer for engineering applications,
Eng Appl Artif Intell. 87 (2020). https://doi.org/10.1016/j.engappai.2019.103300.
[39] M.H. Hassan, E.H. Houssein, M.A. Mahdy, S. Kamel, An improved Manta ray foraging optimizer for cost-effective emission
dispatch problems, Eng Appl Artif Intell. 100 (2021) 104155. https://doi.org/10.1016/J.ENGAPPAI.2021.104155.
[40] K.K. Ghosh, R. Guha, S.K. Bera, N. Kumar, R. Sarkar, S-shaped versus V-shaped transfer functions for binary Manta ray
foraging optimization in feature selection problem, Neural Comput Appl. (2021). https://doi.org/10.1007/s00521-020-05560-9.
21
[41] A. Fathy, H. Rezk, D. Yousri, A robust global MPPT to mitigate partial shading of triple-junction solar cell-based system using
manta ray foraging optimization algorithm, Solar Energy. 207 (2020) 305–316.
https://doi.org/10.1016/J.SOLENER.2020.06.108.
[42] A. Tang, H. Zhou, T. Han, L. Xie, A modified manta ray foraging optimization for global optimization problems, IEEE Access.
9 (2021) 128702–128721. https://doi.org/10.1109/ACCESS.2021.3113323.
[43] J. Feng, X. Luo, M. Gao, A. Abbas, Y.P. Xu, S. Pouramini, Minimization of energy consumption by building shape optimization
using an improved Manta-Ray Foraging Optimization algorithm, Energy Reports. 7 (2021) 1068–1078.
https://doi.org/10.1016/J.EGYR.2021.02.028.
[44] B. Morales-Castañeda, D. Zaldívar, E. Cuevas, F. Fausto, A. Rodríguez, A better balance in metaheuristic algorithms: Does it
exist?, Swarm Evol Comput. 54 (2020) 100671. https://doi.org/10.1016/J.SWEVO.2020.100671.
[45] H.R. Tizhoosh, Opposition-based learning: A new scheme for machine intelligence, Proceedings - International Conference on
Computational Intelligence for Modelling, Control and Automation, CIMCA 2005 and International Conference on Intelligent
Agents, Web Technologies and Internet. 1 (2005) 695–701. https://doi.org/10.1109/CIMCA.2005.1631345.
[46] A.B. Meng, Y.C. Chen, H. Yin, S.Z. Chen, Crisscross optimization algorithm and its application, Knowl Based Syst. 67 (2014)
218–229. https://doi.org/10.1016/J.KNOSYS.2014.05.004.
[47] J.G. Digalakis, K.G. Margaritis, On benchmarking functions for genetic algorithms, Int J Comput Math. 77 (2007) 481–506.
https://doi.org/10.1080/00207160108805080.
[48] S. Mirjalili, A. Lewis, The Whale Optimization Algorithm, Advances in Engineering Software. 95 (2016) 51–67.
https://doi.org/10.1016/J.ADVENGSOFT.2016.01.008.
[49] Y. Liu, G. Chong, A.A. Heidari, H. Chen, G. Liang, X. Ye, Z. Cai, M. Wang, Horizontal and vertical crossover of Harris hawk
optimizer with Nelder-Mead simplex for parameter estimation of photovoltaic models, Energy Convers Manag. 223 (2020)
113211. https://doi.org/10.1016/j.enconman.2020.113211.
[50] H. Jia, J. Ma, W. Song, Multilevel Thresholding Segmentation for Color Image Using Modified Moth-Flame Optimization,
IEEE Access. 7 (2019) 44097–44134. https://doi.org/10.1109/ACCESS.2019.2908718.
[51] Q. Huynh-Thu, M. Ghanbari, Scope of validity of PSNR in image/video quality assessment, Electron Lett. 44 (2008) 800–801.
https://doi.org/10.1049/EL:20080522.
[52] Z. Wang, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli, Image quality assessment: From error visibility to structural similarity,
IEEE Transactions on Image Processing. 13 (2004) 600–612. https://doi.org/10.1109/TIP.2003.819861.
[53] L. Zhang, L. Zhang, X. Mou, D. Zhang, FSIM: A feature similarity index for image quality assessment, IEEE Transactions on
Image Processing. 20 (2011) 2378–2386. https://doi.org/10.1109/TIP.2011.2109730.
[54] A.Y. Yang, J. Wright, Y. Ma, S.S. Sastry, Unsupervised segmentation of natural images via lossy data compression, Computer
Vision and Image Understanding. 110 (2008) 212–225. https://doi.org/10.1016/j.cviu.2007.07.005.
[55] Y. Ma, H. Derksen, W. Hong, J. Wright, Segmentation of multivariate mixed data via lossy data coding and compression, IEEE
Trans Pattern Anal Mach Intell. 29 (2007) 1546–1562. https://doi.org/10.1109/TPAMI.2007.1085.
[56] D. Martin, C. Fowlkes, D. Tal, J. Malik, A database of human segmented natural images and its application to evaluating
segmentation algorithms and measuring ecological statistics, in: Proceedings of the IEEE International Conference on Computer
Vision, 2001: pp. 416–423. https://doi.org/10.1109/ICCV.2001.937655.
[57] S. Mirjalili, SCA: A Sine Cosine Algorithm for solving optimization problems, Knowl Based Syst. 96 (2016) 120–133.
https://doi.org/10.1016/J.KNOSYS.2015.12.022.
[58] D. Karaboga, B. Basturk, A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC)
algorithm, Journal of Global Optimization 2007 39:3. 39 (2007) 459–471. https://doi.org/10.1007/S10898-007-9149-X.
[59] J. Kennedy, R. Eberhart, Particle swarm optimization, in: Proceedings of ICNN'95 - International Conference on Neural
Networks, IEEE, 1995: pp. 1942–1948. https://doi.org/10.1109/ICNN.1995.488968.
[60] S. Mirjalili, A.H. Gandomi, S.Z. Mirjalili, S. Saremi, H. Faris, S.M. Mirjalili, Salp Swarm Algorithm: A bio-inspired optimizer
for engineering design problems, Advances in Engineering Software. 114 (2017) 163–191.
https://doi.org/10.1016/J.ADVENGSOFT.2017.07.002.
[61] X.S. Yang, M. Karamanoglu, X. He, Flower pollination algorithm: A novel approach for multiobjective optimization,
Engineering Optimization. 46 (2014) 1222–1237. https://doi.org/10.1080/0305215X.2013.832237.
[62] S. Mirjalili, S.M. Mirjalili, A. Hatamlou, Multi-Verse Optimizer: a nature-inspired algorithm for global optimization, Neural
Comput Appl. 27 (2016) 495–513. https://doi.org/10.1007/S00521-015-1870-7/TABLES/12.
[63] Z.W. Geem, J.H. Kim, G. V. Loganathan, A New Heuristic Optimization Algorithm: Harmony Search, Simulation. 76 (2016)
60–68. https://doi.org/10.1177/003754970107600201.
[64] S. Mirjalili, The ant lion optimizer, Advances in Engineering Software. 83 (2015) 80–98.
https://doi.org/10.1016/j.advengsoft.2015.01.010.
[65] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, B. Schiele, The Cityscapes Dataset
for Semantic Urban Scene Understanding, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
2016: pp. 3213–3223.
22
[66] R. Mohan, A. Valada, EfficientPS: Efficient Panoptic Segmentation, Int J Comput Vis. 129 (2021) 1551–1579.
https://doi.org/10.1007/S11263-021-01445-Z/FIGURES/7.
[67] S. Li, H. Chen, M. Wang, A.A. Heidari, S. Mirjalili, Slime mould algorithm: A new method for stochastic optimization, Future
Generation Computer Systems. 111 (2020) 300–323. https://doi.org/10.1016/j.future.2020.03.055.
[68] Y. Yang, H. Chen, A.A. Heidari, A.H. Gandomi, Hunger games search: Visions, conception, implementation, deep analysis,
perspectives, and towards performance shifts, Expert Syst Appl. 177 (2021). https://doi.org/10.1016/j.eswa.2021.114864.
[69] I. Ahmadianfar, A.A. Heidari, A.H. Gandomi, X. Chu, H. Chen, RUN beyond the metaphor: An efficient optimization algorithm
based on Runge Kutta method, Expert Syst Appl. 181 (2021) 115079. https://doi.org/10.1016/J.ESWA.2021.115079.
[70] G. Dhiman, V. Kumar, Seagull optimization algorithm: Theory and its applications for large-scale industrial engineering
problems, Knowl Based Syst. 165 (2019) 169–196. https://doi.org/10.1016/j.knosys.2018.11.024.
[71] S. Kaur, L.K. Awasthi, A.L. Sangal, G. Dhiman, Tunicate Swarm Algorithm: A new bio-inspired based metaheuristic paradigm
for global optimization, Eng Appl Artif Intell. 90 (2020). https://doi.org/10.1016/j.engappai.2020.103541.
[72] Intel Image Classification | Kaggle, (n.d.). https://www.kaggle.com/datasets/puneet6060/intel-image-classification (accessed
December 2, 2022).
