No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Application of fuzzy expert system in test case selection for system regression test
Abstract
Effective system test for consecutive releases of modern telecommunications system relies heavily on the selection of the test cases, because running all test cases in an existing test suite will consume an inordinate amount of time and money. Although many test selection techniques have been proposed on how to select test cases so that the number of test cases does not grow too large as the software evolves, to our knowledge they are all based on the assumption that the analysis of the source code is available. Very few studies have been made to explore the possibility of selecting test cases when the information of the source code is not available to testers. We present using fuzzy expert systems to solve the problem. The fuzzy expert system will resemble a human judgment via correlating knowledge that represents one or more of the following: customer profile, analysis of past test results, system failure rate, and wherever possible change in system architecture, to identify potentially critical test cases for system test. We piloted this fuzzy expert system in large telecommunication systems and the results showed that test effectiveness and efficiency was significantly improved.
... Optimization of test suites can be achieved by test case prioritization or reduction. Some of the popular techniques reported for regression test suite optimization use genetic algorithms [24,46], non-dominating sorting genetic algorithm [43], swarm algorithm [16], neural networks [38] and fuzzy logic [3,45]. ...
... Therefore, researchers have used optimization techniques for both white box-and black boxbased testing methods. We identified various optimization objectives from the literature[9,30,32,41,45,46] which are enlisted below:Optimization Objectives for White Box Testing References:[18,25,41,45] -Increase defect detection rate -Implementation complexity -Cost-effectiveness -Coverage (statement, code and function) -Increased defect detection capability -Faults detection rate -Test execution time -Decision trees -Changes in class diagram -CostOptimization Objectives for Black Box Testing References:[9,30,32,41,46] -Requirement change -Behavior of test case in previous release -Changes in architecture; configuration files; database ...
... Therefore, researchers have used optimization techniques for both white box-and black boxbased testing methods. We identified various optimization objectives from the literature[9,30,32,41,45,46] which are enlisted below:Optimization Objectives for White Box Testing References:[18,25,41,45] -Increase defect detection rate -Implementation complexity -Cost-effectiveness -Coverage (statement, code and function) -Increased defect detection capability -Faults detection rate -Test execution time -Decision trees -Changes in class diagram -CostOptimization Objectives for Black Box Testing References:[9,30,32,41,46] -Requirement change -Behavior of test case in previous release -Changes in architecture; configuration files; database ...
Regression testing is a mandatory activity of software development life cycle, which is performed to ensure that modifications have not caused any adverse effects on the system’s functionality. With every change in software in the maintenance phase, the size of regression test suite grows as new test cases are written to validate changes. The bigger size of regression test suite makes the testing expensive and time-consuming. Optimization of regression test suite is a possible solution to cope with this problem. Various techniques of optimization have been proposed; however, there is no perfect solution for the problem and therefore, requires better solutions to improve the optimization process. This paper presents a novel technique named as hybrid-adaptive neuro-fuzzy inference system tuned with genetic algorithm and particle swarm optimization algorithm that is used to optimize the regression test suites. Evaluation of the proposed approach is performed on benchmark test suites including “previous date problem” and “Siemens print token.” Experimental results are compared with existing state-of-the-art techniques, and results show that the proposed approach is more effective for the reduction in a regression test suites with higher requirement coverage. The size of regression test suites can be reduced up to 48% using the proposed approach without reducing the fault detection rate.
... Their approach was compared with random selection, and the proposed model is 34% better in precision and 20% better in recall [14]. Xu et al. [53] used the fuzzy expert system for black box-based testing methods to select some test cases. Schedule factor, defect impact factor, and coverage factor were selected as objective functions. ...
... In this research, we optimize the regression test suites for the black box testing method. Input variables for the ANFIS network are selected from the content analysis of papers [53,54,9,21,38,40,42,48]. The following input variables are used in our research because these are important factors for regression testing. ...
... The fuzzy logic, MOGA, NSGA-II, and MOPSO are already implemented for regression test suite optimization in various studies. The fuzzy logic is used for the regression test suite optimization in [3,6,20,53]. We used the implementation of [6] to validate our approach. ...
Regression testing is a type of testing activity, which ensures that source code changes do not affect the unmodified portions of the software adversely. This testing activity may be very expensive in, some cases, due to the required time to execute the test suite. In order to execute the regression tests in a cost-effective manner, the optimization of regression test suite is crucial. This optimization can be achieved by applying test suite reduction (TSR), regression test selection (RTS), or test case prioritization (TCP) techniques. In this paper, we designed and implemented an expert system for TSR problem by using neuro-fuzzy modeling-based approaches known as “adaptive neuro-fuzzy inference system with grid partitioning” (ANFIS-GP) and “adaptive neuro-fuzzy inference system with subtractive clustering” (ANFIS-SC). Two case studies were performed to validate the model and fuzzy logic, multi-objective genetic algorithms (MOGAs), non-dominated sorting genetic algorithm (NSGA-II) and multi-objective particle swarm optimization (MOPSO) algorithms were used for benchmarking. The performance of the models were evaluated in terms of reduction of test suite size, reduction in fault detection rate, reduction in test suite execution time, and reduction in requirement coverage. The experimental results showed that our ANFIS-based optimization system is very effective to optimize the regression test suite and provides better performance than the other approaches evaluated in this study. Size and execution time of the test suite is reduced up to 50%, whereas loss in fault detection rate is between 0% and 25%.
... But, performing a comparison with an already defined fitness function value is not suitable in every case. For overcoming this issue and the introduction of expert judgment in test case selection, fuzzy logic is used for optimizing the test suite for regression testing [17,18]. Fuzzy set theory has been proposed by Zadeh [19] with intention to generalize the classical notion of a set. ...
... Xu et al. [18] employed a Fuzzy Expert System for selecting test cases for regression testing. C language is used for creating a fuzzy expert system and data is gathered from a GSM project and 9768 test cases are used for optimization. ...
In the process of software development, regression testing is one of the major activities that is done after making modifications in the current system or whenever a software system evolves. But, the test suite size increases with the addition of new test cases and it becomes inefficient because of the occurrence of redundant, broken, and obsolete test cases. For that reason, it results in additional time and budget to run all these test cases. Many researchers have proposed computational intelligence and conventional approaches for dealing with this problem and they have achieved an optimized test suite by selecting, minimizing or reducing, and prioritizing test cases. Currently, most of these optimization approaches are single objective and static in nature. But, it is mandatory to use multi-objective dynamic approaches for optimization due to the advancements in information technology and associated market challenges. Therefore, we have proposed three variants of self-tunable Adaptive Neuro-fuzzy Inference System i.e. TLBO-ANFIS, FA-ANFIS, and HS-ANFIS, for multi-objective regression test suites optimization. Two benchmark test suites are used for evaluating the proposed ANFIS variants. The performance of proposed ANFIS variants is measured using Standard Deviation and Root Mean Square Error. A comparison of experimental results is also done with six existing methods i.e. GA-ANFIS, PSO-ANFIS, MOGA, NSGA-II, MOPSO, and TOPSIS and it is concluded that the proposed method effectively reduces the size of regression test suite without a reduction in the fault detection rate.
... Regression testing using fuzzification has been conceptualized by a few researchers as per the literature review. Fuzzy expert system is selected as the decisions made by it are better than the normal expert system [3]. The work improves the existing fuzzy logic based technique and removes the faults in it. ...
... In the work carried out by Zhewei Xu, Kehan Gao and Taghi M Khoshgoftaar, a fuzzy expert system has been proposed in order to select the test cases when information of the source code is not available to testers [3]. A study has been done on the sample data from the GSM system test database. ...
Regression testing is an immensely important process in the maintenance phase. The prioritization of test case becomes all the more important owing to the fact that it is not feasible to run all the test cases after each and every change. The proposed work dwells on the power of fuzzy expert system to make decisions which are better than the normal expert systems. The technique uses concepts like defect impact and coverage for test case selection. The work intends to propose a new framework for a fuzzy expert system based regression testing technique. The proposed architecture enhances the existing fuzzy logic based technique and removes faults therein. The proposed system architecture has been designed and initial results are encouraging. The system is poised to prove pivotal in the discipline.
... [4][5][6][7] Fuzzy systems are suitable for decision making and expert judgment. First implementation of fuzzy logic in testing domain was an expert system for test suite selection proposed by Xu et al. (2005) [8]. This system required intervention of an expert to select the suitable test cases. ...
... [4][5][6][7] Fuzzy systems are suitable for decision making and expert judgment. First implementation of fuzzy logic in testing domain was an expert system for test suite selection proposed by Xu et al. (2005) [8]. This system required intervention of an expert to select the suitable test cases. ...
Regression testing is an important activity performed to ensure that
changes in the baseline version of the system do not influence the
already tested part of the system. It becomes difficult to run the entire
test suite due to constrained or limited resources. A subset of test cases
that is as efficient as the original test suite is searched as optimal suite.
Computational intelligence approaches has been used to search the
representative subset. The major concern is safe reduction of test suite.
Safe reduction has been achieved by fuzzy optimization. Neural
networks are known for their ability to learn and fuzzy based systems
for their quality to judge and make decisions. Neuro-fuzzy systems can
be used to learn and make expert judgment. We have implemented
Adaptive Neuro-Fuzzy Inference System (ANFIS) for test suite
optimization. The resultant suite was found to be better than fuzzy
based optimization in reducing the time and improving the coverage of
resulting test suite. We concluded that ANFIS can be used to automate
the optimization process. We will implement it on sufficiently large
sized test suite available in software infrastructure repository provided
by Siemens to validate our findings.
... Fuzzy expert systems have been proposed for optimization of test suite [29][30][31][32]. Xu et al. [29] proposed an expert system for test case selection in regression testing. ...
... Fuzzy expert systems have been proposed for optimization of test suite [29][30][31][32]. Xu et al. [29] proposed an expert system for test case selection in regression testing. They generated casual model using different parameters and then developed rules from this casual model. ...
Systems are frequently regression tested for
frequently occurring changes due to corrective, preventive,
adaptive or perfective actions. Regression testing is used to
prevent the undesired effect of these changes on the previously
tested version. Due to these changes, new test cases become part
of the test suite making it huge and inefficient for ‘retest all’
strategy. The ultimate solution of this problem is optimization or
reduction of the test suite. Computational intelligence (CI) based
approaches like evolutionary computation, fuzzy logic, neural
networks and swarm optimization has been used for test suite
reduction. Optimization approaches reduce the test suite by
compromising its safety. Ideally optimization of test suite must
guarantee safe reduction. In this work, we have optimized the
test suite using some CI based approaches and then analysed the
test suite for ‘safe reduction’. Safe reduction can be gauged using
control flow graphs. Test cases of optimal solutions were
traversed on these graphs. We found that these solutions
partially cover control flow graph. This showed that optimal
solutions returned by CI based approaches except fuzzy logic are
not safe and will be inadequate for regression testing.
... Previous research on using fuzzy logic for the purpose of test case prioritization is scant. In [30], a fuzzy expert system is reported in which the system is used during regression testing of a telecommunication application. To build the required knowledge base for the expert system reported in this research, the researchers had to acquire knowledge from different sources, such as customer profiles, past test results, system failure rates, and the history of system architecture changes. ...
... Although this expert system has shown promising results with respect to the specific application that it was designed for, it is necessary to acquire a new knowledge base for new applications. The proposed test case selection model in [30] treats the software under test as a black box; therefore, it cannot be used for the purpose of regression testing programs with assertions. ...
Program assertions have been recognized as a supporting tool during software development, testing, and maintenance. Therefore, software developers place assertions within their code in positions that are considered to be error prone or that have the potential to lead to a software crash or failure. Similar to any other software, programs with assertions must be maintained. Depending on the type of modification applied to the modified program, assertions also might have to undergo some modifications. New assertions may also be introduced in the new version of the program, while some assertions can be kept the same. This paper presents a novel approach for test case prioritization during regression testing of programs that have assertions using fuzzy logic. The main objective of this approach is to prioritize the test cases according to their estimated potential in violating a given program assertion. To develop the proposed approach, we utilize fuzzy logic techniques to estimate the effectiveness of a given test case in violating an assertion based on the history of the test cases in previous testing operations. We have conducted a case study in which the proposed approach is applied to various programs, and the results are promising compared to untreated and randomly ordered test cases.
... Fuzzy based expert systems have also been used for test suite optimization and incorporate knowledge base and user interaction to reduce the test suite. Xu et al. (2005) [27] proposed an expert system to select test case in reduced regression testing. This system included inference engine, knowledge base and user interface. ...
... Fuzzy based expert systems have also been used for test suite optimization and incorporate knowledge base and user interaction to reduce the test suite. Xu et al. (2005) [27] proposed an expert system to select test case in reduced regression testing. This system included inference engine, knowledge base and user interface. ...
Test suite optimization is mandatory activity to cope with the limited resources and economical constraints. Aside from conventional optimization approaches, computational intelligence based approaches have also focused this research area and evolutionary algorithms have been successful to remarkably reduce the test suite size. These approaches classify the decision space by generating an optimal front called Pareto front. Despite the tremendous research work on interpretation of Pareto front, its visualization and usefulness for higher objectives is a challenging task. Pareto fronts contain multiple suitable solutions that can be converged using heuristics to a reduced set of possible solutions. We have proposed fuzzy based optimization approach in combination with all path coverage criterion to safely reduce a test suite to a single solution. We have initially implemented our proposed work on a testing problem having three objectives; however, it is scalable to any finite number of optimization objectives. We validated our approach by comparing it with evolutionary algorithms. We found that our approach significantly reduces the test suite to a precise test suite. We have concluded that our approach is capable to be automated and provide ‘on the fly’ optimal solution.
... The earlier attempts of regression testing were studied [8][9][10][11][12][13]. The work by Zhiwei Xu [2] proposed a fuzzy expert system which makes use of classes, design specifications and rules and stores the rule firing history and infer the explanations. The inference engine in turn has an agenda and procedures to derive conclusions. ...
Regression testing calls for selection of appropriate test cases after modification have been made in the software. The task becomes difficult when the source code is not available. The work presents the use of fuzzy logic to develop an expert system, capable of selecting the test cases retaining the effectiveness and efficiency while at the same time reducing the number of test cases. The work is a continuation of an earlier work and the results obtained are encouraging.
... In the very next year Adi Srikanth et.al had introduced a software test case optimization method using honey bees[34].Researches for the purpose of regression testing using fuzzy logic are very scant. It started in the year 2005 when Xu, Gao et.al had firstly shown the application of fuzzy for regression testing[35]. Recently in 2012 and 2013 Ali M. Alakeel and H.B. Gupta proposed a technique for the regression testing using fuzzy logic respectively[36] [37]. ...
Maintenance of the software is concerned with the changes and modifications implemented to the
software. It needs to be validated that the modifications has not led to the degradation in the quality of the
software. Software regression testing is required to instill confidence that changes are valid. Regression testing
is very expensive. It requires the optimization of regression test cases. Recently, evolutionary and metaheuristic
algorithms are coming into existence as search and optimization tools. This paper proposes a
regression test case optimization technique based on honey bee mating optimization and fuzzy rule base, which
reduces the size of the test suite by selecting the test cases from the existing test suite. The test cases which
are necessary for validating the recent changes in the software and have the ability to find the faults and cover
maximum coding under testing in minimum time are selected. An algorithm is designed which takes as input
the test suite containing several test cases and based on their execution time and fault coverage, reduces the
number of test cases. The test suite is reduced by 50% which optimizes the overall regression testing process.
The proposed algorithm reduces the test suite data by approximately 50%. Although the percentage reduction
is found little bit low in comparison to existing algorithm but the overall algorithm is found efficient as it
provides an authentic way of selecting test cases using fuzzy rule base.
... Safe regression test selection techniques have also been widely investigated in the literature [70,71]. To solve the test selection problem, other innovative techniques have also been proposed, such as the use of fuzzylogic [72], or by using semantic differences and similarities between old and new programs to avoid test cases that will produce the same results [73]. ...
Context: In many domains, engineers build simulation models (e.g., Simulink) before developing code to simulate the behavior of complex systems (e.g., Cyber-Physical Systems). Those models are commonly heavy to simulate which makes it difficult to execute the entire test suite. Furthermore, it is often difficult to measure white-box coverage of test cases when employing such models. In addition, the historical data related to failures might not be available.
Objective: The objective of the approach presented in this paper is to cost-effectively select test cases without making use of white-box coverage information or historical data related to fault detection.
Method: We propose a cost-effective approach for test case selection that relies on black-box data related to inputs and outputs of the system. The approach defines in total six effectiveness measures and one cost measure followed by deriving in total 21 objective combinations and integrating them within Non-Dominated Sorting Genetic Algorithm-II (NSGA-II). The proposed six effectiveness metrics are specific to simulation models and are based on anti-patterns and similarity measures.
Results: We empirically evaluated our approach with these 21 combinations using six case studies by employing mutation testing to assess the fault revealing capability. We compared our approach with Random Search (RS), two many-objective algorithm, as well as three white-box metrics. The results demonstrated that our approach managed to improve Random Search by up to around 28% in terms of the Hypervolume quality indicator. Similarly, black-box metrics-based test case selection also significantly outperformed those of white-box metrics.
Conclusion: We demonstrate that test case selection is a non-trivial problem in the context of simulation models. We also show that the proposed effectiveness metrics performed significantly better than traditional white-box metrics. Thus, we show that black-box test selection approaches are appropriate to solve the test case selection problem within simulation models.
... Xu et al [13] presented a fuzzy expert system to resemble the human judgment, correlating knowledge that represents the customer profile, analysis of past test results, system failure rate, and the possible changes in system architecture, to identify potentially critical test cases for system test. ...
Software testing consists in the dynamic verification of the behavior of a program on a set of test cases. When a program is modified, it must be tested to verify if the changes did not imply undesirable effects on its functionality. The rerunning of all test cases can be impossible, due to cost, time and resource constraints. So, it is required the creation of a test cases subset before the test execution. This is a hard problem and the use of standard Software Engineering techniques could not be suitable. This work presents an approach for test case prioritization and selection, based in relevant inputs obtained from a software development environment. The approach uses Software Quality Function Deployment (SQFD) to deploy the features relevance among the system components, Mamdani fuzzy inference systems to infer the criticality of each class and Ant Colony Optimization to select test cases. An evaluation of the approach is presented, using data from simulations with different number of tests.
... Auf der Ebene des Systemtests ist dies allerdings ungleich schwieriger, aufgrund des wesentlich größeren und komplexeren Testobjekts sowie Anforderungen an die Testdurchführung, für die sich kein direkter Code-Bezug mehr herstellen lässt. Die Regeln zur Testplanerstellung beruhen zum großen Teil auf ungenauen und unscharfen Aussagen, die oft nur in verbaler Form vorliegen [12]. Bei der Berücksichtigung des Faktors Mensch stoßen die bekannten mathematischen Algorithmen ebenfalls schnell an Grenzen. ...
Automatisierte Systeme unterliegen hohen Qualitatsanforderungen. Den entscheidenden Einfluss auf die Qualitat hat dabei der Software-Systemtest. In diesem Beitrag wird ein Testmanagementsystem vorgestellt, das auf agentenbasierter Softwareentwicklung und Fuzzy-Logik basiert. Dieses System kann fehleranfallige Stellen identifizieren und die dazu passenden Testfalle so priorisieren, dass fruhzeitig viele Fehler gefunden werden. Die authoren zeigen eine Moglichkeit auf, die Testfallpriorisierung durchzufuhren, bei der einerseits die Vielzahl an relevanten Eigenschaften und Abhangigkeiten berucksichtigt wird und andererseits unscharfes Wissen modelliert werden kann. Hierzu hat das Institut fur Automatisierungs- und Softwaretechnik (IAS) das adaptive Testmanagementsystem (ATMS) entwickelt, das mithilfe von Softwareagenten relevante Informationen fur das Testmanagement sammelt, verarbeitet und auf Grundlage von Fuzzy-Logik eine Reihenfolge zur Ausfuhrung von Testfallen bestimmt.
... Researches using fuzzy logic for the purpose of regression test case selection and prioritization are very scant. Xu, Gao and Khoshgoftaar had firstly shown the application of fuzzy expert system in regression test selection in 2005 [34]. Later in 2011 Praveen, Sirish and Raghurama of BITS pilani had given fuzzy criteria for assessing the software testing effort [35]. ...
Software maintenance is performed regularly for enhancing and adapting the functionalities of the existing software, which modifies the software and breaks the previously verified functionalities. This sets a requirement for software regression testing, making it a necessary maintenance activity. As the evolution of software takes place the size of the test suite tends to grow, which makes it difficult to execute the entire test suite in a time constrained environment. There are many existing techniques for regression test case selection. Some are based on dataflow analysis technique, slicing-based technique, bio-inspired techniques, and genetic algorithm based techniques. This paper gives a regression test case selection technique based on fuzzy model, which reduces the size of the test suite by selecting test cases from existing test suite. The test cases, which are necessary for validating the recent changes in the software and have the ability to find the faults and cover maximum coding under testing in minimum time, are selected. A fuzzy model is designed which takes three parameters namely code covered, execution time and faults covered as input and produces the estimation for the test case selection probability as very low, low, medium, high and very high.
... Fuzzy expert systems have been proposed for optimization of test suite [28][29][30]. Significance of using fuzzy logic in test suite optimization problem has been outlined by Kumar et al. [31][32]. Fuzzy clustering approach used to classify the similar test cases for test suite selection has been proposed by Kumar et al. [33]. ...
Systems are frequently regression tested during the maintenance phase due to corrective, preventive, adaptive or perfective actions. Regression testing is used to prevent the undesirable effects of these changes on the previously tested version. Due to these changes, new test cases become part of the test suite making it huge and inefficient for 'retest all' strategy. The ultimate solution of this problem is optimization or reduction of the test suite. Computational Intelligence (CI) based approaches like evolutionary computation, fuzzy logic, neural networks and swarm optimization have been used for test suite reduction. Optimization approaches reduce the test suite by compromising its safety. Ideally optimization of test suite must guarantee safe reduction. In this work, we have optimized the test suite using some CI based approaches and then analysed the test suite for 'safe reduction'. Safe reduction can be gauged using control flow graphs. Test cases of optimal solutions were traversed on these graphs. We found that these solutions partially cover control flow graph. This showed that optimal solutions returned by CI based approaches except fuzzy logic are not safe and will be inadequate for regression testing.
... The tree representation of a knowledge base is not suitable for reasoning multiple conclusions because the entire tree needs to be searched based on a directed tree graph. The matrix representation of a knowledge base requires large memory space for storing knowledge so that the knowledge search speed is slow (Xu et al., 2005). Therefore, the IBCMC system adopts the relational list knowledge representation, which is a simple data structure that enables easy modification, fast search and quick reasoning process to generate multiple conclusions. ...
Belt conveyors have been used worldwide in continuous material transport for about 250 years. Traditional inspection and monitoring of large-scale belt conveyors focus on individual critical components and response to catastrophic system failures. To prevent operational problems caused by the lack of experience of maintenance personnel, the monitoring and operational control of belt conveyors can be automated.
This research presents the possibilities to improve belt conveyor performance by means of integrating the information derived from traditional individual monitoring systems to automate and optimize the process of maintenance and operational control decision-making, based on the overall status of belt conveyors. This research concerns the areas of belt conveyor monitoring and artificial intelligence. Besides an embedded conductive detection system that has been developed as a novel belt conveyor monitoring technology, the outputs of this research project demonstrate the effectiveness of an intelligent monitoring and control system. The developed intelligent system is able to automatically acquire data from belt conveyor components, identify system abnormalities and discover potential causes, assess system health condition and operational status, store and retrieve experience and knowledge to optimize maintenance and operational control strategies.
... A fuzzy set A in a universe of discourse X is characterised by MF which associates with each element x in X (x ∈ X) a real number in the interval [0, 1], which indicates the extent to which x belongs to the fuzzy set X (Xu et al., 2005). A fuzzy number is a fuzzy subset in the universe of discourse X that is both convex and normal. ...
The advent of the Internet (World Wide Web) has facilitate greater access to the myriad of job opportunities available globally. Currently there exist many job applications submission portals that are being used for online job recruitment purposes. However, the task of many of these job submission portals is limited to matching the professional and academic qualifications of applicants with the requirements of employers and several organizations and does not involve the ranking of applicants' credentials according to their relative suitability for the jobs applied for. In this paper, we describe the design of Fuzzy Expert System (FES) for evaluation of the credentials of job applicants with the aim of reducing the riguor of the candidate selection process and minimizing the level of subjectivity in decision making.
... A fuzzy set A in a universe of discourse X is characterised by MF which associates with each element x in X (x ∈ X) a real number in the interval [0, 1], which indicates the extent to which x belongs to the fuzzy set X (Xu et al., 2005). A fuzzy number is a fuzzy subset in the universe of discourse X that is both convex and normal. ...
The advent of the internet has facilitated greater access to the myriad
of job opportunities available globally. Currently there exist many job
application submission portals that are being used for online job recruitment
purposes. However, the task of many of these job submission portals is limited to matching the professional and academic qualifications of applicants with the requirements of employers and several organisations and does not involve the ranking of applicants’ credentials according to their relative suitability for the jobs applied for. In this paper, we describe the implementation of fuzzy expert system (FES) tool for selection of qualified job applicants with the aim of minimising the rigour and subjectivity associated with the candidate selection process. A performance evaluation of the FES tool that was conducted confirmed the viability of a FES-based approach in handling the fuzziness that is associated with the problem of personnel recruitment.
... A fuzzy set A in a universe of discourse X is characterised by MF which associates with each element x in X (x ∈ X) a real number in the interval [0, 1], which indicates the extent to which x belongs to the fuzzy set X (Xu et al., 2005). A fuzzy number is a fuzzy subset in the universe of discourse X that is both convex and normal. ...
The advent of the internet has facilitated greater access to the myriad of job opportunities available globally. Currently there exist many job application submission portals that are being used for online job recruitment
purposes. However, the task of many of these job submission portals is limited to matching the professional and academic qualifications of applicants with the requirements of employers and several organisations and does not involve the ranking of applicants’ credentials according to their relative suitability for the jobs applied for. In this paper, we describe the implementation of fuzzy expert system (FES) tool for selection of qualified job applicants with the aim of minimising the rigour and subjectivity associated with the candidate selection process. A performance evaluation of the FES tool that was conducted confirmed the viability of a FES-based approach in handling the fuzziness that is associated with the problem of personnel recruitment.
For simulation-based systems, finding a set of test cases with the least cost by exploring multiple goals is a complex task. Domain-specific optimization goals (e.g. maximize output variance) are useful for guiding the rapid selection of test cases via mutation. But evaluating the selected test cases via mutation (that can distinguish the current program from the mutated systems) is a different goal to domain-specific optimizations. While the optimization goals can be used to guide the mutation analysis, that guidance should be viewed as a weak indicator since it can hurt the mutation effectiveness goals by focusing too much on the optimization goals. Based on the above, this paper proposes
DoLesS
(
Do
mination with
Le
a
s
t
S
quares Approximation) that selects the minimal and effective test cases by averaging over a coarse-grained grid of the information gained from multiple optimizations goals.
DoLesS
applies an inverted least squares approximation approach to find a minimal set of tests that can distinguish better from worse parts of the optimization goals. When tested on multiple simulation-based systems,
DoLesS
performs as well or even better as the prior state-of-the-art, while running 80-360 times faster on average (seconds instead of hours). For replication purposes, all our code is on-line:
https://github.com/ai-se/DoLesS
.
Regression testing ensures that modified software code changes have not adversely affected existing code modules. The test suite size increases with modification to the software based on the end-user requirements. Regression testing executes the complete test suite after updates in the software. Re-execution of new test cases along with existing test cases is costly. The scientific community has proposed test suite prioritization techniques for selecting and minimizing the test suite to minimize the cost of regression testing. The test suite prioritization goal is to maximize fault detection with minimum test cases. Test suite minimization reduces the test suite size by deleting less critical test cases. In this study, we present a four-fold methodology of test suite prioritization based on reinforcement learning. First, the testers’ and users’ log datasets are prepared using the proposed interaction recording systems for the android application. Second, the proposed reinforcement learning model is used to predict the highest future reward sequence list from the data collected in the first step. Third, the proposed prioritization algorithm signifies the prioritized test suite. Lastly, the fault seeding approach is used to validate the results from software engineering experts. The proposed reinforcement learning-based test suite optimization model is evaluated through five case study applications. The performance evaluation results show that the proposed mechanism performs better than baseline approaches based on random and t-SANT approaches, proving its importance for regression testing.
For cyber-physical systems, finding a set of test cases with the least cost by exploring multiple goals is a complex task. For example, Arrieta et al. reported that state-of-the-art optimizers struggle to find minimal test suites for this task. To better manage this task, we propose DoLesS (Domination with Least Squares Approximation) which uses a domination predicate to sort the space of possible goals to a small number of representative examples. Multi-objective domination then divides these examples into a "best" set and the remaining "rest" set. After that, DoLesS applies an inverted least squares approximation approach to learn a minimal set of tests that can distinguish best from rest in the reduced example space. DoLesS has been tested on four cyber-physical models: a tank flow model; a model of electric car windows; a safety feature of an AC engine; and a continuous PID controller combined with a discrete state machine. Comparing to the recent state-of-the-art paper attempted the same task, DoLesS performs as well or even better as state-of-the-art, while running 80-360 times faster on average (seconds instead of hours). Hence, we recommend DoLesSas a fast method to find minimal test suites for multi-goal cyber-physical systems. For replication purposes, all our code is on-line:https://github.com/hellonull123/Test_Selection_2021.
Purpose: Expert systems are computer-based systems that mimic the logical processes of human experts or organizations to give advice in a specific domain of knowledge. Fuzzy expert systems use fuzzy logic to handle uncertainties generated by imprecise, incomplete, and/or vague information. This paper presents a comprehensive review of the methods and applications in fuzzy expert systems.
Design/methodology/approach: We have carefully reviewed 281 journal publications and 149 conference proceedings published over the past 37 years since 1982. We grouped the journal publications and conference proceedings separately accordingly to the methods, application domains, tools, and inference systems.
Findings: We have synthesized our findings and proposed useful suggestions for future research directions. We show that the most common use of fuzzy expert systems is in the medical field.
Originality/value: Fuzzy logic can be used to manage uncertainty in expert systems and solve problems that cannot be solved effectively with conventional methods. In this study, we present a comprehensive review of the methods and applications in fuzzy expert systems which could be useful for practicing managers developing expert systems under uncertainty.
Object-oriented software has some features such as encapsulation, inheritance, cohesion, coupling and polymorphism. These features significantly affect the testing, maintenance and repair activities of object-oriented softwares. The successfully conclusion of these activities, depends on correct detect and correct interpretation of metrics will be evaluated. Also safely and rapidly evaluation of this metrics, is possible but by developing and using the software testing tools. Nowadays, many software testing tools that there are for object-oriented metrics. However, there are some limitations on the capabilities of this tools. Many of the this tools produces only metric findings, does not take any action in the point what this findings mean and how it affects the quality of the software. Therefore, diversifying and increasing capabilities of these tools is required. Based on the limitations of software testing tools mentioned in this study, is intended to provide the automation by expert system approach to object-oriented testing process. For this purpose, an expert module that review and interpret the metric findings, that providing information about the internal quality attributes of a project (.jar extension) developed according to object-oriented programming methodology have been developed. Metric suite that developed by Chidamber and Kemerer (C&K) have been used in this modul. This module, metric findings obtained by software testing tool named Metrics, has the evaluation feature by methods automatic, manuel and cross-version comparison and the interpreting the relationship between internal quality specifications. It has been expected that the betreffend module to be a support element to white-box tests been valid in software testing activities.
Regression testing is an important part of the software development life cycle. It is also very expensive. Many different techniques have been proposed for reducing the cost of regression testing. However, research has shown that the effectiveness of different techniques varies under different testing environments and software change characteristics. In prior work, we developed strategies to investigate ways of choosing the most cost-effective regression testing technique for a particular regression testing session. In this work, we empirically study the existing strategies presented in prior work as well as develop two additional Adaptive Test Prioritization (ATP) strategies using fuzzy Analytical Hierarchy Process (AHP) and the Weighted Sum Model (WSM). We also provide a comparative study examining each of the ATP strategies presented to date. This research will provide researchers and practitioners with strategies to utilize in regression testing plans as well as provide data to use when deciding which of the strategies would best fit their testing needs. The empirical studies provided in this research show that utilizing these strategies can improve the cost-effectiveness of regression testing.
In this paper an agent-based concept for the Adaptive Test Management System (ATMS) was presented. The test units, the test cases and the test resources are represented by software agents, which through negotiations and by integrated fuzzy-controllers plan the test. They prioritize test cases, request new test cases and allocate test resources based on a desired quality of test and the available test time. Due to dynamic events during the test, the test plan is continuously adapted from a test cycle to the other. In addition, due to the feedback about found defects during the test and during the operation, the test planning and control strategies are continuously optimized.
In Software Engineering Regression Testing is a mandatory activity. Whenever, a change in existing system occurs and new version appears, the unchanged portions need to be regression tested for any resulting undesirable effects. During process of Regression Testing, same test cases are executed repeatedly for un-modified portion of software. This activity is an overhead and consumes huge resources and budget. To save time and resources, researches have proposed various techniques for Regression Test Suite Optimization. In this research regression test suites are minimized using three Computational Intelligence multi-objective techniques for black box testing methods. These include; 1- Multi-Objective Genetic Algorithms (MOGA), 2- Non-Dominated Sorting Genetic Algorithm (NSGA-II) and 3- Multi-Objective Particle Swarm Optimization (MOPSO). Said techniques are applied on two published case studies and through experimentation, the quality of these techniques is analyzed. Four quality metrics are defined to perform this analysis. The results of research show that MOGA is better for reducing the size and thus execution time of the regression test suites as compared to MOPSO and NSGA-II. It was also found that use of MOGA, NSGA-II and MOPSO are not safe for regression test suite optimization. This is because fault detection rate and requirement coverage is reduced after optimization of Regression Test Suites.
Context: The use of system requirements and their risks enables software testers to identify more important test cases that can reveal the faults associated with system components.
Objective: The goal of this research is to make the requirements risk estimation process more systematic and precise by reducing subjectivity using a fuzzy expert system. Further, we provide empirical results that show that our proposed approach can improve the effectiveness of test case prioritization.
Method: In this research, we used requirements modification status, complexity, security, and size of the software requirements as risk indicators and employed a fuzzy expert system to estimate the requirements risks. Further, we employed a semi-automated process to gather the required data for our approach and to make the risk estimation process less subjective.
Results: The results of our study indicated that the prioritized tests based on our new approach can detect faults early, and also the approach can be effective at finding more faults earlier in the high-risk system components compared to the control techniques.
Conclusion: We proposed an enhanced risk-based test case prioritization approach that estimates requirements risks systematically with a fuzzy expert system. With the proposed approach, testers can detect more faults earlier than with other control techniques. Further, the proposed semi-automated, systematic approach can easily be applied to industrial applications and can help improve regression testing effectiveness.
Regression Testing is performed on already tested programs to ensure that modifications have not revealed defects into the unmodified portions of programs. Regression Test Suites are always growing due to addition of Test Cases. Many broken and redundant test cases are also part of regression test suites. Running the all regression test suite is always not feasible; therefore optimization of regression test suites is required to meet the constraints. In this work we proposed multi-objective optimization of regression test suites with Fuzzy Logic (Sugeno Model) for Black Box based testing methods. Proposed approach was implemented on two published case studies. Results indicates that optimization can be achieved by using Fuzzy Logic that is a safe technique for optimization and we can reduce the execution time and size of regression test suites up to 50%.
Test case prioritization schedules the test cases in an order that increases the effectiveness in achieving some performance goals. One of the most important performance goals is the rate of fault detection. Test cases should run in an order that increases the possibility of fault detection and also detects the most severe faults at the earliest in its testing life cycle. Test case prioritization techniques have proved to be beneficial for improving regression testing activities. While code coverage based prioritization techniques are found to be studied by most scholars, test case prioritization based on requirements in a cost effective manner has not been used for studies so far. Hence, in this paper, we propose to put forth a model for system level Test Case Prioritization (TCP) from software requirement specification to improve user satisfaction with quality software that can also be cost effective and to improve the rate of severe fault detection. The proposed model prioritizes the system test cases based on six factors: customer priority, changes in requirement, implementation complexity, usability, application flow and fault impact. The proposed prioritization technique is experimented in three phases with student projects and two sets of industrial projects and the results show convincingly that the proposed prioritization technique improves the rate of severe fault detection.
Different testing environments and software change characteristics can affect the choice of regression testing techniques. In our prior work, we developed adaptive regression testing (ART) strategies to investigate this problem. While the ART strategies showed promising results, we also found that the multiple criteria decision making processes required for the ART strategies are time-consuming, often inaccurate and inconsistent, and limited in their scalability. To address these issues, in this research, we develop and empirically study a fuzzy expert system (FESART) to aid decision makers in choosing the most cost-effective technique for a particular software version. The results of our study show that FESART is consistently more cost-effective than the previously proposed ART strategies. One of the biggest contributors to FESART being more cost-effective is the reduced time required to apply the strategy. This contribution has significant impact because a strategy that is less time-consuming will be easier for researchers and practitioners to adopt, and will provide even greater cost-savings for regression testing sessions.
Regression Test Suite Optimization (RTO) is an active research area. A Regression Test Suite is always growing due to changes in software, which increases testing time. To save time and resources optimization of regression test suites is mandatory. Researchers have optimized test suites using conventional and Computational Intelligence based approaches and achieve optimization of regression test suites through selection techniques, minimization or reduction techniques and ranking or prioritization techniques. This paper surveys existing techniques for regression test suite optimization, various tools and mathematical models being used for RTO. During this survey we found many interesting facts about regression test suite optimization that will be shared in the conclusion.
Regression Test suite optimization is an effective technique to reduce time and cost of testing. Many researchers have used computational intelligence techniques to enhance the effectiveness of test suite. These approaches optimize test suite for a single objective. Introduction of fuzzy logic with genetic algorithm and swarm optimization may be used to optimize test suite for multi-objective selection criteria. Secondly, human intervention or expert judgment is required to opt for level of testing, technique used and quality aspect to be tested. Fuzzy logic has proved its worth in many other domains like communication, bio informatics, embedded applications, industrial and engineering control and network optimization. We propose an expert system that finds a trade off among the quality aspects, technique used and level of testing based on objective function defined by the tester, quite similar to human judgment using fuzzy logic based classification. Main focus of our approach is to find a test suite that is optimal for multi-objective regression testing. For proof of our concept, initially we focused on three quality aspects, i.e., performance, throughput and code coverage.
In this paper an Adaptive Test Management System (ATMS) based on software agents is presented which prioritizes test cases considering available information from test teams and from developments teams about the software system and the test cases. The goal of the ATMS is to increase the number of found faults in the available test time with the determined prioritization order.
Test case prioritization schedules the test cases in an order that increases the effectiveness in achieving some performance goals. One of the most important performance goals is the rate of fault detection. Test cases should run in an order that increases the possibility of fault detection and also that detects the most severe faults at the earliest in its testing life cycle. Test case prioritization techniques have proved to be beneficial for improving regression testing activities. While code coverage based prioritization techniques are found to be taken by most scholars, test case prioritization based on requirements in a cost effective manner has not been taken for study so far. Hence, in this paper, we propose to put forth a model for system level test case prioritization (TCP) from software requirement specification to improve user satisfaction with quality software that can also be cost effective and to improve the rate of severe fault detection. The proposed model prioritizes the system test cases based on the six factors: customer priority, changes in requirement, implementation complexity, usability, application flow and fault impact. The proposed prioritization technique is experimented in three phases with student projects and two sets of industrial projects and the results show convincingly that the proposed prioritization technique improves the rate of severe fault detection.
When software is modified, some functionality that had been working can be affected. The reliable way to guarantee that the software is working correctly after those changes is to test the whole system again, but generally there is not sufficient time. Then, it is necessary to select significant test cases to be executed, in order to guarantee that the system is working as it should be. Although there are already works regarding on the regression test case selection problem, some important features which can influence in the test case selection are not considered in them. In this work, we state a new and more complete multi-objective formulation for this problem. The work also shows the results of the solution for the problem using a multi-objective genetic algorithm, comparing it with a random algorithm.
Wireless sensor network is the core support technology in the framework of the Internet of Things. Environmental monitoring and devices control of intelligent building based on wireless sensor network is considered as one of the most crucial applications. It can perceive several environmental parameters and feedback control information to devices to provide more comfortable environment for people. However, it is difficult to ensure performance of wireless sensor network deployed in the buildings, because there usually are several serious disturbances from coexistence wireless systems and human actions. In this paper, we deploy a practical Intelligent Building Wireless Sensor Network (IBWSN) based on the analysis of actual spectrum, Link Quality Indicator, Received Signal Strength Indicator and Package Loss Rate. Performance evaluation proves that the presented system can satisfy the needs of the proposed applications for environmental monitoring and air-conditioning control.
In the scope of industrial automation systems, a relatively high quality of the software system test has to be achieved by the test management in order to guarantee the required reliability of the system. A major challenge of the test manager is the limited number of available test resources. The test manager cannot execute a complete test with all test cases in each test cycle. He has to decide with much manual work, which test cases are executed. In this paper, the Adaptive Test Management System (ATMS) based on software agents is presented to relieve the test manager from this manual work. The goal of the ATMS is to increase the test efficiency and reduce the test costs.
A big challenge of software test managers is the limited test time. Especially the system test, where the whole integrated
software system is tested shortly before delivery to the customer, is affected by this limitation. During the system test
usually several test cycles are needed. However, a test manager cannot execute all available test cases in each test cycle
due to the limited test time. He/she has to decide which test cases have to be executed in each test cycle in order to find
new possible faults of the software. In this paper the Adaptive Test Management System (ATMS) based on software agents is
presented which relieves the test manager from this complex manual work by using software agents for prioritizing test cases
based on current information about the software system, the test cases and the human factors of the developers. The goal of
the ATMS is to maximize the number of found faults in the available test time with the determined prioritization order.
The paper analyzes knowledge mining of the test case System. Widespread use of test case systems and explosive growth of databases require traditional manual data analysis to be coupled with methods for efficient computer-assisted analysis It is very important for us to utilize this kind of information effectively. Today, the design of test case with enhanced reliability is a real challenge as it needs expert designers with perfect knowledge about the whole system and also the traditional test case generation approach faces a challenge in test results analysis, because of the difference between the generated test cases and the expected results. So, it will be better if we could establish a method for automatic test case mining based on both program structures and the functional requirements in specifications. Mining approach can be used to have a perfect knowledge about the whole system. In order to improve the accuracy and efficiency of knowledge acquisition, it establishes a knowledge mining model.
From the Publisher:Fuzzy and Neural Approaches in Engineering presents a detailed examination of the fundamentals of fuzzy systems and neural networks and then joins them synergistically - combining the feature extraction and modeling capabilities of the neural network with the representation capabilities of fuzzy systems. Exploring the value of relating genetic algorithms and expert systems to fuzzy and neural technologies, this forward-thinking text highlights an entire range of dynamic possibilities within soft computing. With examples of specifically designed to illuminate key concepts and overcome the obstacles of notation and overly mathematical presentations often encountered in other sources, plus tables, figures, and an up-to-date bibliography, this unique work is both an important reference and a practical guide to neural networks and fuzzy systems.
Faults discovered by customers are an important aspect of software quality. The working hypothesis of this paper is that variables derived from an execution profile can be useful in software quality models. An execution profile of a software system consists of the probability of execution of each module during operations. Execution represents opportunities for customers to discover faults. However, an execution profile over an entire customer-base can be difficult to measure directly. Deployment records of past releases can be a valuable source of data for calculating an approximation to the probability of execution. We analyze a metric derived from deployment records which is a practical surrogate for an execution profile in the context of a software quality model. We define usage as the proportion of systems in the field which have a module deployed. We present a case study of a very large legacy telecommunications system. We developed models using a standard statistical technique to predict whether software modules will have any faults discovered by customers on systems in the field. Static software product metrics and usage were independent variables. The significance levels of variables in logistic regression models were analyzed, and models with and without usage as an independent variable were compared. The case study was empirical evidence that usage can be a significant contributor to a software quality model
In traditional knowledge-based system development environments, the fundamental representational building blocks are mechanisms such as frames, rules, and attached procedures. The KEE system has been extended to include both a context (worlds) system and a truth maintenance system.
Many evolving mission‐critical systems must have high software reliability. However, it is often difficult to identify fault‐prone
modules early enough in a development cycle to guide software enhancement efforts effectively and efficiently. Software quality
models can yield timely predictions of membership in the fault‐prone class on a module‐by‐module basis, enabling one to target
enhancement techniques. However, it is an open empirical question, “Can a software quality model remain useful over several
releases?” Most prior software quality studies have examined only one release of a system, evaluating the model with modules
from the same release. We conducted a case study of a large legacy telecommunications system where measurements on one software
release were used to build models, and three subsequent releases of the same system were used to evaluate model accuracy.
This is a realistic assessment of model accuracy, closely simulating actual use of a software quality model. A module was
considered fault‐prone if any of its faults were discovered by customers. These faults are extremely expensive due to consequent
loss of service and emergency repair efforts. We found that the model maintained useful accuracy over several releases. These
findings are initial empirical evidence that software quality models can remain useful as a system is maintained by a stable
software development process.
Typescript (Photocopy). Thesis (M.S.C.S.)--Florida Atlantic University, 1996. Bibliography: leaves 57-59.
Typescript (Photocopy). Thesis (Ph. D.)--Florida Atlantic University, 2001. Includes bibliographical references (leaves 262-274).
Thesis (M.S)--Florida Institute of Technology, 1993. Includes bibliographical references (leaves 73-75).
Test case prioritization techniques schedule test cases for regression testing in an order that increases their ability to meet some performance goal. One performance goal, rate of fault detection, measures how quickly faults are detected within the testing process. In previous work (S. Elbaum et al., 2000; G. Rothermel et al., 1999), we provided a metric, APFD, for measuring rate of fault detection, and techniques for prioritizing test cases to improve APFD, and reported the results of experiments using those techniques. This metric and these techniques, however, applied only in cases in which test costs and fault severity are uniform. We present a new metric for assessing the rate of fault detection of prioritized test cases that incorporates varying test case and fault costs. We present the results of a case study illustrating the application of the metric. This study raises several practical questions that might arise in applying test case prioritization; we discuss how practitioners could go about answering these questions.
Test case prioritization orders tests so that they help you meet your testing goals earlier during regression testing. Prioritization techniques can, for example, order tests to achieve coverage at the fastest rate possible, exercise features in order of expected frequency of use, or reveal faults as early as possible. We focus on the last goal, which we describe as "increasing a test suite's rate of fault detection" or the speed with which the test suite reveals faults. A faster fault detection rate during regression testing provides earlier feedback on a system under test, supporting earlier strategic decisions about release schedules and letting engineers begin debugging sooner. Also, if testing time is limited or unexpectedly reduced, prioritization increases the chance that testing resources will have been spent as cost effectively as possible in the available time.
Test case prioritization techniques schedule test cases for execution in an order that attempts to increase their effectiveness at meeting some performance goal. Various goals are possible; one involves rate of fault detection, a measure of how quickly faults are detected within the testing process. An improved rate of fault detection during testing can provide faster feedback on the system under test and let software engineers begin correcting faults earlier than might otherwise be possible. One application of prioritization techniques involves regression testing, the retesting of software following modifications; in this context, prioritization techniques can take advantage of information gathered about the previous execution of test cases to obtain test case orderings. We describe several techniques for using test execution information to prioritize test cases for regression testing, including: 1) techniques that order test cases based on their total coverage of code components; 2) techniques that order test cases based on their coverage of code components not previously covered; and 3) techniques that order test cases based on their estimated ability to reveal faults in the code components that they cover. We report the results of several experiments in which we applied these techniques to various test suites for various programs and measured the rates of fault detection achieved by the prioritized test suites, comparing those rates to the rates achieved by untreated, randomly ordered, and optimally ordered suites
Answer justification refers to the ability of an expert system to explain how or why it arrived at certain conclusions (such as a patient's differential diagnosis or treatment recommendations). In this paper, we describe an "abductive" inference method suitable for use in medical expert systems. We then demonstrate how this method can support a clinically plausible form of answer justification in functioning expert systems. A companion paper (Part II) provides the technical details of how the answer justification method described in this paper is implemented, and compares it to previous answer justification methods developed during the last several years.
A novel adaptive neural network based technique to enhance software sys-tem assurance for regression testing
- Z Xu
- N Samaan
Z. Xu and N. Samaan. A novel adaptive neural network based technique to enhance software sys-tem assurance for regression testing. In Proceed-ings: Sixth IEEE International High-Assurance Sys-tems Engineering Symposium, Boca Raton, Florida, USA, Oct. 2001. IEEE Computer Society.