Figure 4 - uploaded by Dr. Praveen Ranjan Srivastava
Content may be subject to copyright.
Source publication
Software Testing is the process of testing the software in order to ensure that it is free of errors andproduces the desired outputs in any given situation. Model based testing is an approach in whichsoftware is viewed as a set of states. A usage model describes software on the basis of its statistical usagedata. One of the major problems faced in...
Contexts in source publication
Context 1
... 1.0 1 0.6 2 0.635 3 0.1 4 0.1 5 0.1 6 0.47 7 0.0 Now as the ant approaches from node to make the first decision i.e. either {0,1} or {0,2}, it is assumed that this decision is random. Ant colony optimization has been avoided at this stage in order to avoid bias towards any branch of the graph emanating from this start state. Any branch emanating from this point carrying a very critical state could force the ants to move over this same path over and over again leaving other less critical paths unvisited for a long time (eventually they will be covered in any case). Once the ant makes this first decision it makes calculations of the P-factors of all nodes immediately reachable from the current node and as described above makes the traversal to the node with the maximum P-factor. Now many such ants are consequently made to traverse the graph. Each leaves its own pheromone trail and hence guides the subsequent ants to the optimal paths. The pheromone levels are updated periodically according to equation (3). Figure 4 shows three paths obtained by three ants in their traversals. Each path is essentially a test sequence. As the test sequences are being generated the Cumulative Cost (equation (7)) is simultaneously calculated. The moment it approaches or exceeds the cost_limit the process is stopped. Once it has been ensured that all the critical nodes and their attached transitions have been covered in the test sequences, the subsequent traversals focus on covering the unvisited nodes and edges giving them a priority over visited, yet more critical nodes (with a higher P-factor). This has been done to ensure complete coverage as long as the cost limitations allow it. In this example it is clear the most critical states (refer Table 3) are 3, 4, 5 and 6. A sample traversal (see test sequence-1 Figure 4) of the model using proposed technique is given below. The ant starts at start state 0. Say the random decision taken is to go ahead to state 2. So the status of current sequence is 0 2. At state 2 the ant has to calculate the P-factor values of all the states that can be reached from 2 in one transition. Namely, these states are 3, 4, 5and 7. The P-factor values calculated is as follow. Using equation (5) for ...
Context 2
... 1.0 1 0.6 2 0.635 3 0.1 4 0.1 5 0.1 6 0.47 7 0.0 Now as the ant approaches from node to make the first decision i.e. either {0,1} or {0,2}, it is assumed that this decision is random. Ant colony optimization has been avoided at this stage in order to avoid bias towards any branch of the graph emanating from this start state. Any branch emanating from this point carrying a very critical state could force the ants to move over this same path over and over again leaving other less critical paths unvisited for a long time (eventually they will be covered in any case). Once the ant makes this first decision it makes calculations of the P-factors of all nodes immediately reachable from the current node and as described above makes the traversal to the node with the maximum P-factor. Now many such ants are consequently made to traverse the graph. Each leaves its own pheromone trail and hence guides the subsequent ants to the optimal paths. The pheromone levels are updated periodically according to equation (3). Figure 4 shows three paths obtained by three ants in their traversals. Each path is essentially a test sequence. As the test sequences are being generated the Cumulative Cost (equation (7)) is simultaneously calculated. The moment it approaches or exceeds the cost_limit the process is stopped. Once it has been ensured that all the critical nodes and their attached transitions have been covered in the test sequences, the subsequent traversals focus on covering the unvisited nodes and edges giving them a priority over visited, yet more critical nodes (with a higher P-factor). This has been done to ensure complete coverage as long as the cost limitations allow it. In this example it is clear the most critical states (refer Table 3) are 3, 4, 5 and 6. A sample traversal (see test sequence-1 Figure 4) of the model using proposed technique is given below. The ant starts at start state 0. Say the random decision taken is to go ahead to state 2. So the status of current sequence is 0 2. At state 2 the ant has to calculate the P-factor values of all the states that can be reached from 2 in one transition. Namely, these states are 3, 4, 5and 7. The P-factor values calculated is as follow. Using equation (5) for ...
Context 3
... P 23 = {( τ ij ) α x ( η ij ) β } / Σ ( τ ij ) α x ( η ij ) β = {(1.0) 1 x (0.25) 2 } /6.5 = 0.3824 Putting these values in equation (2) CR 3 = 30.0 from Table 2 AV 3 = 0.1 from Table 3 Similarly, P-factor 4 = P 24 + CR 4 + AV 4 = 26.138 P-factor 5 = P 25 + CR 5 + AV 5 = 27.138 P-factor 7 = P 27 + CR 7 + AV 7 = 0.0384 Hence the ant selects state 3 for next transition. The test sequence till now is 0 2 3. At state 3 the only transition possible is {3,7}. Final test sequence generated is 0 2 3 7. Following are some of the outputs obtained at various cost_limit values. In each of the following obtained outputs, the number of critical states covered will be checked. This is done in order to verify whether the critical states are being covered even under very strict cost limitations. The following test sequences were obtained when cost_limit was 20.0. As it can be seen, the number of test sequences is 6 and there is no redundancy in the set of test sequences. Moreover this set of test sequences is providing a full coverage (state and transition). All the critical states have been covered here (as can be inferred from a full state coverage). When the cost_limit was 15.0 the following set of test sequences was obtained. 0 2 3 7 0 1 6 2 3 7 0 1 1 0 2 5 7 0 2 4 7 0 2 7 Here the number of test sequences has reduced to 5. It is still providing full state coverage and all critical states have been covered. Although all transition coverage has not been achieved it is trade-off between cost considerations and coverage. In any case coverage of all critical states has been achieved. As the cost_limit is further decreased it can be observed that the number of test sequences will quite obviously decrease. When the cost_limit was set to 10.0 the following output was obtained. 0 2 3 7 0 1 6 2 3 7 0 1 1 0 2 5 7 Here it can be seen that the number of test sequences is 3 and that although complete state and transitions coverage has not been achieved, the most critical nodes have been covered (all except state 4, namely states 3, 5 and 6). When cost_limit was set to 25.0 the following output was obtained. 0 2 3 7 0 1 6 2 3 7 0 1 1 0 2 5 7 0 2 4 7 0 2 7 0 1 6 7 It is evident that this is same result as when the limit was 20.0. Similar results were obtained when the cost_limit was set to values above 25.0. So it can be inferred that after a certain threshold value of cost_limit the there is no change in the output and that further cost investment is not required. The optimal test sequence set (giving full coverage) is obtained at a certain threshold value. A similar trend is seen in case of number of critical nodes covered. In Figure 4 and 5 the cost_limit has been plotted versus the number of test sequences and the number of critical nodes covered. It is clearly visible that after the threshold value of 20.0 the curve is a constant. Li and Lam [20] have worked on the use of Ant colony optimization in test sequence generation in a state based approach to testing. But the work does not factor in criticality of the states or give conclusively optimized test sequences. A comparative study has been made using results obtained by applying the above mention technique [20] and the proposed technique. This work has been chosen for a comparative study because as per our research this is the only work that incorporates ACO in model based testing, and the purpose of this comparative study is to prove that a markov chain usage model is better than a simple state based model for the application of ACO. An equal number of test sequences are generated by both the techniques. These are test sequences generated by traversals by equal number of ants. The comparative study has been made on the basis of the number of nodes and edges covered in the same number of traversals. The n test sequences considered here are derived from the first n traversals of the model. This has been done because higher coverage within lesser number of test sequences strongly implies more the cost efficiency and hence a more optimal solution. The model used for this study is given in Figure 3. We consider the states 3, 4, 5 and 6 as most critical (refer Table 3). There are 15 edges (transitions) in all and 8 nodes (states). When the number of test sequences is 4 the set of test sequences obtained is as follows Existing technique 0 2 7 0 2 3 7 0 2 4 7 0 2 5 7 Number of nodes covered = 6 (3 out of 4 most critical) Number of edges covered = 8 Percentage of node coverage = (6/8) x 100 = 75 Percentage of edge coverage = (8/15) x 100 = 53.33 Proposed technique 0 2 3 7 0 1 6 2 3 7 0 1 1 0 2 5 7 0 2 4 7 Number of nodes covered = 8 (all the 4 most critical) Number of edges covered = 12 Percentage of node coverage = (8/8) x 100 = 100 Percentage of edge coverage = (12/15) x 100 = 80 Figure 7 and Figure 8 show a graph describing the results obtained above. When the number of test sequences is 4 the set of test sequences obtained is as follows Existing technique 0 2 7 0 2 3 7 Number of nodes covered = 4 (1 out of 4 most critical) Number of edges covered = 3 Percentage of node coverage = (4/8) x 100 = 50 Percentage of edge coverage = (3/15) x 100 = 20 Proposed technique 0 2 3 7 0 1 6 2 3 7 Number of nodes covered = 6 (2 out of 4 most critical) Number of edges covered = 6 Percentage of node coverage = (6/8) x 100 = 75 Percentage of edge coverage = (6/15) x 100 = 40 Figure 9 and Figure 10 show a graph describing the results obtained above. It is quite evident that the proposed technique not only ensures a better coverage but also ensures that the more critical states are covered too. Doerner and Gutjahr [11] have used Ant colony optimization to generate test sequences from markov chain based models. But the technique described [11] does not take the criticality of the states into account. The technique proposed in this paper uses criticality of state explicitly as well computes its own measure (average number of visits) to gauge the criticality of any state. It gives better coverage of critical nodes, especially when there are strict cost limitations. Under such conditions although a lesser number of test sequences are generated, the set is optimal for that given cost. This is a very desirable result. The above discussed technique has proven to be effective in generating optimal set of test cases from a markov chain based usage model. Especially with the growth in software with extensive graphical interfaces, usage modelling will prove to be very fruitful. The technique takes into account the average number of visits to any state; this is a very important factor when testing graphical interfaces. The discussed technique has also proved to be capable of deriving test cases with priority given to the most critical states and transitions. Even with strict cost limitations it has been observed that the most critical states have been covered. Future works upon this technique could involve the presence of multiple transitional probabilities between states [6]. The use of ant colony optimization has proven to be effective and provides an efficient means to generate test sequences especially in graph based models. The complexity of algorithm used in this technique is O (n 2 ) for a traversal and hence proves to be efficient and less computation ...
Context 4
... pheromone levels are updated periodically according to equation (3). Figure 4 shows three paths obtained by three ants in their traversals. Each path is essentially a test sequence. ...
Context 5
... this example it is clear the most critical states (refer Table 3) are 3, 4, 5 and 6. A sample traversal (see test sequence-1 Figure 4) of the model using proposed technique is given below. ...
Citations
... In this method, the SUT model contains the probabilities of transitions between individual application states based on which test cases are effectively generated that match the desired importance. The proposed method yields good coverage of critical nodes using a small number of test sequences [21]. Sayyari and Emadi proposed a similar approach [22]. ...
... each n in ∈ in(L) do19 SET path as empty ; ▷ path is a sequence of nodes 20 n ← n in ; ▷ n is a currently traversed node21 while n ̸ = n out do22 n ← x ∈ L such that distance(n, x) is minimal23 ADD n at the end of path 24 end 25 ADD n at the end of path GetN extShortestP athInLCZ(C, P, n in , U in , U out ): ...
Contemporary Internet-of-Things (IoT) systems are hindered by several reliability-related issues, especially, the dynamic behavior of IoT systems caused by limited and often unstable network connectivity. Several intuitive ad-hoc approaches can be employed to test this behavior; however, the effectiveness of these approaches in detecting defects and their overall testing costs remain questionable. Therefore, we present a new specialized path-based technique to test the processes of an IoT system in scenarios wherein parts of these processes are influenced by limited or disrupted network connectivity. The proposed technique can be scaled using two levels of test coverage criteria to determine the strengths of the test cases. For this purpose, we propose two algorithms for generating test cases to implement the technique: an ant colony optimization-based search and a graph-traversal-based test case composition. We compared the efficiency of the proposed approach with possible solutions obtained using a standard path-based testing approach based on prime paths computed by a set-covering algorithm. We consider the total number of test case steps as the main proxy for test effort in experiments employing 150 problem models. For the less intensive of the two used test-coverage criteria, EachBorderOnce, an ant colony optimization-based algorithm, produced test sets with the same averaged number of steps as the graph traversal-based test-case composition; however, this algorithm performed with averaged number of steps 10% lower than a prime paths-based algorithm. For the more intensive test coverage criterion, AllBorderCombinations, these differences favoring the ant colony optimization-based algorithm were 18% and 25%, respectively. For these two types of defined test coverage criteria, the ant colony optimization-based search, graph-traversal-based algorithm, and standard path-based testing approach based on prime paths achieved the best results for 93 and 78, 14 and 24, and 13 and 17 models for AllBorderCombinations and EachBorderOnce criterion, respectively. Therefore, to guarantee the best test set, all compared algorithms are combined in a portfolio strategy that yields the best results based on the potential of the produced test sets to detect simulated defects caused by limited network connectivity. Additionally, this portfolio strategy also yields test sets, implying the lowest test effort for experimental problem instances.
... Sayyari and Emadi [19] exploit ant colony algorithms to generate TPs covering PPs. Sirvastava et al. [20] extract a Markov chain model and produce an optimal test set. Bidgoli et al. [6] apply swarm intelligence algorithms using a normalized fitness function to ensure the coverage of PPs. ...
... Other nature inspired algorithms have also been proposed, for example, Sayyari and Emadi [19] propose a solution based on ant colony optimization algorithm to develop TPs. Srivastava et al. [20] propose a technique using ant colony optimization to generate optimized test sequences from markov chain model of the software under test. Bidgoli et al. [6] utilize two prominent swarm intelligence algorithms, i.e., ACO and PSO, along with a normalized fitness function to provide an approach for covering PPs. ...
... Hosseini et al [13] PP Coverage Genetic algorithm Sayyari et al [19] PP Coverage Ant colony optimization Srivastava et al [20] PP Coverage Ant colony optimization Bidgoli et al [6] PP Coverage Ant colony and PSO Lin et al [15] PP Coverage Genetic algorithm Bueno et al [7] PP Coverage Genetic algorithm ...
This paper presents a novel scalable GPU-based method for Test Paths (TPs) and Prime Paths (PPs) Generation, called TPGen, used in structural testing and in test data generation. TPGen outperforms existing methods for PPs and TPs generation in several orders of magnitude, both in time and space efficiency. Improving both time and space efficiency is made possible through devising a new non-contiguous and hierarchical memory allocation method, called Three-level Path Access Method (TPAM), that enables efficient storage of maximal simple paths in memory. In addition to its high time and space efficiency, a major significance of TPGen includes its self-stabilizing design where threads execute in a fully asynchronous and order-oblivious way without using any atomic instructions. TPGen can generate PPs and TPs of structurally complex programs that have an extremely high cyclomatic and Npath complexity.
... Generating test sequences from a specification using multiagent systems has been suggested by Kruse [25]. Srivastava et al. use ant colony optimization to generate test sequence that achieve either state coverage or transition coverage of a given FSM [33]. In this latter case, probabilities are associated to each transition, making the model a Markov chain. ...
... Sayyari and Emadi [17] utilize ant colony optimization to generate TPs. Srivastava et al. [18] take a Markov Chain model of PUT and generate optimized test sequences. Bidgoli et al. [6] exploit swarm intelligence algorithms along with a normalized fitness function to ensure prime paths coverage. ...
This paper investigates the problem of prime and test paths generation, which is an important problem in ensuring path coverage in software testing. Most existing methods for prime/test paths generation have little success in generating the set of all prime/test paths of structurally complex programs with high Npath complexity. This paper puts forward two novel methods for the generation of prime and test paths of highly complex programs, namely a vertex-based algorithm and a compositional method. The proposed vertex-based method enables a time-efficient approach for the generation of all prime paths, and the compositional method provides a highly time and space-efficient method for the generation of prime and test paths in cyclic control flow graphs with extremely large Npath complexity. We also implement the proposed algorithms as a software toolset for the generation of prime and test paths. Our experimental results on a set of complex programs indicate that the proposed approaches significantly outperform existing methods, especially for large and structurally complex programs.
... Additionally, genetic algorithms have been employed to generate the prime paths [6] or to generate basis test paths [16]. Other natureinspired algorithms have also been proposed, for example, ant colony optimization algorithms [5], [17], the firefly algorithm [18] and algorithms inspired by microbiology [4]. ...
Executing various sequences of system functions in a system under test represents one of the primary techniques in software testing. The natural way to create effective, consistent and efficient test sequences is to model the system under test and employ an algorithm to generate the tests that satisfy a defined test coverage criterion. Several criteria of test set optimality can be defined. In addition, to optimize the test set from an economic viewpoint, the priorities of the various parts of the system model under test must be defined. Using this prioritization, the test cases exercise the high priority parts of the system under test more intensely than those with low priority. Evidence from the literature and our observations confirm that finding a universal algorithm that produces an optimal test set for all test coverage and test set optimality criteria is a challenging task. Moreover, for different individual problem instances, different algorithms provide optimal results. In this paper, we present a path-based strategy to perform optimal test selection. The strategy first employs a set of current algorithms to generate test sets; then, it assesses the optimality of each test set by the selected criteria, and finally, chooses the optimal test set. The experimental results confirm the validity and usefulness of this strategy. For individual instances of 50 system under test models, different algorithms provided optimal results; these results varied by the required test coverage level, the size of the priority parts of the model, and the selected test set optimality criteria.
... Srivastava et al. [4] worked on generation of optimal set of test sequences from markov chain based usage model using ACO. Factors like cost, average number of visits and criticality of the various states in the model were kept in mind. ...
Software testing is one of the most arduous and challenging phase which is to be implemented with the intention of finding faults with the execution of minimum number of test cases to increase the overall quality of the product at the time of delivery or during maintenance phase. With the ever increasing demand of web applications and to meet never ending customer expectations, updations are to incorporate which will be validated through testing process. The structure of the web applications (dynamic website) can be modeled using weighted directed graph which consists of numerous paths starting from homepage (index page) of the website. For thorough testing of the website each and every path of the graph should be tested but due to various constraints like time, money and human resources it becomes very much impractical. This scenario ultimately gives rise to the motivation for the development of technique which reduces the number of paths to be tested so that tester community can test only these numbers of path instead of all possible paths so that satisfactory number of faults can be exposed. In this proposed approach assignment of weights on the edges of the directed graph takes place on the basis of the organization of the website, changes in the structure of the website at page level, experience of the coder and the behaviour of the users who have visited the website earlier. The most fault prone paths are identified using random, greedy, Ant Colony Optimization (ACO) and Artificial Bee Colony Optimization (ABCO) algorithms. Two small size websites and one company's website, and their two versions, were considered for experimentation. Results obtained through ACO and ABCO are promising in nature. This approach will support testing process to be completed in time and delivery of the updated version within given hard deadlines.
... This problem can be efficiently solved using a greedy algorithm [8,Chapter 10]. Test sequence generation has also been studied when the specification is expressed as UML statecharts [1,5,6,12], using various algorithms including multi-agent systems [7], ant colony optimization [11] and genetic algorithms [2]. ...
SealTest is a Java library for generating test sequences based on a formal specification. It allows a user to easily define a wide range of coverage metrics using multiple specification languages. Its simple and generic architecture makes it a useful testing tool for dynamic software systems, as well as an appropriate research testbed for implementing and experimentally comparing test sequence generation algorithms.
... To avoid this stagnation, pheromone evaporation is implemented. In other words, the pheromone associated with a solution disappears after a period of time [20]. ...
... So it is not suitable of strong level software coverage. Srivastava [14], [15] has proposed a technique to generate optimized test sequences from a Markov chain based usage model. Major merits of his work are that this technique has proved to be capable of deriving test cases giving priority to the most critical states and transitions. ...
... It is known (Pressman, 2001) that not all possible test cases can be tested in polynomial time owing to the problem showing characteristics typical of NP Hard complex problems (Alander et al., 1998). This amalgamates artificial intelligence concepts into software testing and discards the traditional approach to give way to much faster techniques based on heuristics and meta-heuristics (Srivastava et al., 2010a(Srivastava et al., , 2010bDoungsa-ard et al., 2005). These terms often describe the primary subfield of stochastic optimisation. ...
... Work done by Li and Lam (2004) and Srivastava et al. (2010a), to generate test data based on ant colony optimisation technique along with the branch function technique and problem of local optimisation has also been solved. The model takes into account factors such as cost and criticality of various states in the model. ...
... This section fulfils its purpose of analysing the empirical results of the presented approach along with a few previous techniques. The techniques suggested by Genetic Algorithm (Doungsa-ard et al., 2005), Ant Colony Optimisation (Srivastava et al., 2010a), Cuckoo Search Algorithm (Srivastava et al., 2012a) and the Intelligent Water Drop (Srivastava et al., 2012b) have already provided feasible solutions to many software testing problems. The results of the proposed approach have been compiled and statistically tested with those of the above mentioned techniques, and they show promising results in continuing the research ahead. ...
Software testing has always been a significant component of the software development
life cycle. Also, in any project a good amount of time and cost is required for testing. For optimal
results, maximum testing in minimum time is desired with fewest repetitions. In large and
complex systems, this is possible only through use of meta-heuristic algorithms to help decide
which portions to test first. A similar approach is presented in this paper which demonstrates how
the bat algorithm works upon a given state chart diagram and suggests favourable test sequences
keeping in mind the critical system modules, which often need to be tested first. A graphical
representation and comparative results with other algorithms currently in use reflect the basis of
choosing the new alternative technique that this paper presents.
