Content uploaded by Gregory Provan
Author content
All content in this area was uploaded by Gregory Provan on Mar 06, 2014
Content may be subject to copyright.
A preview of the PDF is not available
Computing Minimal Diagnoses by Greedy Stochastic Search.
Abstract and Figures
Most algorithms for computing diagnoses within a model-based diagnosis framework are deterministic. Such algorithms guarantee soundness and completeness, but are Σ2P-hard. To overcome this complexity problem, which prohibits the computation of high-cardinality diagnoses for large systems, we propose a novel approximation approach for multiple-fault diagnosis, based on a greedy stochastic algorithm called SAFARI (StochAstic Fault diagnosis Algo-RIthm). We prove that SAFARI can be configured to compute diagnoses which are of guaranteed minimality under subsumption. We analytically model SAFARI search as a Markov chain, and show a probabilistic bound on the minimality of its minimal diagnosis approximations. We have applied this algorithm to the 74XXX and ISCAS85 suites of benchmark combinatorial circuits, demonstrating order-of-magnitude speedups over two state-of-the-art deterministic algorithms, CDA* and HA*, for multiple-fault diagnoses.
Figures - uploaded by Gregory Provan
Author content
All figure content in this area was uploaded by Gregory Provan
Content may be subject to copyright.
... In the following, we assume that an algorithm A addresses (some manifestation of) the diagnosis computation problem (cf. Problem 1) and is given as input a DPI and possibly some meta information (such as component failure rates that allow to derive diagnosis probabilities [13], or algorithm-specific parameters, e.g., stop, pruning or restart criteria [16,1]). 2 We describe each feature by giving a definition of its possible manifestations, a brief explanation of its relevance wrt. algorithm selection for a diagnostic task, a short discussion of the practical impact of different feature manifestations, and a comment on the relationship to other features. ...
... Apart from that, the soundness requirement is in line with the generally accepted principles of parsimony [45] and Occam's razor [4], which postulate that from two different (fault) explanations, the simpler one is preferable. Impact: Forgoing the requirement of soundness can lead to a higher efficiency of diagnosis computation, as certain unsound algorithms are designed to drop soundness to the benefit of performance (e.g., [38,16]). There are basically two forms of unsoundness for returned diagnoses, i.e., they might be (a) non-minimal diagnoses (intuitively: "too large" component sets; cf., e.g., [16]), or (b) non-diagnoses (intuitively: "too small" component sets; cf., e.g., [38]). ...
... Impact: Forgoing the requirement of soundness can lead to a higher efficiency of diagnosis computation, as certain unsound algorithms are designed to drop soundness to the benefit of performance (e.g., [38,16]). There are basically two forms of unsoundness for returned diagnoses, i.e., they might be (a) non-minimal diagnoses (intuitively: "too large" component sets; cf., e.g., [16]), or (b) non-diagnoses (intuitively: "too small" component sets; cf., e.g., [38]). Both cases can be handled by a suitable post-processing of the returned solutions (cf., e.g., [34]), the cost of which depends on the number of solutions that are non-(minimal) diagnoses and on their degree of unsoundness (i.e., how much "too small" or "too large" they are). ...
Model-based diagnosis is a powerful, versatile and well-founded approach to troubleshooting a wealth of different types of systems. Diagnosis algorithms are both numerous and highly heterogeneous. In this work, we propose a taxonomy that allows their standardized assessment, classification and comparison. The aim is to (i) give researchers and practitioners an impression of the diverse landscape of available techniques, (ii) allow them to easily retrieve and compare the main features as well as pros and cons, and (iii) facilitate the selection of the “right” algorithm to adopt for a particular problem case, e.g., in practical diagnostic settings, for comparison in experimental evaluations, or for reuse, modification, extension, or improvement in the course of research. Finally, we demonstrate the value and application of the taxonomy by assessing and categorizing a range of more than 30 important diagnostic methods, and we point out how using the taxonomy as a common guideline for algorithm analysis would benefit the research community in various regards.
... Motivated by different diagnosis scenarios and application fields, these algorithms feature greatly different properties. For instance, while some are designed to guarantee soundness and completeness (i.e., the computation of only and all minimal diagnoses), e.g., to ensure the localization of the actual diagnosis in critical applications (medicine [21], aircrafts [11], etc.), others drop one or both of these properties, e.g., to allow for higher diagnostic efficiency [22,23]. Since the computation of all (minimal) diagnoses is intractable 2 , all diagnosis searches have to focus on a (computationally feasible) subset of the diagnoses in general. ...
... Basically, there are two possibilities how RBF-HS may specify the F -value of a child node n i : either the F -value of the parent n is inherited to the child node, or n i 's (original) f -value is used. In fact, the algorithm first checks whether n has already been explored before, which is true if f (n) > F (n) (line 22). ...
... In the minimum-cardinality case, we can deduce 22 from the findings of [31] that RBF-HS explores O(n) nodes, i.e., for sufficiently large problem size, no more than a constant number as many as HS-Tree does. Intuitively, the plausibility of this can be verified by considering (i) and (ii). ...
Various model-based diagnosis scenarios require the computation of most preferred fault explanations. Existing algorithms that are sound (i.e., output only actual fault explanations) and complete (i.e., can return all explanations), however, require exponential space to achieve this task. As a remedy, we propose two novel diagnostic search algorithms, called RBF-HS (Recursive Best-First Hitting Set Search) and HBF-HS (Hybrid Best-First Hitting Set Search), which build upon tried and tested techniques from the heuristic search domain. RBF-HS can enumerate an arbitrary predefined finite number of fault explanations in best-first order within linear space bounds, without sacrificing the desirable soundness or completeness properties. The idea of HBF-HS is to find a trade-off between runtime optimization and a restricted space consumption that does not exceed the available memory. In extensive experiments on real-world diagnosis cases we compared our approaches to Reiter's HS-Tree, a state-of-the-art method that gives the same theoretical guarantees and is as general(ly applicable) as the suggested algorithms. For the computation of minimum-cardinality fault explanations, we find that (1) RBF-HS reduces memory requirements substantially in most cases by up to several orders of magnitude, (2) in more than a third of the cases, both memory savings and runtime savings are achieved, and (3) given the runtime overhead is significant, using HBF-HS instead of RBF-HS reduces the runtime to values comparable with HS-Tree while keeping the used memory reasonably bounded. When computing most probable fault explanations, we observe that RBF-HS tends to trade memory savings more or less one-to-one for runtime overheads. Again, HBF-HS proves to be a reasonable remedy to cut down the runtime while complying with practicable memory bounds.
... Impact: Forgoing the requirement of soundness can lead to a higher efficiency of diagnosis computation, as certain unsound algorithms are designed to drop soundness to the benefit of performance (e.g., [21; 16]). There are basically two forms of unsoundness for returned diagnoses, i.e., they might be (a) nonminimal diagnoses (intuitively: "too large" component sets; cf., e.g., [16]), or (b) non-diagnoses (intuitively: "too small" component sets; cf., e.g., [21]). Both cases can be handled by a suitable post-processing of the returned solutions (cf., e.g., [22]), the cost of which depends on the number of solutions that are non-(minimal) diagnoses and on their degree of unsoundness (i.e., how much "too small" or "too large" the diagnoses are). ...
... Direct techniques, some of which (e.g., [25; 26]) are based on the Duality Property (cf. Sec. 2), sometimes allow to escape computational bottlenecks concerning memory consumption [26] or time [16] by forgoing a systematic enumeration of the diagnoses. Most of the algorithms in the literature appear to be conflict-dependent (cf. ...
This work proposes a taxonomy for diagnosis computation methods which allows their standardized assessment, classification and comparison. The aim is to (i) give researchers and practitioners an impression of the diverse landscape of available diagnostic techniques, (ii) allow them to easily retrieve the main features as well as pros and cons of the approaches, (iii) enable an easy and clear comparison of the techniques based on their characteristics wrt. a list of important and well-defined properties, and (iv) facilitate the selection of the "right" algorithm to adopt for a particular problem case, e.g., in practical diagnostic settings, for comparison in experimental evaluations, or for reuse, modification, extension, or improvement in the course of research.
... The diagnostic problem occurs when there exists an inconsistency in the model of the system and observations. Since Reiter raised MBD problem and its algorithm, a large number of significant and improved [2][3][4][5][6][7][8][9][10][11][14][15][16][17][18][19][20][21][22][23][24]. These algorithms provide a way to analyze the complex system in various areas, including software fault localization, type error debugging, debugging of relational specifications, the automotive industry, and design debugging, among many others. ...
... For traditional algorithms, system description for diagnosis is formed as propositional logic, exceptionally, there is a novel model Simulated Annealing (SA) [17] which translates MBD into a polynomial minimization problem. For the mainstream algorithms with propositional logic formulation, SAFARI [2,18] gets a solution with mutiple times stochastic search. Although this method reduces the runtime, it cannot guarantee that its solution is a cardinality-minimal diagnosis. ...
Model-based Diagnosis (MBD) with multiple observations is a currently complicated problem with many applications and solving this problem is attracting more and more attention. This paper propose an improved algorithm, called Improved implicit Hitting Set Dualization (IHSD), which is the integration of gate domination in recent works for computing cardinality-minimal aggregated diagnoses in MBD problems. First, our approach works by separating components into dominated components and non-dominated components according to structure of diagnosis system. The separated components are modelled as hard clauses and soft clauses separately. Additionally, two feasible approaches, called IHSDa and IHSDb, are proposed to expand one cardinality-minimal aggregated diagnosis to more diagnoses. Experimental results on 74XXX and ISCAS85 benchmarks clearly show that IHSD algorithm improves HSD, DC and DC*. Moreover, IHSDa and IHSDb outperform HSD on solving more diagnoses.
... Incomplete approaches, in contrast, are usually geared towards computational efficiency, at the cost of not giving a completeness guarantee. Examples of (generally) imcomplete algorithms are Genetic Algorithm (Li & Yunfei, 2002), SAFARI (Feldman et al., 2008), STACCATO (Abreu & van Gemund, 2009), CDA * (Williams & Ragno, 2007), HDiag (Siddiqi, Huang, et al., 2007), and NGDE (de Kleer, 2009). Also, incompleteness might be a consequence of a special focus of the diagnosis search, e.g., if the goal is to determine only (the) minimal cardinality (of) diagnoses (Siddiqi et al., 2007;Shi & Cai, 2010;de Kleer, 2011). ...
... Weak fault models, in constrast to strong fault models, define only the normal behavior of the system components, and do not specify any behavior in case components are at fault(Feldman, Provan, & van Gemund, 2008). The weak fault model is also referred to as "Ignorance of Abnormal Behavior" property(de Kleer, 2008). ...
Given a system that does not work as expected, Sequential Diagnosis (SD) aims at suggesting a series of system measurements to isolate the true explanation for the system's misbehavior from a potentially exponential set of possible explanations. To reason about the best next measurement, SD methods usually require a sample of possible fault explanations at each step of the iterative diagnostic process. The computation of this sample can be accomplished by various diagnostic search algorithms. Among those, Reiter's HS-Tree is one of the most popular due its desirable properties and general applicability. Usually, HS-Tree is used in a stateless fashion throughout the SD process to (re)compute a sample of possible fault explanations in each iteration, each time given the latest (updated) system knowledge including all so-far collected measurements. At this, the built search tree is discarded between two iterations, although often large parts of the tree have to be rebuilt in the next iteration, involving redundant operations and calls to costly reasoning services. As a remedy to this, we propose DynamicHS, a variant of HS-Tree that maintains state throughout the diagnostic session and additionally embraces special strategies to minimize the number of expensive reasoner invocations. In this vein, DynamicHS provides an answer to a longstanding question posed by Raymond Reiter in his seminal paper from 1987. Extensive evaluations on real-world diagnosis problems prove the reasonability of the DynamicHS and testify its clear superiority to HS-Tree wrt. computation time. More specifically, DynamicHS outperformed HS-Tree in 96% of the executed sequential diagnosis sessions and, per run, the latter required up to 800% the time of the former. Remarkably, DynamicHS achieves these performance improvements while preserving all desirable properties as well as the general applicability of HS-Tree.
... Plenty of methods have been studied for deriving MHSs, including the HS-tree [1], HST-tree [10], HS-DAG [11] and its related improvements [12], binary HS-tree (BHS-tree) [13], bipartite graph-based method [14], our HSSE-tree [15], and CSSEtree/CSISE-tree [16]. In addition to these tree-based/graphbased algorithms, many other methods have been studied, such as the Boolean algebra-based approach (abbreviated to "BAMHS" in this article) [13] and its related optimizations [17], a conflict-based A * algorithm [18], and two stochastic search-based approximation approaches, i.e., SAFARI [19] and STACCATO [20]. ...
For many high-tech fields such as space exploration, nuclear technology, and smart automobiles, it is vital to timely find faulty components of man-made devices to ensure safety. However, there is nearly
no
enough diagnostic experience accumulated in these new devices, and thus, it is hardly suitable to only apply the traditional expert/experience-based fault diagnosis approach. Thus, model-based diagnosis was proposed for efficient detection of faulty components; this approach explores the behavioral and structural information of the device to be diagnosed, and no experience is required. In model-based diagnosis, for a device to be diagnosed, minimal conflict sets of components are first generated, and all minimal hitting-sets for them will be derived as candidate diagnoses. Therefore, it is vital to efficiently generate all minimal hitting-sets to find the final diagnosis. Unfortunately, it is proven to be NP-hard when deriving all minimal hitting-sets for given minimal conflict sets. To improve the computing efficiency, in this article, we propose a novel approach called
TreeMerge
, which considers a special type of
tree
structure of minimal conflict sets of large sizes since structural information usually plays an important role in solving complex problems. Theoretically, compared with other algorithms, the time complexity of the new algorithm is greatly reduced, as the time complexity of the new algorithm becomes
linear
rather than
quadratic
. Furthermore, experimental results on multiple synthetic and benchmark examples show that the proposed
TreeMerge
algorithm is more efficient than many other state-of-the-art methods, with a reduction of
several orders of magnitude
runtime (seconds).
... It is important to mention that especially when calculating the first n-diagnoses (for n > 1, i.e., not a single diagnosis), FASTDIAG can also exploit the mentioned algorithms of [20,29] for the calculation of more than one diagnosis, i.e., it is not bound to the usage of the original HSDAG algorithm. Lin et al. [19] introduce an approach to determine hitting sets on the basis of genetic algorithms; a similar approach to the determination of diagnoses is presented in [8] who introduce a stochastic fault diagnosis algorithm which is based on greedy stochastic search. Such approaches show to significantly improve search performance, however, there is no general guarantee of completeness and diagnosis minimality. ...
Constraint sets can become inconsistent in different contexts. For example, during a configuration session the set of customer requirements can become inconsistent with the configuration knowledge base. Another example is the engineering phase of a configuration knowledge base where the underlying constraints can become inconsistent with a set of test cases. In such situations we are in the need of techniques that support the identification of minimal sets of faulty constraints that have to be deleted in order to restore consistency. In this paper we introduce a divide-and-conquer based diagnosis algorithm (FastDiag) which identifies minimal sets of faulty constraints in an over-constrained problem. This algorithm is specifically applicable in scenarios where the efficient identification of leading (preferred) diagnoses is crucial. We compare the performance of FastDiag with the conflict-directed calculation of hitting sets and present an in-depth performance analysis that shows the advantages of our approach.
Given a system that does not work as expected, sequential diagnosis aims at suggesting a series of system measurements to isolate the true explanation for the system’s misbehavior from a potentially large set of possible explanations. To reason about the best next measurement, sequential diagnosis methods usually require a sample of possible fault explanations at each step of the iterative diagnostic process. The computation of this sample can be accomplished by various diagnostic search algorithms. Among those, Reiter’s HS-Tree is one of the most popular due to its desirable properties and general applicability. Usually, HS-Tree is used in a stateless fashion throughout the diagnosis process to (re) compute a sample of possible fault explanations per iteration, each time given the latest (updated) system knowledge including all so-far collected measurements. At this, the built search tree is discarded between two iterations, albeit often large parts of the tree have to be rebuilt in the next iteration, involving redundant operations and calls to costly reasoning services.
As a remedy to this, we propose DynamicHS, a variant of HS-Tree that maintains state throughout the diagnostic session and embraces special strategies to minimize the number of expensive reasoner invocations. DynamicHS provides an answer to a longstanding question posed by Raymond Reiter in his seminal paper from 1987, where he wondered if there is a reasonable strategy to reuse an existing search tree to compute fault explanations after new system information is obtained.
We conducted extensive evaluations on real-world diagnosis problems from the domain of knowledge-based systems—a field where the usage of HS-Tree is state-of-the-art—under various diagnosis scenarios in terms of the number of fault explanations computed and the heuristic for measurement selection used. The results prove the reasonability of the novel approach and testify its clear superiority to HS-Tree wrt. computation time. More specifically: (1) DynamicHS required less time than HS-Tree in 96 % of the executed sequential diagnosis sessions. (2) DynamicHS exhibited substantial and statistically significant time savings over HS-Tree in most scenarios, with median and maximal savings of 52 % and 75 %, respectively. (3) The relative amount of saved time appears to neither depend on the number of computed fault explanations nor on the used measurement selection heuristic. (4) In the hardest (most time-intensive) cases per diagnosis scenario, DynamicHS achieved even higher savings than on average, and could avoid median and maximal time overheads of over 175 % and 800 %, respectively, as opposed to a usage of HS-Tree.
Remarkably, DynamicHS achieves these performance improvements while preserving all desirable properties as well as the general applicability of HS-Tree.
Various model-based diagnosis scenarios require the computation of the most preferred fault explanations. Existing algorithms that are sound (i.e., output only actual fault explanations) and complete (i.e., can return all explanations), however, require exponential space to achieve this task. As a remedy, and to enable successful diagnosis both on memory-restricted devices and for memory-intensive problem cases, we propose two novel diagnostic search algorithms which build upon tried and tested techniques from the heuristic search domain. The first method, dubbed Recursive Best-First Hitting Set Search (RBF-HS), is based on Korf's well-known Recursive Best-First Search (RBFS) algorithm. We show that RBF-HS can enumerate an arbitrary predefined finite number of fault explanations in best-first order within linear space bounds, without sacrificing the desirable soundness or completeness properties. The second algorithm, called Hybrid Best-First Hitting Set Search (HBF-HS), is a hybrid between RBF-HS and Reiter's seminal HS-Tree. The idea is to find a trade-off between runtime optimization and a restricted space consumption that does not exceed the available memory. Notably, both suggested algorithms are generally applicable to any model-based diagnosis problem, regardless of the used (monotonic) logical language to describe the diagnosed system and of the used reasoning mechanism.
We conducted extensive experiments on real-world benchmarks from the knowledge-based systems field, a domain where the features soundness, completeness, the best-first property as well as a general applicability are pivotal and where Reiter's HS-Tree is the predominantly used diagnostic search. The evaluation reveals that, when computing fault explanations minimal-cardinality-first, RBF-HS compared to HS-Tree reduces memory requirements substantially in most cases by up to several orders of magnitude, while also saving runtime in more than a third of the cases. When computing fault explanations most-probable-first, RBF-HS compared to HS-Tree tends to trade memory savings more or less one-to-one for runtime overheads. Whenever runtime overheads were significant, using HBF-HS instead of RBF-HS reduced the runtime to values comparable with HS-Tree while keeping the used memory reasonably bounded.
This is not a technical paper, but a reference to the magnetic tape containing “ISCAS’85 combinational benchmark circuits”. The tape was distributed to the authors who contributed to the special session of ISCAS’85. It was requested to refer the tape in any publication as follows.
Franc Brglez and Hideo Fujiwara, "A neutral netlist of 10 combinational circuits and a targeted translator in FORTRAN," Special Session on Recent Algorithms for Gate-Level ATPG with Fault Simulation and Their Performance Assessment, 1985 IEEE Int. Symp. on Circuits and Systems, June 5-7, 1985, Kyoto, Japan.
For many large systems the computational complexity of complete model-based diagnosis is prohibitive. In this paper we investigate the speedup of the diagnosis process by ex- ploiting the hierarchy/locality as is typically present in well- engineered systems. The approach comprises a compile-time and a run-timestep. In thefirststep, a hierarchicalCNFrepre- sentation of the system iscompiled tohierarchical DNFof ad- justable hierarchical depth. In the second step, the diagnoses are computed from the hierarchical DNFand the actual obser- vations. Our hierarchical algorithm, while sound and com- plete, allows large models to be diagnosed, where compile- time investment directly translates to run-time speedup. The benefits of our approach are illustrated by using weak-fault models of real-world systems, including the ISCAS-85 com- binatorial circuits. Even for these non-optimally partitioned problems the speedup compared to traditional approaches ranges in the hundreds.
Artificial Intelligence has traditionally used constraint satisfaction and logic to frame a wide range of problems, including planning, diagnosis, cognitive robotics and embedded systems control. However, many decision making problems are now being re-framed as optimization problems, involving a search over a discrete space for the best solution that satisfies a set of constraints. The best methods for finding optimal solutions, such as A * , explore the space of solutions one state at a time. This paper introduces conflict-directed A * , a method for solving optimal constraint satisfaction problems. Conflict-directed A * searches the state space in best first order, but accelerates the search process by eliminating subspaces around each state that are inconsistent. This elimination process builds upon the concepts of conflict and kernel diagnosis used in model-based diagnosis. Conflict-directed A * is a fundamental tool for building model-based embedded systems, and has been used to solve a range of problems, including fault isolation, diagnosis, mode estimation and repair, model-compilation and model-based programming.
Diagnostic tasks require determining the differences between a model of an artifact and the artifact itself. The differences between the manifested behavior of the artifact and the predicted behavior of the model guide the search for the differences between the artifact and its model. The diagnostic procedure presented in this paper is model-based, inferring the behavior of the composite device from knowledge of the structure and function of the individual components comprising the device. The system (GDE—general diagnostic engine) has been implemented and tested on many examples in the domain of troubleshooting digital circuits.This research makes several novel contributions: First, the system diagnoses failures due to multiple faults. Second, failure candidates are represented and manipulated in terms of minimal sets of violated assumptions, resulting in an efficient diagnostic procedure. Third, the diagnostic procedure is incremental, exploiting the iterative nature of diagnosis. Fourth, a clear separation is drawn between diagnosis and behavior prediction, resulting in a domain (and inference procedure) independent diagnostic procedure. Fifth, GDE combines model-based prediction with sequential diagnosis to propose measurements to localize the faults. The normally required conditional probabilities are computed from the structure of the device and models of its components. This capability results from a novel way of incorporating probabilities and information theory into the context mechanism provided by assumption-based truth maintenance.
The problem of abduction can be characterized as finding the best explanation of a set of data. In this paper we focus on one type of abduction in which the best explanation is the most plausible combination of hypotheses that explains all the data. We then present several computational complexity results demonstrating that this type of abduction is intractable (NP-hard) in general. In particular, choosing between incompatible hypotheses, reasoning about cancellation effects among hypotheses, and satisfying the maximum plausibility requirement are major factors leading to intractability. We also identify a tractable, but restricted, class of abduction problems.
Suppose one is given a description of a system, together with an observation of the system's behaviour which conflicts with the way the system is meant to behave. The diagnostic problem is to determine those components of the system which, when assumed to be functioning abnormally, will explain the discrepancy between the observed and correct system behaviour.We propose a general theory for this problem. The theory requires only that the system be described in a suitable logic. Moreover, there are many such suitable logics, e.g. first-order, temporal, dynamic, etc. As a result, the theory accommodates diagnostic reasoning in a wide variety of practical settings, including digital and analogue circuits, medicine, and database updates. The theory leads to an algorithm for computing all diagnoses, and to various results concerning principles of measurement for discriminating among competing diagnoses. Finally, the theory reveals close connections between diagnostic reasoning and nonmonotonic reasoning.
Most approaches to model-based diagnosis describe a diagnosis for a system as a set of failing components that explains the symptoms. In order to characterize the typically very large number of diagnoses, usually only the minimal such sets of failing components are represented. This method of characterizing all diagnoses is inadequate in general, in part because not every superset of the faulty components of a diagnosis necessarily provides a diagnosis. In this paper we analyze the concept of diagnosis in depth exploiting the notions of implicate/implicant and prime implicate/implicant. We use these notions to consider two alternative approaches for addressing the inadequacy of the concept of minimal diagnosis. First, we propose a new concept, that of kernel diagnosis, which is free of this problem with minimal diagnosis. This concept is useful to both the consistency and abductive views of diagnosis. Second, we consider restricting the axioms used to describe the system to ensure that the concept of minimal diagnosis is adequate.