Figure - available from: Machine Learning
This content is subject to copyright. Terms and conditions apply.
Source publication
In recent research, human-understandable explanations of machine learning models have received a lot of attention. Often explanations are given in form of model simplifications or visualizations. However, as shown in cognitive science as well as in early AI research, concept understanding can also be improved by the alignment of a given instance fo...
Citations
... For a new samplex ∈X that we want to generate a counterfactual for, we derive the model prediction and feature space encoding κ(x) and compare it to the encodings of the reference dataset. Hereby, based on the feature space encodings, the closest reference point with a differing class prediction is extracted, resembling the near miss approach [26]. The counterfactual target y c is then defined as the predicted class of the reference point x ′ . ...
... It's prediction is chosen as counterfactual target. The approach is related to the concept of near misses [26]. Table 1 shows the influence of the target selection on the generated samples' quantitative performance metrics. ...
Recent advancements in generative AI have introduced novel prospects and practical implementations. Especially diffusion models show their strength in generating diverse and, at the same time, realistic features, positioning them well for generating counterfactual explanations for computer vision models. Answering "what if" questions of what needs to change to make an image classifier change its prediction, counterfactual explanations align well with human understanding and consequently help in making model behavior more comprehensible. Current methods succeed in generating authentic counterfactuals, but lack transparency as feature changes are not directly perceivable. To address this limitation, we introduce Concept-guided Latent Diffusion Counterfactual Explanations (CoLa-DCE). CoLa-DCE generates concept-guided counterfactuals for any classifier with a high degree of control regarding concept selection and spatial conditioning. The counterfactuals comprise an increased granularity through minimal feature changes. The reference feature visualization ensures better comprehensibility, while the feature localization provides increased transparency of "where" changed "what". We demonstrate the advantages of our approach in minimality and comprehensibility across multiple image classification models and datasets and provide insights into how our CoLa-DCE explanations help comprehend model errors like misclassification cases.
... Specially selected examples can (1) show which data a model evaluates to be particularly typical for a certain class-a prototypical representant with respect to the decision region the model induced for this class; instead of identifying a prototypical example, a synthetic representant for a concept might be constructed, as it is usually proposed in psychology and philosophy; (2) examples which are situated near to the decision boundary for a class help to get insights in the discriminative features of the model. This can be a borderline case for the considered class or a near-miss example (Rabold et al., 2022), that is, an example similar to objects of the considered class but being classified as a member of a different class (see Fig. 1). Fig. 1 Explaining image classifications by example. ...
With the digital transformation, artificial intelligence (AI) applications are also finding their way into more and more areas of work and life. In particular, models learned from data are being used, which are mostly opaque black boxes. The fact that people can understand why an AI system behaves the way it does is necessary for various reasons: The model developers themselves must be able to assess properties of the learned models—in particular, possible biases due to overfitting to the data used for learning. For safety-critical applications, aspects of certification and testing are also becoming increasingly relevant. Domain experts—for example, in medical diagnostics or quality control in industrial production—must be able to comprehend, verify and, if necessary, correct system decisions. Consumers should understand why a system—a smart home control, a driving assistance—behaves in a certain way and why they are recommended certain products, offered certain tariffs or denied certain offers. After a brief introduction to the topic of AI, the chapter gives an overview of methods of the so-called third wave of AI. Central to this are approaches of explainable AI (XAI), which are intended to make the decisions of AI systems comprehensible. The main approaches are characterized and shown for which objectives and applications they are suitable in each case. It is shown that in addition to the highly regarded methods for visualization, methods that allow system decisions to be described in a differentiated manner are also particularly important. It is also argued that, in addition to comprehensibility, interactivity and correctability of AI systems are necessary so that AI systems do not restrict human competences but support them in partnership.
... Our approach can be extended to RL, using rules learned from normal drone executions to guide optimal policy search and early identifying anomalous regions of the state-action space. Moreover, ILP can be used offline to generate explanations for registered anomalous behaviors and provide useful contrastive explanations [29], as well as to discover high-level safety specifications implicitly embedded in the nominal behavior. ...
This paper discusses the challenges of applying reinforcement techniques to real-world environmental monitoring problems and proposes innovative solutions to overcome them. In particular, we focus on safety, a fundamental problem in RL that arises when it is applied to domains involving humans or hazardous uncertain situations. We propose to use deep neural networks, formal verification, and online refinement of domain knowledge to improve the transparency and efficiency of the learning process, as well as the quality of the final policies. We present two case studies, specifically (i) autonomous water monitoring and (ii) smart control of air quality indoors. In particular, we discuss the challenges and solutions to these problems, addressing crucial issues such as anomaly detection and prevention, real-time control, and online learning. We believe that the proposed techniques can be used to overcome some limitations of RL, providing safe and efficient solutions to complex and urgent problems.
... In a relational context, an observation o is described as a Herbrand interpretation of some datalog language (see Section 3) that we represent as a conjunction of literals. The classifier D is as well a first order formula of this language and an explanation for assigning the class label c to an observation may be defined as a ground clause [19] whose head is the label or, as we do hereunder, as the body of such a clause, i.e. a ground relational motif. However when turning to an explanation common to a group of observations, we rather consider general relational motifs, following the definition of abdictive explanations in first order logics first proposed by P. Marquis [13] and then further investigated from an operational point of view [12,8]. ...
... One of our originality is to search for shared explanations by observations of a given subgroup sharing the same label. [19] addresses the problem of finding contrastive explanations for an observed instance as minimal changes to an instance so that its class shifts. The authors rely on the notion of near-miss example introduced by Winston (a negative example closest to a positive one) for defining a near miss explanation. ...
We propose a definition of common explanation for the label shared by a group of observations described as first order interpretations, and provide algorithms to enumerate minimal common explanations. This was motivated by explaining how performing some action, for instance a card played during a card game play, results in winning a maximum total reward at the end of the trajectory. As there are various ways to reach this reward, each associated to a group of trajectories, we propose to first build groups of trajectories and then build minimal common explanations for each group. The whole method is illustrated on a simplified Bridge game.
... This involves the selection of appropriate contrastive examples for a concept (near misses) that differ only in small significant characteristics from a target example. Near misses research follows mainly two directions [6]: near miss generation [31,17,16] and near miss selection [19]. While methods that generate near misses, usually do this by adapting the target example, near miss selection strategies search the input space in order to find contrastive examples that are most similar to the target example. ...
... This approach can be applied in domains, where modification operators can be easily defined, such as functions over physical properties. Another approach to generating near miss explanations was published lately by Rabold et al. [31]. Their method explains a target example by modifying the rule(s) that apply to it taken from a set of rules learned with Inductive Logic Programming. ...
Explaining concepts by contrasting examples is an efficient and convenient way of giving insights into the reasons behind a classification decision. This is of particular interest in decision-critical domains, such as medical diagnostics. One particular challenging use case is to distinguish facial expressions of pain and other states, such as disgust, due to high similarity of manifestation. In this paper, we present an approach for generating contrastive explanations to explain facial expressions of pain and disgust shown in video sequences. We implement and compare two approaches for contrastive explanation generation. The first approach explains a specific pain instance in contrast to the most similar disgust instance(s) based on the occurrence of facial expressions (attributes). The second approach takes into account which temporal relations hold between intervals of facial expressions within a sequence (relations). The input to our explanation generation approach is the output of an interpretable rule-based classifier for pain and disgust.We utilize two different similarity metrics to determine near misses and far misses as contrasting instances. Our results show that near miss explanations are shorter than far miss explanations, independent from the applied similarity metric. The outcome of our evaluation indicates that pain and disgust can be distinguished with the help of temporal relations. We currently plan experiments to evaluate how the explanations help in teaching concepts and how they could be enhanced by further modalities and interaction.
... Explaining by a counterfactual [89] or a contrastive example [22] helps to point out what is missing from a specific instance such that it would be classified differently. For structural data, near miss explanations can be constructed by identifying the minimal change in a rule resulting in a different class [66]. This principle has been introduced as alignment based reasoning in cognitive science and has been shown to be highly helpful to understand what the relevant aspects of a concept are [28]. ...
With the perspective on applications of AI-technology, especially data intensive deep learning approaches, the need for methods to control and understand such models has been recognized and gave rise to a new research domain labeled explainable artificial intelligence (XAI). In this overview paper we give an interim appraisal of what has been achieved so far and where there are still gaps in the research. We take an interdisciplinary perspective to identify challenges on XAI research and point to open questions with respect to the quality of the explanations regarding faithfulness and consistency of explanations. On the other hand we see a need regarding the interaction between XAI and user to allow for adaptability to specific information needs and explanatory dialog for informed decision making as well as the possibility to correct models and explanations by interaction. This endeavor requires an integrated interdisciplinary perspective and rigorous approaches to empirical evaluation based on psychological, linguistic and even sociological theories.
... The research approach considered a "bottom-up" approach that assists to understand the observations, patterns and drawing a conclusion [8]. It used to enhance the large project to invalidate the study conclusion. ...
The research study discusses the relevance of “time series analysis” applications that will help an organisation to understand business growth. The present study affects an individual’s knowledge that ensures the research's relevance. The present study focuses on several business metrics that have used to discuss the effectiveness of time series. The researcher has also focused on an appropriate methodology that increases knowledge of the research study by adopting a research design, type, approach, and others. The study chooses a secondary qualitative data analysis method that affected the entire research study. Accessing through time series of analytical approach processes manage to process through an effective decision-making range is essential to provide effective decisions and its courses through the systematic process of review. The benefits of time series analysis personal concepts of series resources and its importance through the decision-making range are essential. A systematic approach in businesses access through proposed factors in business beneficial range and attribution of specific concepts helps to organize this aspect. A systematic source of decision-making range provides optimization of proposed business provides concepts of business making process. Accessing through the kinetic model summary has been provided in terms of beneficial business courses.
... Furthermore, the post-hoc interpretability approach has been used to prevent model bias [Ho21]. The interpretability methods show the features that influence the decision of the model the most and help the user understand how the model is deciding to detect hate speech in dialogues [RSS21]. The presented work provides insights into the decision points and feature importance used to make predictions about the hate speech disposition of conversations. ...
... The tweaked data points are weighed as a function of their proximity to the original data points, then fitting a surrogate model such as linear regression on the dataset with variations using those sample weights. Each original data point can then be explained with the newly trained explanation model [RSS21]. The learned model generates a local prediction model while it may or may not provide a precise global approximation. ...
There have been remarkable breakthroughs in Machine Learning (ML) and Artificial Intelligence (AI), notably in the areas of Natural Language Processing (NLP) and Deep Learning. Additionally, hate speech detection in dialogues has been gaining popularity among Natural Language Processing researchers with the increased use of social media. However, as evidenced by the recent trends, the need for the dimensions of explainability and interpretability in AI models has been deeply realised. Taking note of the factors above, the research goal of this paper is to bridge the gap between hate speech prediction and the explanations generated by the system to support its decision. This has been achieved by first predicting the classification of a text and then providing a post-hoc, model-agnostic and surrogate interpretability approach for explainability and to prevent model bias. The bidirectional transformer model BERT has been used for prediction because of its state-of-the-art efficiency over other Machine Learning(ML) models. The model-agnostic algorithm LIME generates explanations for the output of a trained classifier and predicts the features that influence the model's decision. The predictions generated from the model were evaluated manually, and after thorough evaluation, we observed that the model performs efficiently in predicting and explaining its prediction. Lastly, we suggest further directions for the expansion of the provided research work.
... Both performance and general usefulness of the system can be improved by including more background knowledge, possibly on the basis of more detailed ontologies. Additionally, the explanatory component can be extended by and combined with various other approaches, for example contrastive and other example based explanations (Rabold, Siebers, and Schmid 2021). Finally, we plan to evaluate our approach in terms of a user study with respect to applicability and usefulness of explanations in the manufacturing domain. ...
Machine learning based image classification algorithms, such as deep neural network approaches, will be increasingly employed in critical settings such as quality control in industry, where transparency and comprehensibility of decisions are crucial. Therefore, we aim to extend the defect detection task towards an interactive human-in-the-loop approach that allows us to integrate rich background knowledge and the inference of complex relationships going beyond traditional purely data-driven approaches. We propose an approach for an interactive support system for classifications in an industrial quality control setting that combines the advantages of both (explainable) knowledge-driven and data-driven machine learning methods, in particular inductive logic programming and convolutional neural networks, with human expertise and control. The resulting system can assist domain experts with decisions, provide transparent explanations for results, and integrate feedback from users; thus reducing workload for humans while both respecting their expertise and without removing their agency or accountability.
... Both performance and general usefulness of the system can be improved by including more background knowledge, possibly on the basis of more detailed ontologies. Additionally, the explanatory component can be extended by and combined with various other approaches, for example contrastive and other example based explanations (Rabold, Siebers, and Schmid 2021). Finally, we plan to evaluate our approach in terms of a user study with respect to applicability and usefulness of explanations in the manufacturing domain. ...
Machine learning based image classification algorithms, such as deep neural network approaches, will be increasingly employed in critical settings such as quality control in industry, where transparency and comprehensibility of decisions are crucial. Therefore, we aim to extend the defect detection task towards an interactive human-in-the-loop approach that allows us to integrate rich background knowledge and the inference of complex relationships going beyond traditional purely data-driven approaches. We propose an approach for an interactive support system for classifications in an industrial quality control setting that combines the advantages of both (explainable) knowledge-driven and data-driven machine learning methods, in particular inductive logic programming and convolutional neural networks, with human expertise and control. The resulting system can assist domain experts with decisions, provide transparent explanations for results, and integrate feedback from users; thus reducing workload for humans while both respecting their expertise and without removing their agency or accountability.
