FIGURE 1 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Source publication
Building models that are both interpretable and accurate is an unresolved challenge for many pattern recognition problems. In general, rule-based and linear models lack accuracy, while deep learning interpretability is based on rough approximations of the underlying inference. However, recently, the rule-based Tsetlin Machines (TMs) have obtained c...
Context in source publication
Context 1
... time per epoch for the TM approaches is also small, amounting to 0.001 seconds, which is the lowest of all the algorithms. The TMs also maintain better F1-Scores across all training data sizes in comparison with the other techniques, as seen from the sample complexity analysis in FIGURE 10. ...
Similar publications
This study aims to determine the effect of transformative leadership, and locus of control on fraud detection with environmental performance as a moderating variable. This research was conducted by the government agency XYZ with a sample of 40 respondents. The data comes from a questionnaire that is filled in by the respondent. The results of this...
![The principals of PCA and KPCA [6].](publication/360185012/figure/fig4/AS:1155213058687011@1652435664383/The-principals-of-PCA-and-KPCA-6_Q320.jpg)
Credit card fraud is a major problem in today’s financial world. It induces severe damage to financial institutions and individuals. There has been an exponential increase in the losses due to fraud in recent years. Hence, effectively detecting fraudulent behavior is of vital importance for either financial institutions or individuals. Since credit...
This white paper is the result of a research project by Hogeschool Utrecht, Copenhagen Business School and the Dutch Association of Insurers in the period
February-June 2023.
The goal of the research project was to provide an overview of the practical implementation of artificial intelligence (AI) in fraud detection of non-life insurance claims in...
The rise of e-commerce payment systems has been swift due to our society's rapid advancement. However, the transparency and vulnerability of the internet have opened new avenues for illicit access, resulting in a significant surge in financial fraud cases, particularly credit card transaction fraud. Detecting and mitigating such incidents has becom...
This paper introduces a novel approach to representing continuous inputs in Tsetlin Machines (TMs). Instead of using one Tsetlin Automaton (TA) for every unique threshold found when Booleanizing continuous input, we employ two Stochastic Searching on the Line (SSL) automata to learn discriminative lower and upper bounds. The two resulting Boolean f...
Citations
... However, it is important to consider that there is a trade-off between explainability and performance. [34][35][36][37] Our study made three pathologists determine the initial diagnosis of 1,031 image patches, revise their diagnoses based on the AI's diagnosis, and then compare their results. After noting that some of their diagnoses were different from AI readings, the pathologists reviewed and revised at least one of their misdiagnoses. ...
... Clause integer weights, as described in [3,24], are not utilized in this solution due to the small image size. Thus, an alternative implementation could have been to employ a simple popcount solution - without pipelining -to find the class sums, i.e., the numbers of odd and even clauses that evaluate to 1, see Eq. (11) in Appendix. ...
... For any input X, the probability of updating a clause gradually drops to zero as the TM output sum in Eq. (11) approaches a user-configured target/hyperparameter T . A higher T increases the robustness of learning by allocating more clauses to learn each sub-pattern [24]. Optimum settings of m, T and s are dependent on the specific ML problem. ...
... The Thresholding TM booleanizes the data using Adaptive Gaussian Thresholding 1 per color channel. We here use 8 000 weighted clauses per class [3], a voting margin T = 2000, specificity s = 10.0, and a 10 × 10 convolution window (see [8] for an explanation of the hyperparameters). Along the x-axis, we rank the 10 000 test images of CIFAR-100 from lowest to highest max class sum, c max in Eqn. 4. The y-axis shows accuracy on the test images from the x-axis confidence level and upwards. ...
Tsetlin Machines (TMs) provide a fundamental shift from arithmetic-based to logic-based machine learning. Supporting convolution, they deal successfully with image classification datasets like MNIST, Fashion-MNIST, and CIFAR-2. However, the TM struggles with getting state-of-the-art performance on CIFAR-10 and CIFAR-100, representing more complex tasks. This paper introduces plug-and-play collaboration between specialized TMs, referred to as TM Composites. The collaboration relies on a TM's ability to specialize during learning and to assess its competence during inference. When teaming up, the most confident TMs make the decisions, relieving the uncertain ones. In this manner, a TM Composite becomes more competent than its members, benefiting from their specializations. The collaboration is plug-and-play in that members can be combined in any way, at any time, without fine-tuning. We implement three TM specializations in our empirical evaluation: Histogram of Gradients, Adaptive Gaussian Thresholding, and Color Thermometers. The resulting TM Composite increases accuracy on Fashion-MNIST by two percentage points, CIFAR-10 by twelve points, and CIFAR-100 by nine points, yielding new state-of-the-art results for TMs. Overall, we envision that TM Composites will enable an ultra-low energy and transparent alternative to state-of-the-art deep learning on more tasks and datasets.
... Computing [27], [28], [29], [30], [31], [32], [33], [34], [ Q2: What is the goal proposed by the author(s) for the work? ...
... This analysis showed that works usually explores the already existing algorithms, their fusion to a hybrid approach or the proposition of new algorithms (Six works make propositions like these [27], [28], [29], [14], [35], and [30]). The main goal is to understand what is behind the intelligent algorithms usually presented as a black box. ...
... Others [27], [6], [8], [10], [30] 5 There are cases where the combination of techniques for model comprehension can be observed to improve the results of the approach. Therefore, some papers appear more than once, which justifies the absence of a percentage calculation in this table. ...
In a technological world, in which data is generated exponentially, financial analysis has gradually become more important to avoid large losses due to fraud. Considering the large volume and the difficulty of human data checking, machine learning technologies have become one of the main tools to solve the problem. However, due to the creation of data protection laws in several countries, in some scenarios the detection of fraud through intelligence algorithms becomes insufficient. Therefore, it is necessary to understand how the algorithm actually labels a transaction as fraudulent or not. In this work, presented as a systematic literature review, we look for answers on how explicable/interpretable fraud detection algorithms have been applied in order to solve the problem of illegal activities in the financial sector. As a result of the mapping of the current state of the art, this work highlights the gaps in the literature and present the scenario of interpretable techniques used for fraud detection comprehension.
... Tsetlin Machines (TsMs) [6] have recently demonstrated competitive accuracy, learning speed, low memory, and low energy footprint on several tasks, including tasks related to image classification [5], [14], natural language classification [3], [13], [20], [21], speech processing [10], spanning tabular data [1], [18], and regression tasks. TsMs are less prone to overfitting, as the training algorithm does not rely on minimising an error function. ...
... We also perform some tests to compare TsMs and BNNs. To perform our experiments, we wrote the code in Python 3.8 and we used the Glucose SAT solver [2], which is the same SAT solver employed by Narodytska et al. in her mentioned work 1 . We now describe the datasets used for training in more detail. ...
Tsetlin Machines (TsMs) are a promising and interpretable machine learning method which can be applied for various classification tasks. We present an exact encoding of TsMs into propositional logic and formally verify properties of TsMs using a SAT solver. In particular, we introduce in this work a notion of similarity of machine learning models and apply our notion to check for similarity of TsMs. We also consider notions of robustness and equivalence from the literature and adapt them for TsMs. Then, we show the correctness of our encoding and provide results for the properties: adversarial robustness, equivalence, and similarity of TsMs. In our experiments, we employ the MNIST and IMDB datasets for (respectively) image and sentiment classification. We discuss the results for verifying robustness obtained with TsMs with those in the literature obtained with Binarized Neural Networks on MNIST.
... As an alternative to explaining black-box models, pattern recognition approaches that are inherently interpretable attract an increasing number of researchers. The TM is one such approach, distinguished by combining interpretability with competitive accuracy [3,9,27]. Yet, work on interpretable approaches to reinforcement learning is scarce, with the TM being fundamentally supervised. ...
... Type I feedback is given to clauses with positive polarity when y = 1 and to clauses with negative polarity when -Include is rewarded and exclude is penalized with probability s−1 s whenever c j = 1 and l k = 1. This reinforcement is strong (triggered with high probability) and makes the clause remember and refine the pattern it recognizes in X. 3 -Include is penalized and exclude is rewarded with probability 1 s whenever c j = 0 or l k = 0. This reinforcement is weak (triggered with low probability) and coarsens infrequent patterns, making them frequent. ...
... Just like for the TM, the input to an RTM is a vector X of o propositional features x k , X ∈ {0, 1} o . Again, the features are expanded with their negationsx k = 1 − x k producing a 3 Note that the probability s−1 s is replaced by 1 when boosting true positives. literal vector: L = (x 1 , . . . ...
The Tsetlin Machine is a recent supervised learning algorithm that has obtained competitive accuracy- and resource usage results across several benchmarks. It has been used for convolution, classification, and regression, producing interpretable rules in propositional logic. In this paper, we introduce the first framework for reinforcement learning based on the Tsetlin Machine. Our framework integrates the value iteration algorithm with the regression Tsetlin Machine as the value function approximator. To obtain accurate off-policy state-value estimation, we propose a modified Tsetlin Machine feedback mechanism that adapts to the dynamic nature of value iteration. In particular, we show that the Tsetlin Machine is able to unlearn and recover from the misleading experiences that often occur at the beginning of training. A key challenge that we address is mapping the intrinsically continuous nature of state-value learning to the propositional Tsetlin Machine architecture, leveraging probabilistic updates. While accurate off-policy, this mechanism learns significantly slower than neural networks on-policy. However, by introducing multi-step temporal-difference learning in combination with high-frequency propositional logic patterns, we are able to close the performance gap. Several gridworld instances document that our framework can outperform comparable neural network models, despite being based on simple one-level AND-rules in propositional logic. Finally, we propose how the class of models learnt by our Tsetlin Machine for the gridworld problem can be translated into a more understandable graph structure. The graph structure captures the state-value function approximation and the corresponding policy found by the Tsetlin Machine.
... There is a need to have a more formalized definition of interpretability in the context of DNNs [51]. An interpretable model can obtain transparency by providing the reasons behind their output [52]. ...
... There is a trade-off between the accuracy and interpretability of the models, as shown in Fig. 2, where non-linear models with better performance, such as DNNs are less explainable, and linear models which are least accurate, such as decision trees, are more explainable [2,21,45]. It remains a challenge to build models that are both accurate and interpretable [52]. This trade-off can limit the usability of the more accurate models. ...
Artificial Intelligence (AI) techniques of deep learning have revolutionized the disease diagnosis with their outstanding image classification performance. In spite of the outstanding results, the widespread adoption of these techniques in clinical practice is still taking place at a moderate pace. One of the major hindrance is that a trained Deep Neural Networks (DNN) model provides a prediction, but questions about why and how that prediction was made remain unanswered. This linkage is of utmost importance for the regulated healthcare domain to increase the trust in the automated diagnosis system by the practitioners, patients and other stakeholders. The application of deep learning for medical imaging has to be interpreted with caution due to the health and safety concerns similar to blame attribution in the case of an accident involving autonomous cars. The consequences of both a false positive and false negative cases are far reaching for patients' welfare and cannot be ignored. This is exacerbated by the fact that the state-of-the-art deep learning algorithms comprise of complex interconnected structures, millions of parameters, and a 'black box' nature, offering little understanding of their inner working unlike the traditional machine learning algorithms. Explainable AI (XAI) techniques help to understand model predictions which help develop trust in the system, accelerate the disease diagnosis, and meet adherence to regulatory requirements. This survey provides a comprehensive review of the promising field of XAI for biomedical imaging diagnostics. We also provide a categorization of the XAI techniques, discuss the open challenges, and provide future directions for XAI which would be of interest to clinicians, regulators and model developers.
... We can calculate which clauses are described as the most prevalent when ranking each class/item. A potential approach to this is presented with the introduction of the integer weighted TM by [2]. We can then perform relatively cheap elimination of items that do not fulfill the highest weighted clauses for the given input. ...
Recommendation Systems (RSs) are ubiquitous in modern society and are one of the largest points of interaction between humans and AI. Modern RSs are often implemented using deep learning models, which are infamously difficult to interpret. This problem is particularly exasperated in the context of recommendation scenarios, as it erodes the user's trust in the RS. In contrast, the newly introduced Tsetlin Machines (TM) possess some valuable properties due to their inherent interpretability. TMs are still fairly young as a technology. As no RS has been developed for TMs before, it has become necessary to perform some preliminary research regarding the practicality of such a system. In this paper, we develop the first RS based on TMs to evaluate its practicality in this application domain. This paper compares the viability of TMs with other machine learning models prevalent in the field of RS. We train and investigate the performance of the TM compared with a vanilla feed-forward deep learning model. These comparisons are based on model performance, interpretability/explainability, and scalability. Further, we provide some benchmark performance comparisons to similar machine learning solutions relevant to RSs.
... We can calculate which clauses are described as the most prevalent when ranking each class/item. A potential approach to this is presented with the introduction of the integer weighted TM by [2]. We can then perform relatively cheap elimination of items that do not fulfill the highest weighted clauses for the given input. ...
Recommendation Systems (RSs) are ubiquitous in modern society and are one of the largest points of interaction between humans and AI. Modern RSs are often implemented using deep learning models, which are infamously difficult to interpret. This problem is particularly exasperated in the context of recommendation scenarios, as it erodes the user's trust in the RS. In contrast, the newly introduced Tsetlin Machines (TM) possess some valuable properties due to their inherent interpretability. TMs are still fairly young as a technology. As no RS has been developed for TMs before, it has become necessary to perform some preliminary research regarding the practicality of such a system. In this paper, we develop the first RS based on TMs to evaluate its practicality in this application domain. This paper compares the viability of TMs with other machine learning models prevalent in the field of RS. We train and investigate the performance of the TM compared with a vanilla feed-forward deep learning model. These comparisons are based on model performance, interpretability/explainability, and scalability. Further, we provide some benchmark performance comparisons to similar machine learning solutions relevant to RSs.
... While the idea of Tsetlin automaton (TA) (Tsetlin, 1961) have been around since 1960s, using them in pattern recognition is relatively new. TMs have successfully addressed several machine learning tasks, including natural language understanding (Yadav et al., 2021a, b;Bhattarai et al., 2021;Saha et al., 2020;Berge et al., 2019), image analysis , classification (Abeyrathna et al., 2021), regression (Abeyrathna et al., 2019), and speech understanding (Lei et al., 2021). The propositional clauses constructed by a Tsetlin machine (TM) have high discriminative power and constitute a global description of the task learnt (Blakely & Granmo, 2020;Saha et al., 2020). ...
... The propositional clauses constructed by a Tsetlin machine (TM) have high discriminative power and constitute a global description of the task learnt (Blakely & Granmo, 2020;Saha et al., 2020). Apart from maintaining accuracy comparable to state-of-the-art machine learning techniques, the method also has provided a smaller memory footprint and faster inference than more traditional neural network-based models Lei et al., 2020;Abeyrathna et al., 2021;Lei et al., 2021;Phoulady et al., 2019). Furthermore, Shafik et al. (2020) shows that TMs can be faulttolerant, able to mask stuck-at faults. ...
Tsetlin machines (TMs) are a pattern recognition approach that uses finite state machines for learning and propositional logic to represent patterns. In addition to being natively interpretable, they have provided competitive accuracy for various tasks. In this paper, we increase the computing power of TMs by proposing a first-order logic-based framework with Herbrand semantics. The resulting TM is relational and can take advantage of logical structures appearing in natural language, to learn rules that represent how actions and consequences are related in the real world. The outcome is a logic program of Horn clauses, bringing in a structured view of unstructured data. In closed-domain question-answering, the first-order representation produces 10 × more compact KBs, along with an increase in answering accuracy from 94.83 % to 99.48 % . The approach is further robust towards erroneous, missing, and superfluous information, distilling the aspects of a text that are important for real-world understanding
