Figure 3 - uploaded by George Labahn
Content may be subject to copyright.
Source publication
We present a new approach to multi-dimensional parsing using relational grammars and fuzzy sets. A fast, incremental parsing algorithm is developed, motivated by the two-dimensional structure of written mathematics. Our approach makes no hard deci-sions; recognized math expressions are reported to the user in ranked order. A flexible correction mec...
Context in source publication
Similar publications
The article presents a series of proposed extensions to Świdzińki’s formal grammar of Polish which were introduced in the course of automated syntax verification of a corpus of Polish expressions.
This on the web, most structured document collections consist of documents from different sources and marked up with different types of structures. The diversity of structures has lead to the emergence of heterogeneous structured documents. The heterogeneity of structured documents poses new challenges for document representation in structured docu...
Many attempts have been made to apply Natural Language Processing to requirements specifications. However, typical approaches rely on shallow parsing to identify object-oriented elements of the specifications (e.g. classes, attributes, and methods). As a result, the models produced are often incomplete, imprecise, and require manual revision and va...
Citations
... CNN is the second most common estimating method, and it has been explored in approximately 13 (approximately 21%) distinct papers. Several distinct varieties of neural networks (NNs), such as backpropagation networks [167] and recurrent neural networks, have been utilized and incorporated in this research [11,216,219], and fuzzy NN [69,90,118], etc. ANN has been investigated in 11 (18%approx.). While BLSTM has been employed in 6 (11%approx.) and RNN has been implemented in four different studies. ...
... The major non-ML recognition approaches identified are grammar-based approaches like graph grammar ( [56,75,89], stochastic context-free grammar ( [7,108,134,201], probabilistic context-free grammar ( [35,171], definite clause grammar [39], and another algorithmic approach ( [141,142,210]. There have also been instances of the studies which are concentrated towards parsing ( [41,108,206,208], fuzzy methodologies ( [58,60,70,90,102,117,118] and other methods based on relational grammars [120]. Though the total count of studies, 38% of the chosen studies, are purely based on non-ML approaches, the inclination towards these methods cannot be neglected as the initials of research on HMSER thoroughly engaged in their deployment on these approaches. ...
Tools based on machine learning (ML) have seen widespread application in the prediction and categorization of mathematical symbols and phrases. The purpose of this work is to conduct a comprehensive analysis of the machine learning and non-machine learning strategies that are currently in use for the recognition of mathematical expressions. (MEs). The authors collected and analyzed research studies on the recognition of MEs (and closely related models and issues as well), which are published from January 2000 to December 2022 in the SLR. The review has nominated 98 primary studies out of the extracted 202 studies after heedful filtering using inclusion/exclusion criteria and quality assessment. The pertinent data is derived from IEEE explore, Science Direct, Wiley, Scopus, ACM Digital Library, etc. For assiduously reviewing and synthesizing the data, the authors used grounded theory and other qualitative and quantitative techniques. The analysis reveals that the support vector machine as an ML model with CROHME as the dataset and expression recognition rate as an accuracy metric is frequently used in the chosen studies. Recognition is typically fragmented down into three stages—segmenting symbols, recognizing symbols, and analyzing structures—in non-ML studies. In conclusion, this work aims to synthesize the results of existing research to provide a summary of the state-of-the-art in recognizing handwritten MEs.
... The case two analysis shows the divergence of attention of researchers of the domain towards fuzzy-based methodologies and neural network-based algorithmic techniques. There has been significant literature that shows the experimentation using fuzzy techniques MacLean and Labahn 2010;Fitzgerald et al. 2007;Genoe et al. 2006b;Genoe et al. 2006c), dynamic programming Guo et al. 2007) and other grammar-based and parsing algorithms Rhee and Kim 2009;Vuong et al. 2008;LaViola and Zeleznik 2007;Yamamoto et al. 2006) based recognition. A shift of attention has been observed toward machine learning Álvaro and Sánchez 2010) and neural networks Ramteke and Mehrotra 2006) based on recognition models. ...
With the advent and advancement of machine learning and deep learning techniques, machine-based recognition systems for mathematical text have captivated the attention of the research community for the last four decades. Mathematical Expression Recognition systems have been identified based on terms of their techniques, approach, dataset, and accuracies. This study majorly targets a rigorous review of both the published form of literature and the least attended literature, i.e., grey literature. Apart from the digital libraries, the papers and other instances of information have been gathered from the grey sources like google patents, archives, technical reports, app stores, etc., culminating in 262 instances. After the heedful filtration imposed on both white and grey literature, the final pool of studies has been investigated for eight formulated research questions. The answers extracted have been analyzed, providing both quantitative and qualitative insights. The analysis and surveys have systematically summed up the potentials of both white and grey shades of literature present on MER and brought exciting extractions out of 155 formal white literature and 107 grey sources. The survey extracts and brings out the highlighting observations after analysis, which sublimates the fact that 52% of grey literature is composed of mobile applications and user interfaces, whereas the published 63% of white data is presently concentrated in 39 different conferences, and the prominent conference is ICDAR (#30). A list of challenges and open issues has been extracted for directing future research dimensions.
... This property is related with the basic membership concept found on set theory [11]. However, in the recognition of unpredictable input (incomplete sentences or handwritten expressions [9,13,8]), it is impossible to create a grammar that covers all possible cases. If in some cases classical parsing falls short to solve the problem at hand, in other cases it goes beyond the needs. ...
Traditional parsing has always been a focus of discussion among the computer science community. Numerous techniques and algorithms have been proposed along these years, but they require that input texts are correct according to a specific grammar. However, in some cases it's necessary to cope with incorrect or unpredicted inputs that raise ambiguities, making traditional parsing unsuitable. These situations led to the emergence of robust parsing theories, where fuzzy parsing gains relevance. Robust parsing comes with a price by losing precision and decaying performance, as multiple parses of the input may be necessary while looking for an optimal one. In this short paper we briefly describe the main robust parsing techniques and end up proposing a different solution to deal with fuzziness of input texts. It is based on automata where states represent contexts and edges represent potential matches (of constructs of interest) inside those contexts. It is expected that such an approach reduces recognition time and ambiguity as contexts reduce the search space by defining a smaller domain for constructs of interest. Such benefits may be a great addition to the robust parsing area with application on program comprehension, among other research fields.
... Scott et al. [13] and MacLean [21] used a variant of the relational context-free grammar called fuzzy relational context-free grammar (Fuzzy r-CFG) to model mathematical structure. Fuzzy r-CFG is similar to SCFG. ...
Despite the recent advances in handwriting recognition, handwritten two-dimensional (2D) languages are still a challenge. Electrical schemas, chemical equations and mathematical expressions are examples of such 2D languages. In this case, the recognition problem is particularly difficult due to the two dimensional layout of the language. The main goal of our work is to study the application of two-dimensional (2D) languages on mathematical expression recognition, which is a special case of 2D graphical documents. The research work will be focus on context-free grammars which has the potential to cope with structural relations in documents. The first part of this report gives an overview of mathematical expression recognition as well as different kinds of grammars applied in the recognition. The second part of the report presents our developed system, including grammars, segmentation hypothesis generator, parsing algorithm and spatial relation.
... Similarly, a fuzzy online structural analysis algorithm is proposed in in order to cope with the nature of spatial relations. More recently, (Scott & George, 2010) proposed to associate each production rule of a context free grammar to a fuzzy function. We will propose a similar approach by associating each production rule to a Gaussian model specific to each spatial relation as we will see in section 5.2. ...
Despite the recent advances in handwriting recognition, handwritten two-dimensional (2D) languages are still a challenge. Electrical schemas, chemical equations and mathematical expressions (MEs) are examples of such 2D languages. In this case, the recognition problem is particularly difficult due to the two dimensional layout of the language. This paper presents an online handwritten mathematical expression recognition system that handles mathematical expression recognition as a simultaneous optimization of expression segmentation, symbol recognition, and 2D structure recognition under the restriction of a mathematical expression grammar. The originality of the approach is a global strategy allowing learning mathematical symbols and spatial relations directly from complete expressions. A new contextual modeling is proposed for combining syntactic and structural information. Those models are used to find the most likely combination of segmentation/recognition hypotheses proposed by a 2D segmentation scheme. Thus, models are based on structural information concerning the symbol layout. The system is tested with a new public database of mathematical expressions which was used in the CHROME competition. We have also produced a large base of semi-synthetic expressions which are used to train and test the global learning approach. We obtain very promising results on both synthetic and real expressions databases, as well as in the recent CHROME competition.
... Those models with a sufficiently small feature difference are then compared with the candidate stroke group using a fast variant of elastic matching distance [26]. Based on comparison of stroke-based features, the input strokes may be re-ordered and/or individually reversed so as to obtain the smallest match distance. ...
We present a new approach for parsing two-dimensional input using relational grammars and fuzzy sets. A fast, incremental parsing algorithm is devel-oped, motivated by the two-dimensional structure of written mathematics. The approach reports all identifi-able parses of the input. The parses are represented as a fuzzy set, in which the membership grade of a parse measures the similarity between it and the handwrit-ten input. To identify and report parses efficiently, we adapt and apply existing techniques such as rectangular partitions and shared parse forests, and introduce new ideas such as relational classes and interchangeability. We also present a correction mechanism which allows users to navigate parse results and choose the correct interpretation in case of recognition errors or ambigu-ity. Such corrections are incorporated into subsequent incremental recognition results. Finally, we include two empirical evaluations of our recognizer. One uses a novel user-oriented correction count metric, while the other replicates the CROHME 2011 math recognition con-test. Both evaluations demonstrate the effectiveness of our proposed approach.
Despite the increasing prevalence of touch-based tablet devices, little attention has been paid to what effects, if any, this form factor has on sketch behaviours in general and on sketch recognizers in particular. We investigate the title question through an empirical study in the context of mathematical expression recognition. Using a corpus of thirty expressions drawn on Tablet PC and iPad by thirty writers, we show that characteristics of sketching and drawing differ depending on platform. While recognition is most accurate on the Tablet PC due to its technical superiority, recognition is feasible on mobile touch-based devices. However, there are characteristics of sketching on multi-touch tablets that differ, and these physical characteristics of writing impact recognition accuracy. Together, our observations inform the broader Sketch Recognition community as they design systems targeted to multi-touch tablets.
