FIGURE 3 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Source publication
Joint extraction of entities and their relations from the text is an essential issue in automatic knowledge graph construction, which is also known as the joint extraction of relational triplets. The relational triplets in sentence are complicated, multiple and different relational triplets may have overlaps, which is commonly seen in reality. Howe...
Context in source publication
Context 1
... which is the inputs vector for the transformer encoder. We use the transformer encoder refers to [14]. Based on multi-head self-attention mechanism, the transformer encoder is capable of capturing association between each sequence tokens. We adopt the transformer encoder by stacking N numbers of blocks, whose architecture is shown in Fig. 3. There are two main sub-layers in each block, a multi-head self-attention layer and a simple feed forward layer, both with a residual connection and layer normalization. The inputs X is projected into three different types of vectors: key K , value V and query Q. Multi-head attention is calculated ...
Similar publications
Information extraction tasks such as triple extraction and event extraction are of great importance for natural language processing and knowledge graph construction. In this paper, we revisit the end-to-end information extraction task for sequence generation. Since generative information extraction may struggle to capture long-term dependencies and...
Event detection is a particularly challenging problem in information extraction. The current neural network models have proved that dependency tree can better capture the correlation between candidate trigger words and related context in the sentence. However, syntactic information conveyed by the original dependency tree is insufficient for detect...
The availability of big data and affordable hardware have enabled the applications of deep learning on different tasks. With respect to security, several attempts have been made to transfer deep learning’s application from the domain of image recognition or natural language processing into malware detection. In this study, we propose AdMat - a simp...
Citations
... The pipelining technique is a way of sequentially processing NER and relation extension activities. On the other hand, the point extraction approach accomplishes the triplet's outcomes by simultaneously extracting the entity and relation [31]. The combined work may be carried out with current NLP techniques, such as those in [32,33] or [34], based on CNN or RNN. ...
... Considerable efforts have been made to improve BERT since its introduction by Google Inc. in 2018, mainly concentrating on the pretraining procedure and the encoder [31]. BERT uses a transformer construction. ...
Additive Manufacturing (AM) is gaining acceptance as a strategic manufacturing technique and technology for allowing new product development. Due to the lack of knowledge, design for additive manufacturing (DFAM) is now a major challenge in utilizing AM's product innovation and manufacturing capabilities. The AM sector will benefit from developing an intuitive knowledge reasoning method by constructing a Knowledge Graph (KG). We presented Bidirectional Encoder Representations from the Transformers (BERT) model for collaborative entity/ relation recognition to address the issue, allowing us to study and utilize the advantages of AM through knowledge reasoning for Fused Deposition Modeling (FDM) based DFAM. First, the model analyzes preprocessed text to find and extract entities. Then, the relation recognition procedure based on dependency parsing extracts the semantic relationships between the entities. To convert word segments into vectors and improve dependency parsing, we used Continuous-Bag-of-Words (CBOW) to process texts. Therefore, this allowed us to anticipate the probability output of the center word based on the n − 1 words around the input. The extracted knowledge is then visualized as a graph and stored in the Neo4j database. Following the methods above creates a KG for the FDM-based DFAM knowledge. It can be shown that BERT is a good option for handling knowledge-driven issues needing specialists by extracting the process knowledge from text data using our suggested model. We provide evidence demonstrating the model's ability to set reasonable limitations on its predictions through a KG. Additionally, we use experiments and an application case study to demonstrate the effectiveness and competitiveness of our approach.
... To reduce errors in the extraction model, the system automatically modifies the connection weights and learning parameters throughout model training. LBMs have also been deployed as state-of-the-art models that feature various DL-based methods for IE from textual data [17], [18], [33], [35], [45], [46], [47]. ...
... EE aims to detect the existence of an event reported in the text and collect all attributes related to the event. Hybrid techniques [15], [18], [28], [47], [48], [49] have been developed by combining existing techniques with other techniques to strengthen IE strategies based on individual case studies. Other methods include methods that implement rule-based, ontology-based, learning-based, or DL-based methods. ...
... As mentioned earlier, the issue of limited labeled data can be managed using ACE systems [35]. A few examples of RE using the LBM include the CNN-BiLSTM network [43], weakly supervised RE [35], and DNN unsupervised learning [45], [47], [70]. To date, research on RE is still in progress. ...
Information extraction (IE) is a challenging task, particularly when dealing with highly heterogeneous data. State-of-the-art data mining technologies struggle to process information from textual data. Therefore, various IE techniques have been developed to enable the use of IE for textual data. However, each technique differs from one another because it is designed for different data types and has different target information to be extracted. This study investigated and described the most contemporary methods for extracting information from textual data, emphasizing their benefits and shortcomings. To provide a holistic view of the domain, this comprehensive systematic literature review employed a systematic mapping process to summarize studies published in the last six years (from 2017 to 2022). It covers fundamental concepts, recent approaches, applications, and trends, in addition to challenges and future research prospects in this domain area. Based on an analysis of 161 selected studies, we found that the state-of-the-art models employ deep learning to extract information from textual data. Finally, this study aimed to guide novice and experienced researchers in future research and serve as a foundation for this research area.
... Luo et al. [15] constructed a relation extraction method based on the fusion of attention mechanism and neural network, which effectively reduced the error label of the relation,and improves the accuracy of the model by optimizing the loss function. Pang et al. [16] propose a deep neural network model based on the sequence-tosequence learning, namely, the hybrid dual pointer networks (HDP), which extracts multiple pairs of triplets from the given sentence by generating the hybrid dual pointer sequence. ...
Extracting entity, relation, and attribute information from unstructured text is crucial for constructing large-scale knowledge graphs(KG). Existing research approaches either focus on entity recognition before relation extraction or employ unified annotation. However, these methods overlook the intrinsic relation between entity recognition and relation extraction, resulting in ineffective handling of triple overlap issues where multiple relations share the same entity in a sentence. To address these challenges, this paper proposes an entity-relation joint extraction model comprising two independent sub-modules: one for extracting the head entity and the other for extracting the tail entity and its corresponding relation. The model generates candidate entities and relations by enumerating token sequences in sentences, and then uses the two sub-modules to predict entities and relations. The predicted entities and relations are jointly decoded to obtain relational triples, avoiding error propagation and solving redundancy, entity overlap, and poor generalization. Extensive experiments demonstrate that our model achieves state-of-the-art performance on WebNLG, NYT, WebNLG*, and NYT* public benchmarks. It outperforms all baselines on the WebNLG* dataset, showing significant improvements in different types of triples: normal, SEO, and EPO by 3.8%, 2.9%, and 5.5%, respectively, compared to ETL-Span. For the NYT* dataset, our method improves by 5.7% in triples of Normal type, thereby confirming its effectiveness.
... The advantage of this model is that it deals with isolated relationships, but the method is not effective in identifying overlapping relationships, and the association between two corresponding entities still needs to be refined. Based on joint decoding, Pang et al. proposed a deep neural network model for sequence-to-sequence-based learning called a hybrid dual-pointer network (HDP), which was designed to extract multiple-pair triples from a given sentence by generating hybrid dual-pointer sequences [53]. The performance of this method for entity overlap (one entity participating in multiple triads) is better than that for relationship overlap (multiple relationships in a pair of entities). ...
... Converts the multi-pair-triplet extraction into a sequence generation task; generates a hybrid binary-pointer sequence extraction to alleviate the entity overlap problem [53]. ...
As a core task and an important link in the fields of natural language understanding and information retrieval, information extraction (IE) can structure and semanticize unstructured multi-modal information. In recent years, deep learning (DL) has attracted considerable research attention to IE tasks. Deep learning-based entity relation extraction techniques have gradually surpassed traditional feature- and kernel-function-based methods in terms of the depth of feature extraction and model accuracy. In this paper, we explain the basic concepts of IE and DL, primarily expounding on the research progress and achievements of DL technologies in the field of IE. At the level of IE tasks, it is expounded from entity relationship extraction, event extraction, and multi-modal information extraction three aspects, and creates a comparative analysis of various extraction techniques. We also summarize the prospects and development trends in DL in the field of IE as well as difficulties requiring further study. It is believed that research can be carried out in the direction of multi-model and multi-task joint extraction, information extraction based on knowledge enhancement, and information fusion based on multi-modal at the method level. At the model level, further research should be carried out in the aspects of strengthening theoretical research, model lightweight, and improving model generalization ability.
... There are two defects in this method [24]. One is that entity recognition and relationship extraction are regarded as two serial tasks resulting in mutual dependence between the two tasks, and the error of entity recognition will be amplified in the relationship extraction task; Second, although this method can solve the SEO problem, it cannot solve the EPO problem. ...
As an important research field of artificial intelligence, knowledge graph develops rapidly, and triplet extraction is the key to the construction of a knowledge graph. The traditional pipeline extraction method will bring the error of entity recognition into the relationship extraction and affects the extraction effect. Besides, the traditional pipeline extraction method cannot solve the SEO (Single Entity Overlap) and EPO (Entity Pair Overlap) problems. Inspired by this, we compare the advantages and disadvantages of the mainstream methods of entity and relationship triples joint extraction, propose a new joint extraction method of entity relation triples based on a hierarchical cascade labeling model (named HCL model), and the HCL model is based on multi neural network cooperation. Further, we construct a balanced sampling Chinese dataset about the entity and relational triplet extraction which contains SEO and EPO. We carry out the experiments on the balanced data set, and the F1 value of the HCL model reaches 65.4% better than other baseline models.
... In these tasks, spans of interest are identified, and linkages between spans are predicted. Many contemporary IE systems use end-to-end multi-layer neural models that encode an input word sequence using recurrent or transformer layers, classify spans (entities, arguments, etc.), and predict the relationship between spans (coreference, relation, role, etc.) [24][25][26][27][28][29]. Of most relevance to our work is a series of developments starting with Lee et al. [30], which introduces a span-based coreference resolution model that enumerates all spans in a word sequence, predicts entities using a feed-forward neural network (FFNN) operating on span representations, and resolves coreferences using a FFNN operating on entity span-pairs. ...
Coronavirus disease 2019 (COVID-19) is a global pandemic. Although much has been learned about the novel coronavirus since its emergence, there are many open questions related to tracking its spread, describing symptomology, predicting the severity of infection, and forecasting healthcare utilization. Free-text clinical notes contain critical information for resolving these questions. Data-driven, automatic information extraction models are needed to use this text-encoded information in large-scale studies. This work presents a new clinical corpus, referred to as the COVID-19 Annotated Clinical Text (CACT) Corpus, which comprises 1,472 notes with detailed annotations characterizing COVID-19 diagnoses, testing, and clinical presentation. We introduce a span-based event extraction model that jointly extracts all annotated phenomena, achieving high performance in identifying COVID-19 and symptom events with associated assertion values (0.83-0.97 F1 for events and 0.73-0.79 F1 for assertions). Our span-based event extraction model outperforms an extractor built on MetaMapLite for the identification of symptoms with assertion values. In a secondary use application, we predicted COVID-19 test results using structured patient data (e.g. vital signs and laboratory results) and automatically extracted symptom information, to explore the clinical presentation of COVID-19. Automatically extracted symptoms improve COVID-19 prediction performance, beyond structured data alone.
... In these tasks, spans of interest are identified, and linkages between spans are predicted. Many contemporary IE systems use end-to-end multi-layer neural models that encode an input word sequence using recurrent or transformer layers, classify spans (entities, arguments, etc.), and predict the relationship between spans (coreference, relation, role, etc.) [15][16][17][18][19][20]. Of most relevance to our work is a series of developments starting with Lee et al. [21], which introduces a span-based coreference resolution model that enumerates all spans in a word sequence, predicts entities using a feed-forward neural network (FFNN) operating on span representations, and resolves coreferences using a FFNN operating on entity span-pairs. ...
Coronavirus disease 2019 (COVID-19) is a global pandemic. Although much has been learned about the novel coronavirus since its emergence, there are many open questions related to tracking its spread, describing symptomology, predicting the severity of infection, and forecasting healthcare utilization. Free-text clinical notes contain critical information for resolving these questions. Data-driven, automatic information extraction models are needed to use this text-encoded information in large-scale studies. This work presents a new clinical corpus, referred to as the COVID-19 Annotated Clinical Text (CACT) Corpus, which comprises 1,472 notes with detailed annotations characterizing COVID-19 diagnoses, testing, and clinical presentation. We introduce a span-based event extraction model that jointly extracts all annotated phenomena, achieving high performance in identifying COVID-19 and symptom events with associated assertion values (0.83-0.97 F1 for events and 0.73-0.79 F1 for assertions). In a secondary use application, we explored the prediction of COVID-19 test results using structured patient data (e.g. vital signs and laboratory results) and automatically extracted symptom information. The automatically extracted symptoms improve prediction performance, beyond structured data alone.
... The pipeline method is a method of processing tasks in order with the NER task and relation extension task. The point extraction method, on the other hand, achieves the results of the triplet by extracting the entity and relation at the same time [24]. ...
Along with studies on artificial intelligence technology, research is also being carried out actively in the field of natural language processing to understand and process people’s language, in other words, natural language. For computers to learn on their own, the skill of understanding natural language is very important. There are a wide variety of tasks involved in the field of natural language processing, but we would like to focus on the named entity registration and relation extraction task, which is considered to be the most important in understanding sentences. We propose DeNERT-KG, a model that can extract subject, object, and relationships, to grasp the meaning inherent in a sentence. Based on the BERT language model and Deep Q-Network, the named entity recognition (NER) model for extracting subject and object is established, and a knowledge graph is applied for relation extraction. Using the DeNERT-KG model, it is possible to extract the subject, type of subject, object, type of object, and relationship from a sentence, and verify this model through experiments.
Knowledge extraction is meant by acquiring relevant information from the unstructured document in natural language and representing them in a structured form. Enormous information in various domains, including agriculture, is available in the natural language from several resources. The knowledge needs to be represented in a structured format to understand and process by a machine for automating various applications. This paper reviews different computational approaches like rule-based and learning-based methods and explores the various techniques, features, tools, datasets, and evaluation metrics adopted for knowledge extraction from the most relevant literature.
The rapid development of biomedicine has produced a large number of biomedical written materials. These unstructured text data create serious challenges for biomedical researchers to find information. Biomedical named entity recognition (BioNER) and biomedical relation extraction (BioRE) are the two most fundamental tasks of biomedical text mining. Accurately and efficiently identifying entities and extracting relations have become very important. Methods that perform two tasks separately are called pipeline models, and they have shortcomings such as insufficient interaction, low extraction quality and easy redundancy. To overcome the above shortcomings, many deep learning-based joint name entity recognition and relation extraction models have been proposed, and they have achieved advanced performance. This paper comprehensively summarize deep learning models for joint name entity recognition and relation extraction for biomedicine. The joint BioNER and BioRE models are discussed in the light of the challenges existing in the BioNER and BioRE tasks. Five joint BioNER and BioRE models and one pipeline model are selected for comparative experiments on four biomedical public datasets, and the experimental results are analyzed. Finally, we discuss the opportunities for future development of deep learning-based joint BioNER and BioRE models.
