Figure 3 - available via license: Creative Commons Attribution-NonCommercial 4.0 International
Content may be subject to copyright.
Source publication
In order to solve the problem that traditional short text classification methods do not perform well on short text due to the data sparsity and insufficient semantic features, we propose a short text classification method based on convolutional neural network and semantic extension. Firstly, we propose an improved similarity to improve the coverage...
Context in source publication
Context 1
... using CNNs to classify short text, it is necessary to represent the short text as a matrix as the input of the network model. Therefore, it is necessary to cascade the word vector matrix W w of short text, the word vector matrix W c of conceptualization of short text, and the word vector matrix W ′ c of conceptualization of related words, Then form the joint word vector matrix W of short text, as shown in Figure 3, the corresponding formula is defined as follows: ...
Similar publications
Semantic segmentation is a significant method in remote sensing image (RSIs) processing and has been widely used in various applications. Conventional convolutional neural network (CNN)-based semantic segmentation methods are likely to lose the spatial information in the feature extraction stage and usually pay little attention to global context in...
Citations
... Data-driven approaches excel in adaptability and handling large datasets, but their reliance on extensive training and sometimes opaque processes can be limitations. Their ability to capture complex patterns, as discussed in recent studies [38,39], is countered by concerns over interpretability and overfitting risks. Hybrid models, combining rule-based and data-driven methods, enhance effectiveness but add complexity, as highlighted by Wang et al. [40]. ...
Aligned with global Sustainable Development Goals (SDGs) and multidisciplinary approaches integrating AI with sustainability, this research introduces an innovative AI framework for analyzing Modern French Poetry. It applies feature extraction techniques (TF-IDF and Doc2Vec) and machine learning algorithms (especially SVM) to create a model that objectively classifies poems by their stylistic and thematic attributes, transcending traditional subjective analyses. This work demonstrates AI’s potential in literary analysis and cultural exchange, highlighting the model’s capacity to facilitate cross-cultural understanding and enhance poetry education. The efficiency of the AI model, compared to traditional methods, shows promise in optimizing resources and reducing the environmental impact of education. Future research will refine the model’s technical aspects, ensuring effectiveness, equity, and personalization in education. Expanding the model’s scope to various poetic styles and genres will enhance its accuracy and generalizability. Additionally, efforts will focus on an equitable AI tool implementation for quality education access. This research offers insights into AI’s role in advancing poetry education and contributing to sustainability goals. By overcoming the outlined limitations and integrating the model into educational platforms, it sets a path for impactful developments in computational poetry and educational technology.
... Liao et al. [15] treated each category as a subtask and utilized the robustly optimized BERT pre-training method, based on the deep bidirectional Transformer, to extract features from both the text and category tokens. Wang et al. [16] addressed sparsity by semantically expanding short texts. They incorporated an attention mechanism into their neural network model to identify and include related words from the short text. ...
Short message services (SMS), microblogging tools, instant message apps, and commercial websites produce numerous short text messages every day. These short text messages are usually guaranteed to reach mass audience with low cost. Spammers take advantage of short texts by sending bulk malicious or unwanted messages. Short texts are difficult to classify because of their shortness, sparsity, rapidness, and informal writing. The effectiveness of the hidden Markov model (HMM) for short text classification has been illustrated in our previous study. However, the HMM has limited capability to handle new words, which are mostly generated by informal writing. In this paper, a hybrid model is proposed to address the informal writing issue by weighting new words for fast short text filtering with high accuracy. The hybrid model consists of an artificial neural network (ANN) and an HMM, which are used for new word weighting and spam filtering, respectively. The weight of a new word is calculated based on the weights of its neighbor, along with the spam and ham (i.e., not spam) probabilities of short text message predicted by the ANN. Performance evaluations on benchmark datasets, including the SMS message data maintained by University of California, Irvine; the movie reviews, and the customer reviews are conducted. The hybrid model operates at a significantly higher speed than deep learning models. The experiment results show that the proposed hybrid model outperformsother prominent machine learning algorithms, achieving a good balance between filtering throughput and accuracy.
... Feng et al. [19] proposed a regularized nonnegative matrix decomposition topic model for short text, which uses pre-trained distribution vector representation of words to overcome the data sparsity problem of short text, meanwhile, it only considers the data itself and does not use external resources to expand the features of short text. Both Wang and Hao et al. [20,21] started to incorporate an attention mechanism in the neural network model to perform semantic expansion at sentence level and related word level of the short text respectively. Reference [22] proposed a new neural network topic model in a self-coding framework, which uses a new quantization method for topic distribution to generate a peak distribution more suitable for modeling short textbooks, in addition proposed a new negative sampling decoder to avoid generating duplicate topics using negative sampling learning. ...
Existing work generally classifies news headlines as a matter of short text classification. However, due to the strong domain nature and limited text length of news headlines, their classification results are usually determined by several specific keywords, which makes the traditional short text classification method ineffective. In this paper, we propose a new method to identify keywords in news headlines and expand their features from sentence level and word level respectively, and finally use convolutional neural networks (CNN) to extract and classify their features. The proposed model was tested on the Sogou News Corpus dataset and achieved 93.42 $$\%$$ % accuracy.
... Deep features have demonstrated superior performance in numerous domains in comparison to other types of features. Their popularity for feature extraction is on the rise due to their capacity to learn significant representations more effectively [51][52][53][54][55]. ...
Human communication is predominantly expressed through speech and writing, which are powerful mediums for conveying thoughts and opinions. Researchers have been studying the analysis of human sentiments for a long time, including the emerging area of bimodal sentiment analysis in natural language processing (NLP). Bimodal sentiment analysis has gained attention in various areas such as social opinion mining, healthcare, banking, and more. However, there is a limited amount of research on bimodal conversational sentiment analysis, which is challenging due to the complex nature of how humans express sentiment cues across different modalities. To address this gap in research, a comparison of multiple data modality models has been conducted on the widely used MELD dataset, which serves as a benchmark for sentiment analysis in the research community. The results show the effectiveness of combining acoustic and linguistic representations using a proposed neural-network-based ensemble learning technique over six transformer and deep-learning-based models, achieving state-of-the-art accuracy.
... It has been used in a variety of applications such as network analysis, search snippets, question-answering systems (QAS), information retrieval, text categorization, and so on. Earlier, the methods calculated the similarity between longer sentences in a very high dimensional space; thus, the method used has been inefficient and cannot be effective for many NLP tasks [1]. However, the short text reveals information that is important to understand. ...
... Deep learning is becoming more and more prevalent in the field of natural language processing (NLP). Several models of attentionbased neural networks sparked the interest of many academics, such as the convolutional neural network (CNN), the recurrent neural network (RNN), and the bi-directional encoder representation from the transformer (BERT) [1]. Table 7 presents a summary of the convolutional network (CNN) model in various categories of short text, to which Wang et al. [1] contributed their research in the field of short-text semantic similarity (STSS). ...
... Several models of attentionbased neural networks sparked the interest of many academics, such as the convolutional neural network (CNN), the recurrent neural network (RNN), and the bi-directional encoder representation from the transformer (BERT) [1]. Table 7 presents a summary of the convolutional network (CNN) model in various categories of short text, to which Wang et al. [1] contributed their research in the field of short-text semantic similarity (STSS). They used a convolutional neural network (CNN) for short text classification to find the related words, and, further, they used external knowledge to understand the conceptual meaning of answers and words from a short text. ...
In natural language processing, short-text semantic similarity (STSS) is a very prominent field. It has a significant impact on a broad range of applications, such as question–answering systems, information retrieval, entity recognition, text analytics, sentiment classification, and so on. Despite their widespread use, many traditional machine learning techniques are incapable of identifying the semantics of short text. Traditional methods are based on ontologies, knowledge graphs, and corpus-based methods. The performance of these methods is influenced by the manually defined rules. Applying such measures is still difficult, since it poses various semantic challenges. In the existing literature, the most recent advances in short-text semantic similarity (STSS) research are not included. This study presents the systematic literature review (SLR) with the aim to (i) explain short sentence barriers in semantic similarity, (ii) identify the most appropriate standard deep learning techniques for the semantics of a short text, (iii) classify the language models that produce high-level contextual semantic information, (iv) determine appropriate datasets that are only intended for short text, and (v) highlight research challenges and proposed future improvements. To the best of our knowledge, we have provided an in-depth, comprehensive, and systematic review of short text semantic similarity trends, which will assist the researchers to reuse and enhance the semantic information.
... Unlike long texts, Chinese short texts often have the problem of data sparsity, which is a problem that researchers need to focus on. Wang et al. [40] proposed a Chinese short text classification model based on CNN and semantic expansion, which improved the similarity measurement method to improve the coverage of word vector table during short text preprocessing, and added attention mechanism to find related words in short text. Zhang et al. [41] proposed a Chinese short text emotion classification model based on ELECTRA, self-attention mechanism and BiLSTM, and significantly improved the classification accuracy. ...
Short text classification is an important task in Natural Language Processing (NLP). The classification result for Chinese short text is always not ideal due to the sparsity problem of them. Most of the previous classification models for Chinese short text are based on word or character, considering that Chinese radical can also represent the meaning individually, so word, character and radical are all used to build a Chinese short text classification model in this paper, which solves the data sparsity problem of short text. In addition, in the process of segmenting sentences into words, considering that jieba will cause the loss of key information and ngram will generate noise words, both jieba and ngram are used to construct a six-granularity (i.e. word-jieba, word-jieba-radical, word-ngram, word-ngram-radical, character and character-radical) based Chinese short text classification (SGCSTC) model. Additionally, different weights are assigned to the six granularities and are automatically updated in the process of back-propagation using cross-entropy loss due to the different influence of them on the classification results. The classification Accuracy, Precision, Recall and F1 of SGCSTC in THUCNews-S dataset are 93.36%, 94.47%, 94.15% and 94.31% respectively, and that in CNT dataset are 92.67%, 92.38%, 93.15% and 92.76% respectively, and multiple comparative experiment results on THUCNews-S and CNT datasets show that SGCSTC outperforms the state-of-the-art text classification models.
... Various attention-based neural network models, including Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Bidirectional Encoder Representations from Transformers (BERT), have captured the attention of numerous researchers. In the domain of short text semantic similarity (STSS), Wang et al. [8] conducted a study in the field of short text semantic similarity. They utilized a Convolutional Neural Network (CNN) to classify short text and identify related words. ...
... First, previous methods mainly rely on entities and external knowledge to detect them, which suffers from the problem of entity polysemy. Secondly, most of the news graphs constructed using heterogeneous graphs are in the form of short text, which suffers from the problems of data sparsity and insufficient semantic features [5], leading to difficulties in extracting accurate and critical sample features during detection. In order to solve the problems of existing methods, this paper proposes a topicaware fake news detection method FND based on heterogeneous graphs. ...
In recent years, fake news has had a bad impact on individuals and society, which has aroused widespread concern about fake news detection. The existing heterogeneous graph-based fake news detection model (CompareNet) mainly focuses on the semantic consistency analysis between news content and external knowledge, which out-performs traditional content detection models in terms of its efficiency and scalability. However, we found that the framework ignores the fact that the node content of heterogeneous graphs is mostly in the form of short text, and such methods often have difficulty in extracting effective features due to the sparsity problem of short text data. In addition, previous studies have not considered the structural relationship between different writing styles of fake news. Aiming at the above problems this paper proposes a topic-aware fake news detection (FND) method based on heterogeneous graphs, the model investigates the effect of news topics on fake news detection and enhances the discriminative ability of fake news detection. Our model introduces semantically enhanced topic node information in the fake news detector, which fully utilizes three types of information: external knowledge (Wikipedia), news content, and news topics. Therefore, it can better enhance the fake news detection performance.
... The results of both transductive and the generally more useful inductive short text classifier are typically unsatisfactory due to the challenge that short text presents. Recent studies on short texts have emphasized specialized models [29,32,34,37,39,42] to address the issues associated with the short text length. However, State of the Art (SOTA) text classification methods, particularly the pure use of Transformers, have been unexploited. ...
... SECNN. The SECNN [32] is a text classification model built on CNNs that was created specifically for short texts with few and insufficient semantic features. Wang et al. [32] suggested four parts to address this issue. ...
... The SECNN [32] is a text classification model built on CNNs that was created specifically for short texts with few and insufficient semantic features. Wang et al. [32] suggested four parts to address this issue. In order to achieve better coverage on the word vector table, they used an improved Jaro-Winkler similarity during preprocessing to identify any potential spelling mistakes. ...
Short text classification is a crucial and challenging aspect of Natural Language Processing. For this reason, there are numerous highly specialized short text classifiers. However, in recent short text research, State of the Art (SOTA) methods for traditional text classification, particularly the pure use of Transformers, have been unexploited. In this work, we examine the performance of a variety of short text classifiers as well as the top performing traditional text classifier. We further investigate the effects on two new real-world short text datasets in an effort to address the issue of becoming overly dependent on benchmark datasets with a limited number of characteristics. Our experiments unambiguously demonstrate that Transformers achieve SOTA accuracy on short text classification tasks, raising the question of whether specialized short text techniques are necessary.
... Liu et al. [19] presented a BiLSTM model with convolutional layers based on an attention mechanism, where the attention mechanism is used to give different attention to contextual feature information. Wang et al. [20] devised a short text classification technique based on convolutional neural networks and semantic extensions to solve the issue that existing short text classification methods struggle to effectively classify short texts due to sparse data and inadequate semantic characteristics. For the classification of brief texts, Xu et al. [21] established the DE-CNN, a neural network that can combine contextually pertinent concepts into a convolutional neural network. ...
Text classification plays an important role in information science. In order to address the issues of low classification efficiency, low accuracy, and incomplete text feature extraction in existing classification methods, this work offers a two-channel hierarchical attention mechanism short text classification model (TCHAM). First, a layered word vector attention mechanism is developed to improve the capture of keywords and phrases. Second, the TextBERT model is applied to train the word vector representation to solve the problem of multiple meanings of a word. Third, a two-channel neural network is utilized to achieve parallel acceleration. Finally, the output information of the two-channel neural network is fused to raise the accuracy of news text classification. The experimental results show that under the same environment and dataset, TCHAM increases the accuracy of text classification, reaching 98.03%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\%$$\end{document} for the THUCNews dataset and 95.65%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\%$$\end{document} for the SogouNews dataset, and its classification performance outperforms the comparison model.
