Figure 5 - uploaded by Zhiguang Wang
Content may be subject to copyright.
Structure of the tiled convolutional neural network. We fix the size of receptive fields to 8 × 8 in the first convolutional layer and 3 × 3 in the second convolutional layer. Each TICA pooling layer pools over a block of 3 × 3 input units in the previous layer without wraparound at the boarders to optimize for sparsity of the pooling units. The number of pooling units in each map is exactly the same as the number of input units. The last layer is a linear SVM for classification. We construct this network by stacking two Tiled CNNs, each with 6 maps (l = 6) and tiling size k = 2.
Source publication
We propose an off-line approach to explicitly encode temporal patterns
spatially as different types of images, namely, Gramian Angular Fields and
Markov Transition Fields. This enables the use of techniques from computer
vision for feature learning and classification. We used Tiled Convolutional
Neural Networks to learn high-level features from ind...
Context in source publication
Citations
... The features present in an image are also easier for a human to interpret than a raw time series. Moreover, raw time series data typically possesses long temporal correlations, which can be challenging for conventional TSC algorithms to learn [44]. Lastly, imagetransform leverages the extensive computer vision research carried out in the past decade for use in TSC [14] [20]. ...
... However, recurrence plots require training to derive insights from, as they are highly complex [29]. Alternatively, the Gramian angular fields (GAF) image-transform technique is far more human-interpretable because temporal dependency is maintained; time increases from the top left to the bottom right of the image [44]. The GAF representation first involves the scaling of the time series data to [−1, 1] using a min-max scaler [44]. ...
... Alternatively, the Gramian angular fields (GAF) image-transform technique is far more human-interpretable because temporal dependency is maintained; time increases from the top left to the bottom right of the image [44]. The GAF representation first involves the scaling of the time series data to [−1, 1] using a min-max scaler [44]. The data is then converted into the polar coordinate system before computing the gram matrix of that data, assuming that n 2D vectors have a norm of 1 (see Equations 1 and 2), and using a custom inner product function (see Equations 3 and 4) [44]. ...
Banks, investment firms and other financial service providers are required to safeguard customers from unsuitable financial products under Know Your Customer (KYC) regulation, such as FCA Section 5.2. Recent work has proposed to model a customer's risk profile as a heatmap, which can be used to calculate a risk score by classifying the image via a CNN and extracting geometric features from it using contour detection. This provides an interpretable approach to analysing customer spending behaviour. However, there is a lack of comparative evaluation in the literature of alternative classification techniques to the heatmap representation, which is the focus of our paper. The heatmap model evaluated by this study achieved an F1 score of 94.6% when classifying heatmap geometry, far outperforming other configurations, including state-of-the-art algorithms typically employed for TSC such as HIVE-COTE, as well as alternative image-transform techniques such as Gramian angular fields. Our experiments used a transactional dataset produced by Lloyds Banking Group, a major UK retail bank, via agent-based modelling (ABM). This data was computer generated and at no point was real transactional data shared. This study shows that a grouped CNN model paired with the heatmap representation is superior to conventional time series classification and image-transform methods at classifying customer spending.
... The correlation between amplitude and time is the main relationship in time-series data. MTF can convert onedimensional time series data into two-dimensional feature images by considering the time and position information on the basis of Markov chain and using Markov transition probability for coding, so as to maintain the time order and statistical dynamics in the generated images (Wang and Oates, 2015). Due to its excellent time-series information retention ability, MTF coding technology has been partially applied in fault diagnosis (Yan et al., 2022) and surface electromyography signal analysis (Li et al., 2022). ...
In order to solve the difficulty that complex power quality disturbances (PQDs) are difficult to recognize accurately and efficiently under the new power system background, this paper proposes a novel PQDs recognition method based on markov transition field (MTF) and improved densely connected network (DenseNet). Firstly, the one-dimensional PQDs signal is mapped into the two-dimensional image with clear texture features by using MTF encoding method. Then, a DenseNet-S lightweight network is designed and the convolutional attention module (CBAM) is introduced to improve its feature extraction ability, so as to enhance the performance of the network. Finally, the images are input into the improved model for training and learning, and PQDs recognition is realized through the optimal model. In order to verify the effectiveness of the proposed method, experimental tests are carried out based on IEEE 1159 standard simulation dataset and real-world field measured signals dataset, and compared with existing recognition methods. The results show that the proposed method can effectively improve the recognition accuracy and noise robustness of complex PQDs, and has more advantages in disturbances recognition efficiency. It can meet the recognition accuracy and efficiency requirements of massive and complex PQDs events in engineering applications.
... Time series to 2D representation encoding is a popular technique for transforming time-series data into a visual format. It overcomes the traditional feature extraction techniques by capturing non-linear dependencies and being more robust to noise and outliers [10], [11]. Additionally, these techniques can preserve temporal information while reducing the dimensionality of the data, making it easier to process by machine learning classifiers [11]. ...
... It overcomes the traditional feature extraction techniques by capturing non-linear dependencies and being more robust to noise and outliers [10], [11]. Additionally, these techniques can preserve temporal information while reducing the dimensionality of the data, making it easier to process by machine learning classifiers [11]. Several techniques, including Gramian Angular Summation Field (GASF), Recurrence Plot (RP), and Markov Transition Field (MTF), have been proposed [10], [12]- [14]. ...
... MTF has shown superiority in capturing the temporal characteristics of ECG signals compared to GASF and RP. Additionally, MTF has the advantage of being computationally efficient, making it an attractive option for automated emotion recognition [11]. Gray-Level Co-occurrence Matrix (GLCM) and Gray-Level Run-Length Matrix (GLRLM) are two widely used techniques for quantifying texture information in images [15], [16]. ...
Automated emotion recognition using physiological signals, particularly Electrocardiogram (ECG), has diverse applications in human-computer interaction, healthcare, and psychology. This study proposes a novel ECG-based emotion recognition approach, utilizing time-series to image encoding, texture-based features, and machine-learning algorithms. The Continuously Annotated Signals of Emotion dataset is used, and emotional states are categorized based on arousal and valence annotations. ECGs are segmented into 5 and 7-window segments and transformed into 2D representations using Markov Transition Field (MTF). Extracting 43 features from the Gray-Level Co-occurrence Matrix and Gray-Level Run Length Matrix (GLRLM), three classifiers, including Random Forest (RF), Support Vector Machine, and eXtreme Gradient Boosting (XGB), are employed for emotion classification. The 7-window approach yields superior results, achieving a peak accuracy of 76.69% with XGB. High-Valence Low-Arousal emotional states are recognized best, with the highest F-measure of 61.9%. The findings suggest the potential for accurate and efficient emotion recognition using ECG, MTF, and machine-learning classifiers.
... For example, in Figure 1, columns of adjacent pixels indicate spending on consecutive days, while rows of adjacent pixels indicate spending on the same day of the week. Consequently, this interpretability allows complex temporal relationships, which are challenging for conventional TSC techniques to learn (Wang and Oates, 2015), to be clearly depicted in the image's geometry. Lastly, the heatmap technique has rarely been used in the literature (Butler et al., 2022), and thus represents a key gap in the literature to be explored. ...
... In Sinanc et al. (2021), the GAF image-transform technique (Wang and Oates, 2015) has demonstrated high effectiveness at predicting credit card fraud in transactional data, and may also be applicable to the customer-safeguarding aspect of KYC. One way to achieve this is to combine GAF with a heatmap representation where, instead of R, G and B layers (see Section 3.3), each "pixel" is instead composed of a GAF representation of that day's spending. ...
This paper explores a novel technique that can aid firms in ascertaining a customer's risk profile for the purpose of safeguarding them from unsuitable financial products. This falls under the purview of Know Your Customer (KYC), and a significant amount of regulation binds firms to this standard, including the Financial Conduct Authority (FCA) handbook Section 5.2. We introduce a methodology for computing a customer's risk score by converting their transactional data into a heatmap image, then extracting complex geometric features that are indicative of impulsive spending. This heatmap analysis provides an interpretable approach to analysing spending patterns. The model developed by this study achieved an F1 score of 94.6% when classifying these features, far outperforming alternative configurations. Our experiments used a transactional dataset produced by Lloyds Banking Group, a major UK retail bank, via agent-based modelling (ABM). This data was computer generated and at no point was real transactional data shared. This study shows that a combination of ABM and artificial intelligence techniques can be used to aid firms in adhering to financial regulation.
... CNNs are typically used to perform classification tasks on images, for instance on the ImageNet dataset [Deng et al., 2009]. Images are not time series however and in some approaches [Wang and Oates, 2015;Bashivan et al., 2015], a transformation technique is employed to transform the raw time series data into a sequence of images, before providing it as data to some kind of CNN. Other approaches make use of EEG, video or log data to do time series classification with CNNs, for instance in the context of making predictions in the medical domain [Hajinoroozi et al., 2016], human action recognition [Ijjina and Chalavadi, 2016], automatic annotation of clinical texts [Hasan et al., 2016] and automatic identification of predatory conversations in chat logs [Ebrahimi et al., 2016]. ...
... In this study, three different mathematical transformation methods are used to obtain images from the time series. These methods were used as Gramian Angular Field (GAF), Markov Transition Field (MTF) [16], [17], and Recurrence Plot (RP) [18], respectively. The RGB color map represents the values transformed by GAF and MTF methods. ...
... Encoding time series into at least two-dimensional space is required for the reconstruction of original signal using Gram matrix. Wang et al. used bijective mapping at polar coordinates instead of Cartesian coordinates [41]. A time seriesX was rescaled in polar coordinates by encoding all values as angular cosines and time stamps as the radius, as follows: ...
Diabetes mellitus, a chronic disease associated with elevated accumulation of glucose in the blood, is generally diagnosed through an invasive blood test such as oral glucose tolerance test (OGTT). An effective method is proposed to test type 2 diabetes using peripheral pulse waves, which can be measured fast, simply and inexpensively by a force sensor on the wrist over the radial artery. A self-designed pulse waves collection platform includes a wristband, force sensor, cuff, air tubes, and processing module. A dataset was acquired clinically for more than one year by practitioners. A group of 127 healthy candidates and 85 patients with type 2 diabetes, all between the ages of 45 and 70, underwent assessments in both OGTT and pulse data collection at wrist arteries. After preprocessing, pulse series were encoded as images using the Gramian angular field (GAF), Markov transition field (MTF), and recurrence plots (RPs). A four-layer multi-task fusion convolutional neural network (CNN) was developed for feature recognition, the network was well-trained within 30 minutes based on our server. Compared to single-task CNN, multi-task fusion CNN was proved better in classification accuracy for nine of twelve settings with empirically selected parameters. The results show that the best accuracy reached 90.6% using an RP with threshold
$\epsilon$
of 6000, which is competitive to that using state-of-the-art algorithms in diabetes classification.
... Bu belirleme işlemi matristeki tüm elemanlar için yapıldıktan sonra matris ∑ 1 eşitliğine göre normalize edildiğinde Markov geçiş matrisi elde edilmektedir. Elde edilen Markov geçiş matrisi x'in dağılımına duyarsızdır ve zamansal bağımlılığı bulunmamaktadır[23]. Eş. 13'de verilen denklemde, verilerin Q dağılım dilimine bölünmesiyle elde edilen Markov geçiş matrisi gösterilmektedir: ...
Ses dalgalarını kullanarak cismin boyut, uzaklık, yön ve diğer özelliklerinin tespit edilmesi için kullanılan sonar; denizaltı maden, petrol aramalarında, denizaltı haritalamasında, balık sürülerinin takibinde ve mayın tespitlerinde yaygın olarak kullanılmaktadır. Denizaltında mayınların yanı sıra mayınlara şekil ve yapı olarak çok benzeyen başka nesneler de gözlemlenebilmektedir. Sonar sinyallerinin tanımlanması ve sınıflandırılması için kullanılması gereken öznitelik çıkarımı, öznitelik seçimi, en uygun algoritmaların seçilmesi ve bu algoritmaların hiperparametre en iyilemesi çalışmaları, üzerinde uzun yıllardan beri çalışılan bilimsel problemler olarak karşımıza çıkmaktadır. Bu çalışmada, yenilikçi bir yaklaşımla üç farklı matematiksel dönüşüm kullanılarak verinin farklı bir formatta sayısal temsili önerilmekte ve derin öğrenme yöntemlerinin bu problem özelinde başarımlarının klasik makine öğrenmesi ve istatistiksel örüntü tanıma yöntemleri ile karşılaştırılması yapılmaktadır. Önerilen yenilikçi yöntem kapsamında, Markov Dönüşüm Alanı (MDA), Gramian Açısal Alanı (GAA, GATA, GAFA) ve Tekrarlanma Grafiği (TG) matematiksel dönüşümleri verinin zaman serisi türünden görüntü formatında ifade edilebilmesi için kullanılmıştır. Bu yaklaşım sonucunda elde edilen yeni tipteki verilerin kullanılmasıyla, derin öğrenme algoritmalarının çapraz doğrulama (cross validation) metodu ile eğitilmesi sağlanarak, üretilen modellerin performans sonuçları ve klasik algoritmalar ile elde edilen sonuçlar iyi bilinen metrikler kullanılarak karşılaştırılmıştır. Bu sonuçlar ışığında, önerilen zaman serisi verisinin görüntüye dönüştürülmesi yaklaşımlarının, problem çözümünde öznitelik çıkarma gereksinimini ortadan kaldırdığı ve bugüne kadar literatürde tespit edilen en iyi sonuçları verdiği belirlenmiştir. Önerilen yeni yaklaşımın, sadece zaman serisi tabanlı sınıflandırma problemleri için değil, farklı araştırma alanlarında da uygulanabileceği ve verinin sayısal olarak temsili amacıyla önerilen matematiksel dönüşümler ile makine öğrenmesi algoritmalarının başarımının arttırılması için önemli katkılar sağlayacağı değerlendirilmektedir.
... A key factor in the success of human activities recognition using EEG is the effective use of data obtained from measurement sensors. In this paper, the method proposed in [11] is used. In this method, the time series is converted into images, after which the convolutional neural network is used to analyze them. ...
... However, such a network does not work directly with time series. In [11], a method was proposed for using convolutional neural networks to classify time series. In this method, the time series is converted into images in three ways, after which the usual convolutional neural network is used to image. ...
The article is devoted to the problem of recognition of motor imagery based on electroencephalogram (EEG) signals, which is associated with many difficulties, such as the physical and mental state of a person, measurement accuracy, etc. Artificial neural networks are a good tool in solving this class of problems. Electroencephalograms are time signals, Gramian Angular Fields (GAF), Markov Transition Field (MTF) and Hilbert space-filling curves transformations are used to represent time series as images. The paper shows the possibility of using GAF, MTF and Hilbert space-filling curves EEG signal transforms for recognizing motor patterns, which is further applicable, for example, in building a brain-computer interface.
... The most frequently encountered and computer vision inspired feature extraction method for hand engineering approaches is the transformation of time series into images using specific imaging methods such as Gramian fields (Wang and Oates, 2015b;Wang and Oates, 2015a), recurrence plots (Hatami, Gavet, and Debayle, 2017;Tripathy and Acharya, 2018) and Markov transition fields (Wang and Oates, 2015). Unlike image transformation, other feature extraction methods are not domain agnostic. ...
Time series analysis is a field of data science which is interested in analyzing sequences of numerical values ordered in time. Time series are particularly interesting because they allow us to visualize and understand the evolution of a process over time. Their analysis can reveal trends, relationships and similarities across the data. There exists numerous fields containing data in the form of time series: health care (electrocardiogram, blood sugar, etc.), activity recognition, remote sensing, finance (stock market price), industry (sensors), etc. Time series classification consists of constructing algorithms dedicated to automatically label time series data. The sequential aspect of time series data requires the development of algorithms that are able to harness this temporal property, thus making the existing off-the-shelf machine learning models for traditional tabular data suboptimal for solving the underlying task. In this context, deep learning has emerged in recent years as one of the most effective methods for tackling the supervised classification task, particularly in the field of computer vision. The main objective of this thesis was to study and develop deep neural networks specifically constructed for the classification of time series data. We thus carried out the first large scale experimental study allowing us to compare the existing deep methods and to position them compared other non-deep learning based state-of-the-art methods. Subsequently, we made numerous contributions in this area, notably in the context of transfer learning, data augmentation, ensembling and adversarial attacks. Finally, we have also proposed a novel architecture, based on the famous Inception network (Google), which ranks among the most efficient to date.
