Figure 1 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
![Example of a sample covariance matrix of an ERP feature vector for the time points [0.1, 0.15, 0.2, 0.25, 0.3] s with three EEG channels from different head locations: F3 (front left), Cz (center) and O2 (back right). The matrix was calculated on 3896 epochs of a visual ERP paradigm. Black lines delineate the channel-wise (cross-)covariance blocks of the matrix. In order to transform this general covariance matrix into block-Toeplitz form, you average along the block-diagonals, i.e. the blocks sharing the same color border.](publication/364629988/figure/fig3/AS:11431281096625812@1668188507304/Example-of-a-sample-covariance-matrix-of-an-ERP-feature-vector-for-the-time-points-01.png)
Example of a sample covariance matrix of an ERP feature vector for the time points [0.1, 0.15, 0.2, 0.25, 0.3] s with three EEG channels from different head locations: F3 (front left), Cz (center) and O2 (back right). The matrix was calculated on 3896 epochs of a visual ERP paradigm. Black lines delineate the channel-wise (cross-)covariance blocks of the matrix. In order to transform this general covariance matrix into block-Toeplitz form, you average along the block-diagonals, i.e. the blocks sharing the same color border.
Source publication
Objective. Covariance matrices of noisy multichannel electroencephalogram time series data provide essential information for the decoding of brain signals using machine learning methods. However, small datasets and high dimensionality make it hard to estimate these matrices. In brain-computer interfaces (BCI) based on event-related potentials and a...
Contexts in source publication
Context 1
... the authors corrected for baseline drifts-a frequently used technique for ERP analysis-by subtracting the average value of the baseline interval [−0.2, 0] s from each epoch, and omitted two artifact-prone frontal EEG channels. Note that baseline correction violates the assumption of stationarity, as it distorts the variance in an epoch (Sosulski et al 2021). Similarly, using differently sized time interval averages violates stationarity, as the temporal distances are no longer constant between each time feature. ...
Context 2
... the authors corrected for baseline drifts-a frequently used technique for ERP analysis-by subtracting the average value of the baseline interval [−0.2, 0] s from each epoch, and omitted two artifact-prone frontal EEG channels. Note that baseline correction violates the assumption of stationarity, as it distorts the variance in an epoch (Sosulski et al 2021). Similarly, using differently sized time interval averages violates stationarity, as the temporal distances are no longer constant between each time feature. ...
Similar publications
Tactile attention tasks are used in the diagnosis and treatment of neurological and sensory processing disorders, while somatosensory event-related potentials (ERP) measured by electroencephalography (EEG) are used as neural correlates of attention processes. Brain-computer interface (BCI) technology provides an opportunity for the training of ment...
Citations
... Shortening calibration times can be addressed with two different types of algorithms: algorithms able to exploit very well the few available examples of the ongoing session [106,110], or algorithms that can take advantage of existing data recorded before the current session such that only little adaptation is needed for the ongoing session [37,57]. The former is generally not a strong point of deep learning models but rather of classical machine learning models that exploit expert knowledge, e.g., in the form of domainspecific regularization approaches [104,105,106,110]. The latter is better known as transfer learning, and using it is nearly always motivated by the reduction of calibration times. ...
In the field of brain-computer interfaces (BCIs), the potential for leveraging deep learning techniques for representing electroencephalogram (EEG) signals has gained substantial interest. This review synthesizes empirical findings from a collection of articles using deep representation learning techniques for BCI decoding, to provide a comprehensive analysis of the current state-of-the-art. Each article was scrutinized based on three criteria: (1) the deep representation learning technique employed, (2) the underlying motivation for its utilization, and (3) the approaches adopted for characterizing the learned representations. Among the 81 articles finally reviewed in depth, our analysis reveals a predominance of 31 articles using autoencoders. We identified 13 studies employing self-supervised learning (SSL) techniques, among which ten were published in 2022 or later, attesting to the relative youth of the field. However, at the time being, none of these have led to standard foundation models that are picked up by the BCI community. Likewise, only a few studies have introspected their learned representations. We observed that the motivation in most studies for using representation learning techniques is for solving transfer learning tasks, but we also found more specific motivations such as to learn robustness or invariances, as an algorithmic bridge, or finally to uncover the structure of the data. Given the potential of foundation models to effectively tackle these challenges, we advocate for a continued dedication to the advancement of foundation models specifically designed for EEG signal decoding by using SSL techniques. We also underline the imperative of establishing specialized benchmarks and datasets to facilitate the development and continuous improvement of such foundation models.
... The present study chose the LDA algorithm because high-dimension signals (i.e., the entire temporal waveform over half seconds) are used as the input to the classifier. LDA is a commonly used dimension reduction method [11] and has been applied to classify event-related potentials by other researchers in the past [12][13][14]. ...
Motion speed and direction are two fundamental cues for the mammalian visual system. Neurons in various places of the neocortex show tuning properties in term of firing frequency to both speed and direction. The present study applied a 32-channel electroencephalograph (EEG) system to 13 human subjects while they were observing a single object moving with different speeds in various directions from the center of view to the periphery on a computer monitor. Depending on the experimental condition, the subjects were either required to fix their gaze at the center of the monitor while the object was moving or to track the movement with their gaze; eye-tracking glasses were used to ensure that they followed instructions. In each trial, motion speed and direction varied randomly and independently, forming two competing visual features. EEG signal classification was performed for each cue separately (e.g., 11 speed values or 11 directions), regardless of variations in the other cue. Under the eye-fixed condition, multiple subjects showed distinct preferences to motion direction over speed; however, two outliers showed superb sensitivity to speed. Under the eye-tracking condition, in which the EEG signals presumably contained ocular movement signals, all subjects showed predominantly better classification for motion direction. There was a trend that speed and direction were encoded by different electrode sites. Since EEG is a noninvasive and portable approach suitable for brain–computer interfaces (BCIs), this study provides insights on fundamental knowledge of the visual system as well as BCI applications based on visual stimulation.
... Moreover, other classification methods potentially yield better accuracies and could be tested in future research. Recently, Sosulski and Tangermann [66] suggested using block-Toeplitz covariance matrices for LDA classification and outperformed sLDA on multiple data sets. ...
Error perception is known to elicit distinct brain patterns, which can be used to improve the usability of systems facilitating human-computer interactions, such as brain-computer interfaces. This requires a high-accuracy detection of erroneous events, e.g., misinterpretations of the user’s intention by the interface, to allow for suitable reactions of the system. In this work, we concentrate on steering-based navigation tasks. We present a combined electroencephalography-virtual reality (VR) study investigating different approaches for error detection and simultaneously exploring the corrective human behavior to erroneous events in a VR flight simulation. We could classify different errors allowing us to analyze neural signatures of unexpected changes in the VR. Moreover, the presented models could detect errors faster than participants naturally responded to them. This work could contribute to developing adaptive VR applications that exclusively rely on the user’s physiological information.
... Thus, a fairly natural idea is to extend the standard Riemannian approach by using these augmented covariance matrices with lags. This idea of using an augmented covariance matrix with delays was also recently introduced [11] where it was used in combination with a Linear Discriminant Analysis algorithm and applied on Event Related Potential datasets. To the authors' knowledge the approach of combing the augmented covariance with the Riemannian classification is completely a novelty. ...
Objective: Electroencephalography signals are recorded as a multidimensional dataset. We propose a new framework based on the augmented covariance extracted from an autoregressive model to improve motor imagery classification. Methods: From the autoregressive model can be derived the Yule-Walker equations, which show the emergence of a symmetric positive definite matrix: the augmented covariance matrix. The state-of the art for classifying covariance matrices is based on Riemannian Geometry. A fairly natural idea is therefore to extend the standard approach using these augmented covariance matrices. The methodology for creating the augmented covariance matrix shows a natural connection with the delay embedding theorem proposed by Takens for dynamical systems. Such an embedding method is based on the knowledge of two parameters: the delay and the embedding dimension, respectively related to the lag and the order of the autoregressive model. This approach provides new methods to compute the hyper-parameters in addition to standard grid search. Results: The augmented covariance matrix performed noticeably better than any state-of-the-art methods. We will test our approach on several datasets and several subjects using the MOABB framework, using both within-session and cross-session evaluation. Conclusion: The improvement in results is due to the fact that the augmented covariance matrix incorporates not only spatial but also temporal information, incorporating nonlinear components of the signal through an embedding procedure, which allows the leveraging of dynamical systems algorithms. Significance: These results extend the concepts and the results of the Riemannian distance based classification algorithm.
Many brain-computer interfaces make use of brain signals that are elicited in response to a visual, auditory or tactile stimulus, so-called event-related potentials (ERPs). In visual ERP speller applications, sets of letters shown on a screen are flashed randomly, and the participant attends to the target letter they want to spell. When this letter flashes, the resulting ERP is different compared to when any other non-target letter flashes. We propose a new unsupervised approach to detect this attended letter. In each trial, for every available letter our approach makes the hypothesis that it is in fact the attended letter, and calculates the ERPs based on each of these hypotheses. We leverage the fact that only the true hypothesis produces the largest difference between the class means. Note that this unsupervised method does not require any changes to the underlying experimental paradigm and therefore can be employed in almost any ERP-based setup. To deal with limited data, we use a block-Toeplitz regularized covariance matrix that models the background activity. We implemented the proposed novel unsupervised mean-difference maximization (UMM) method and evaluated it in offline replays of brain-computer interface visual speller datasets. For a dataset that used 16 flashes per symbol per trial, UMM correctly classifies 3651 out of 3654 letters ($99.92\,\%$) across 25 participants. In another dataset with fewer and shorter trials, 7344 out of 7383 letters ($99.47\,\%$) are classified correctly across 54 participants with two sessions each. Even in more challenging datasets obtained from patients with amyotrophic lateral sclerosis ($77.86\,\%$) or when using auditory ERPs ($82.52\,\%$), the obtained classification rates obtained by UMM are competitive. In addition, UMM provides stable confidence measures which can be used to monitor convergence.
