Fig 5 - uploaded by Takamitsu Matsubara
Content may be subject to copyright.
Snapshots of the successful robotic hand control with our proposed method by a new user's EMG signals. The intended motions are performed and corresponding robotic motions are surely predicted.
Source publication
In this study, we propose an interface to intuitively control robotic devices by using myoelectric signals detected from human users. In particular, we show experiments in which myoelectric signals measured from a forearm are used to control a robotic hand. When a user performs different motions (e.g., grasping and pinching), different myoelectric...
Context in source publication
Context 1
... proposed method is computationally efficient compared with the com- parisons because it learns the one model for multiple users and the adaptation procedure requires very little extra computation as presented in Eq. (8). Snapshots of the successful robotic hand control by a new user with our method are depicted in Fig.5. ...
Citations
... Castellini, Fiorilla et al. [14] observed that sEMG signals varied greatly between individuals and that models trained on diverse people are mainly subject specified; however, they demonstrated that a pre-trained model was effective in reducing the amount of time needed for an individual to become proficient with a prosthesis. Matsubara, Hyon et al. [15] split myoelectric signals into two independent parts named the motion-dependent part and the user-dependent part, such that models could be reused by rapidly learning only the user-dependent part for a new user. Sensinger, Lock et al. [16] proposed to concatenate the source and target data and showed various classification algorithms on it. ...
Hand motion intentions can be detected by analyzing the surface electromyographic (sEMG) signals obtained from the remaining forearm muscles of trans-radial amputees. This technology sheds new light on myoelectric prosthesis control; however, fewer signals from amputees can be collected in clinical practice. The collected signals can further suffer from quality deterioration due to the muscular atrophy of amputees, which significantly decreases the accuracy of hand motion intention recognition. To overcome these problems, this work proposed a transfer learning strategy combined with a long-exposure-CNN (LECNN) model to improve the amputees’ hand motion intention recognition accuracy. Transfer learning can leverage the knowledge acquired from intact-limb subjects to amputees, and LECNN can effectively capture the information in the sEMG signals. Two datasets with 20 intact-limb and 11 amputated-limb subjects from the Ninapro database were used to develop and evaluate the proposed method. The experimental results demonstrated that the proposed transfer learning strategy significantly improved the recognition performance (78.1%±19.9%, p-value < 0.005) compared with the non-transfer case (73.4%±20.8%). When the source and target data matched well, the after-transfer accuracy could be improved by up to 8.5%. Compared with state-of-the-art methods in two previous studies, the average accuracy was improved by 11.6% (from 67.5% to 78.1%, p-value < 0.005) and 12.1% (from 67.0% to 78.1%, p-value < 0.005). This result is also among the best from the contrast methods.
... Especially, the EMG signals have a user-dependent nature owing to crossuser physiological and anatomical variations. Many factors, such as the muscle geometry, skin impedance, fat tissue depth, and general strength may vary across different users, causing the measured signals, at the same electrodes positioning and for different users performing the same motion, to be largely different from each other [14], [24]- [27]. The above factors are likely to lead to significant performance compromise indicating that the previously trained model cannot be directly used by a new user. ...
Gestural interfaces based on surface electromyographic (sEMG) signal have been widely explored. Nevertheless, due to the individual differences in the sEMG signals, it is very challenging for a myoelectric pattern recognition control system to adapt cross-user variability. Unsupervised domain adaptation (UDA) has achieved unprecedented success in improving the crossdomain robustness, and it is a promising approach to solve the cross-user challenge. Existing UDA methods largely ignore the instantaneous data distribution during model updating, thus deteriorating the feature representation given a large domain shift. To address this issue, a novel framework is proposed to consist of a UDA model with a self-guided adaptive sampling (SGAS) strategy. This strategy is designed to utilize the domain distance in a kernel space as an indicator to screen out reliable instantaneous
samples for updating the classifier. Thus it enables improved alignment of feature representations of myoelectric patterns across users. To evaluate the performance of the proposed framework, sEMG data were recorded from forearm muscles of nine subjects performing six finger and wrist gestures. Experiment results show that the UDA method with the SGAS strategy achieved a mean accuracy of 90.41% ± 14.44% in a cross-user
classification manner, outperformed the state-of-the-art methods with statistical significance (p < 0.05). This study demonstrates the effectiveness of the proposed UDA framework and offers a novel tool for implementing cross-user myoelectric pattern recognition towards a multi-user and user-independent control.
... Hand gesture recognition from multi-channel forearm sEMG and myoelectric control has been widely studied in the past decade as a classification task [1][2][3]. Most related studies rely on signals collected from intact subjects, where the accessibility and quality of the signal can be ensured comparing to signals from amputees [4][5][6]. This proxy measure on the one hand contributes to the fast-technological updating in the field; however, on the other hand, may present overoptimistic classification performance [7]. ...
... First, transferring the known knowledge in a trained classifier to a new user (i.e., data were not involved in the training phase) can save the training time for the new user and improve prosthetic functionality. For example, adaptive learning was applied [4,5] to reuse the past information to improve the performance generalization. Second, to achieve higher and more consistent classification accuracy, Côté-Allard et al. [17,18] employed a Progressive Neural Networks architecture for transfer learning between participants, where each participant was considered as one source-task, and the average classification accuracy was improved up to 5.67% comparing to the same non-transfer approach. ...
Hand gesture recognition from multi-channel surface electromyography (sEMG) have been widely studied in the past decade. By analyzing muscle activities measured from forearm muscles, multiple hand gestures can be recognized. This technology can benefit upper-limb amputees in motion intention recognition, especially for those with trans-radial amputation, in terms of prosthesis control, rehabilitation and further human–computer interaction. However, due to the scarcity of signals collected from amputees, many related studies used signals from intact subjects as a proxy and result in overoptimistic classification performance. Comparing to sEMG signals from intact subjects, signals from upper-limb amputees suffer from signal quality deterioration which relates to the level of amputation and maybe other amputation information. Therefore, this study aims at improving the motion intention recognition performance in trans-radial amputated subjects. To tackle the challenges of data scarcity and signal quality deterioration, we propose a CNN-based transfer learning solution leveraging the knowledge learned from sEMG signals of intact subjects. The proposed method was developed from and tested with NinaPro database where 20 intact subjects and 11 amputees. We obtained 67.5% accuracy in the mDWT feature after transfer. And the results improved by 9.4% after transfer compared to no transfer in the RMS feature. In the end of the study, we further discussed the correlation between classification accuracy and amputation information including the percentage of remaining forearm and the number of years since amputation.
... Adaptive classifiers are first proposed in the upper-limb field based on sEMG signals in the studies [99][100][101]. Recently, researchers have adopted the adaptive design to the lower-limb prosthesis scope to adapt to sEMG pattern variations over time, caused by physical and physiological changes [73,102]. Du et al. have developed an adaptive locomotion mode recognition framework in dealing with gradual sEMG magnitude change [102]. ...
The lower-limb robotic prostheses can provide assistance for amputees’ daily activities by restoring the biomechanical functions of missing limb(s). To set proper control strategies and develop the corresponding controller for robotic prosthesis, a prosthesis user’s intent must be acquired in time, which is still a major challenge and has attracted intensive attentions. This work focuses on the robotic prosthesis user’s locomotion intent recognition based on the noninvasive sensing methods from the recognition task perspective (locomotion mode recognition, gait event detection, and continuous gait phase estimation) and reviews the state-of-the-art intent recognition techniques in a lower-limb prosthesis scope. The current research status, including recognition approach, progress, challenges, and future prospects in the human’s intent recognition, has been reviewed. In particular for the recognition approach, the paper analyzes the recent studies and discusses the role of each element in locomotion intent recognition. This work summarizes the existing research results and problems and contributes a general framework for the intent recognition based on lower-limb prosthesis.
... Therefore, gestures based natural user interface have been experimented in various scenarios, e.g., virtual reality environments [11], robotic fish control [12], magnetic levitation [13] and robotic interaction [14]. Figure 1 shows generalized hand gestures that are used in systems that require interaction between humans and machines, [15]. ...
... Drones or unmanned aerial vehicles are being used for various purposes like aerial photography or videography, surveillance, military tasks, transportation, acrobatics, etc. UAV's [15] play an important role in various applications like search and rescue, surveillance, entertainment, etc. [1]. To achieve the required task perfectly, skilled pilot is required with a controlled input method to drive the drone. ...
... In order to test how a classifier behaves for different users most research uses the leave-one-out approach where information is gathered from many subjects and subsequently the classifier is trained with data from all but one subject and tested on this specific subject (Matsubara et al., 2011;Gibson et al., 2013;Ison and Artemiadis, 2013;Matsubara and Morimoto, 2013;Guo et al., 2015;Park et al., 2016;Stival et al., 2016). Gibson et al. (2013) gathered data from seven users and for the evaluation of performance of the classifier for each user they used a decision tree that uses variable thresholds trained on the data gathered from all the subjects except the one they were investigating. ...
... They performed a comparative study between their topology preserving domain adaptation method with eight other methods from literature or variations of them and their suggested system outperformed them all to address subject based variability. Matsubara et al. (2011) proposed a bilinear model that decomposes the EMG signal into two linear factors, one that is user dependent and one motion dependent and use the latter factor as user-independent features. They use information from multiple users, but in contrast to Orabona et al. (2009) they train and hold in memory a single bilinear model. ...
... Research focused only recently on multi-subject prostheses, hence there is a limited number of real-time experiments (Matsubara et al., 2011;Matsubara and Morimoto, 2013;Guo et al., 2015;Stival et al., 2016). Moreover, the majority of the experiments involves able-bodied subjects and not amputees. ...
Surface Electromyography (EMG)-based pattern recognition methods have been investigated over the past years as a means of controlling upper limb prostheses. Despite the very good reported performance of myoelectric controlled prosthetic hands in lab conditions, real-time performance in everyday life conditions is not as robust and reliable, explaining the limited clinical use of pattern recognition control. The main reason behind the instability of myoelectric pattern recognition control is that EMG signals are non-stationary in real-life environments and present a lot of variability over time and across subjects, hence affecting the system's performance. This can be the result of one or many combined changes, such as muscle fatigue, electrode displacement, difference in arm posture, user adaptation on the device over time and inter-subject singularity. In this paper an extensive literature review is performed to present the causes of the drift of EMG signals, ways of detecting them and possible techniques to counteract for their effects in the application of upper limb prostheses. The suggested techniques are organized in a table that can be used to recognize possible problems in the clinical application of EMG-based pattern recognition methods for upper limb prosthesis applications and state-of-the-art methods to deal with such problems.
... Several studies continued in this direction with different strategies to build more robust models that take advantage of past information from source subjects or, in the context of repeatability, from the target itself. Matsubara et al. [8] proposed to separate myoelectric data in user-dependent and motion-dependent components, and to reuse models by quickly learning just the user-dependent component for new subjects. Sensinger et al. [9] presented different methods based on an appropriate concatenation of target and source data. ...
A number of studies have proposed to use domain adaptation to reduce the training efforts needed to control an upper-limb prosthesis exploiting pre-trained models from prior subjects. These studies generally reported impressive reductions in the required number of training samples to achieve a certain level of accuracy for intact subjects. We further investigate two popular methods in this field to verify whether this result equally applies to amputees. Our findings show instead that this improvement can largely be attributed to a suboptimal hyperparameter configuration. When hyperparameters are appropriately tuned, the standard approach that does not exploit prior information performs on par with the more complicated transfer learning algorithms. Additionally, earlier studies erroneously assumed that the number of training samples relates proportionally to the efforts required from the subject. However, a repetition of a movement is the atomic unit for subjects and the total number of repetitions should therefore be used as reliable measure for training efforts. Also when correcting for this mistake, we do not find any performance increase due to the use of prior models.
... They proposed a cross-subject analysis to show that reusing the pre-trained models from former subjects can shorten the training time for a new user. Matsubara et al. [14] proposed to extract a user-independent component from EMG data that is considered transferable across subjects. ...
Non-invasive myoelectric prostheses require a long training time to obtain satisfactory control dexterity. These training times could possibly be reduced by leveraging over training efforts by previous subjects. So-called domain adaptation algorithms formalize this strategy and have indeed been shown to significantly reduce the amount of required training data for intact subjects for myoelectric movements classification. It is not clear, however, whether these results extend also to amputees and, if so, whether prior information from amputees and intact subjects is equally useful. To overcome this problem, we evaluated several domain adaptation algorithms on data coming from both amputees and intact subjects. Our findings indicate that: (1) the use of previous experience from other subjects allows us to reduce the training time by about an order of magnitude; (2) this improvement holds regardless of whether an amputee exploits previous information from other amputees or from intact subjects.
... After acquiring some data from the new subject, this approach identifies one or several similar subjects in the training data and uses them as a starting point for adaptation to the new user. The method is based on previous work by Matsubara et al. [52], who combined task-and user-specific information in a stylistic myoelectric model. ...
Modern wearable robots are not yet intelligent enough to fully satisfy the demands of end-users, as they lack the sensor fusion algorithms needed to provide optimal assistance and react quickly to perturbations or changes in user intentions. Sensor fusion applications such as intention detection have been emphasized as a major challenge for both robotic orthoses and prostheses. In order to better examine the strengths and shortcomings of the field, this paper presents a review of existing sensor fusion methods for wearable robots, both stationary ones such as rehabilitation exoskeletons and portable ones such as active prostheses and full-body exoskeletons. Fusion methods are first presented as applied to individual sensing modalities (primarily electromyography, electroencephalography and mechanical sensors), and then four approaches to combining multiple modalities are presented. The strengths and weaknesses of the different methods are compared, and recommendations are made for future sensor fusion research.
... This issue calls for machine learning techniques able to boost the learning process of each user. Adaptive methods (Chattopadhyay et al., 2011;Matsubara et al., 2011;Tommasi et al., 2013), i.e., methods able to exploit knowledge gathered from previous experience to accelerate learning by a new subject-are suitable for this task. Indeed, the experience gained over several source subjects can be leveraged to reduce the training time of a new target user. ...
... One general issue pointed out by previous work is the timeand user-dependent nature of the sEMG signals (Sensinger et al., 2009;Matsubara et al., 2011). The first is mainly due to fatigue or electrode displacement, while causes of the second are the personal quantity of sub-cutaneous fat, skin impedance, and differences in muscle synergies. ...
... Still, adaptive techniques have been applied only marginally on this problem. In Matsubara et al. (2011), the authors suggest extracting from the sEMG data a user-independent component that can be transferred across subjects. The source and target data coming from different persons can also be combined together after re-weighting as proposed by Chattopadhyay et al. (2011). ...
One of the hottest topics in rehabilitation robotics is that of proper control of prosthetic devices. Despite decades of research, the state of the art is dramatically behind the expectations. To shed light on this issue, in June, 2013 the first international workshop on Present and future of non-invasive peripheral nervous system (PNS)-Machine Interfaces (MI; PMI) was convened, hosted by the International Conference on Rehabilitation Robotics. The keyword PMI has been selected to denote human-machine interfaces targeted at the limb-deficient, mainly upper-limb amputees, dealing with signals gathered from the PNS in a non-invasive way, that is, from the surface of the residuum. The workshop was intended to provide an overview of the state of the art and future perspectives of such interfaces; this paper represents is a collection of opinions expressed by each and every researcher/group involved in it.
