ArticlePDF Available

Learning Hand Kinematics for Parkinson's Disease Assessment Using a Multimodal Sensor Glove

Wiley
Advanced Science
Authors:

Abstract and Figures

Hand dysfunctions in Parkinson's disease include rigidity, muscle weakness, and tremor, which can severely affect the patient's daily life. Herein, a multimodal sensor glove is developed for quantifying the severity of Parkinson's disease symptoms in patients' hands while assessing the hands' multifunctionality. Toward signal processing, various algorithms are used to quantify and analyze each signal: Exponentially Weighted Average algorithm and Kalman filter are used to filter out noise, normalization to process bending signals, K-Means Cluster Analysis to classify muscle strength grades, and Back Propagation Neural Network to identify and classify tremor signals with an accuracy of 95.83%. Given the compelling features, the flexibility, muscle strength, and stability assessed by the glove and the clinical observations are proved to be highly consistent with Kappa values of 0.833, 0.867, and 0.937, respectively. The intraclass correlation coefficients obtained by reliability evaluation experiments for the three assessments are greater than 0.9, indicating that the system is reliable. The glove can be applied to assist in formulating targeted rehabilitation treatments and improve hand recovery efficiency.
Content may be subject to copyright.
RESEARCH ARTICLE
www.advancedscience.com
Learning Hand Kinematics for Parkinson’s Disease
Assessment Using a Multimodal Sensor Glove
Yu Li, Junyi Yin, Shuoyan Liu, Bing Xue, Cyrus Shokoohi, Gang Ge, Menglei Hu,
Tenghuan Li, Xue Tao, Zhi Rao, Fanye Meng, Hongfeng Shi, Xiaoqiang Ji,*
Peyman Servati, Xiao Xiao,* and Jun Chen*
Hand dysfunctions in Parkinson’s disease include rigidity, muscle weakness,
and tremor, which can severely affect the patient’s daily life. Herein, a
multimodal sensor glove is developed for quantifying the severity of
Parkinson’s disease symptoms in patients’ hands while assessing the hands’
multifunctionality. Toward signal processing, various algorithms are used to
quantify and analyze each signal: Exponentially Weighted Average algorithm
and Kalman filter are used to filter out noise, normalization to process
bending signals, K-Means Cluster Analysis to classify muscle strength grades,
and Back Propagation Neural Network to identify and classify tremor signals
with an accuracy of 95.83%. Given the compelling features, the flexibility,
muscle strength, and stability assessed by the glove and the clinical
observations are proved to be highly consistent with Kappa values of 0.833,
0.867, and 0.937, respectively. The intraclass correlation coefficients obtained
by reliability evaluation experiments for the three assessments are greater
than 0.9, indicating that the system is reliable. The glove can be applied to
assist in formulating targeted rehabilitation treatments and improve hand
recovery efficiency.
1. Introduction
Parkinson’s disease (PD) is the leading fastest-growing neurode-
generative disease around the world.[1] The prevalence of PD
Y. Li, T. Li, X. Tao, Z. Rao, F. Meng, X. Ji
School of Life Science and Technology
Changchun University of Science and Technology
Changchun 130022, P. R. China
E-mail: jixq2012@cust.edu.cn
J. Yin, C. Shokoohi, X. Xiao, J. Chen
Department of Bioengineering
University of California, Los Angeles
Los Angeles, CA 90095, USA
E-mail: xiao.xiao@u.nus.edu; jun.chen@ucla.edu
The ORCID identification number(s) for the author(s) of this article
can be found under https://doi.org/10.1002/advs.202206982
© 2023 The Authors. Advanced Science published by Wiley-VCH GmbH.
This is an open access article under the terms of the Creative Commons
Attribution License, which permits use, distribution and reproduction in
any medium, provided the original work is properly cited.
DOI: 10.1002/advs.202206982
over the age of 60 is 1%,[2] which costs
healthcare $20 billion annually in the
United States.[3] The main clinical man-
ifestations of the PD patients’ hands in-
clude static tremor, muscle weakness, and
rigidity.[4] More than 70% of PD patients
suffer from static tremor, which is a hall-
mark of PD.[2] Unfortunately, there is no
uniform standard test or biomarker to
track the disease progression as the symp-
toms and signs of PD vary with each
individual.[5–7 ] Clinical scales are the pri-
mary method for physicians to examine
and evaluate the severity of the neurological
signs (such as movement inflexibility, hand
muscle rigidity, and tremors) that patients
show.[8–10] Conventional clinical scales in-
clude the total active movement (TAM),[11]
manual muscle testing,[12] unified Parkin-
son’s disease rating scale (UPDRS),[13] and
the movement disorder society criteria.[14]
However, this method is susceptible to the
patient’s status during the clinical visit,
and to the variability in individual disease
characteristics.[15] And these scales are imprecise, and lack the
ability for the physician to have an objective response during
the assessment process, which makes monitoring and assessing
PD patients’ hand functions challenging, especially in the milder
S. Liu, B. Xue
Department of Materials Science and Engineering
National University of Singapore
Singapore 117583, Singapore
G. Ge, X. Xiao
Department of Electrical and Computer Engineering
National University of Singapore
Singapore 117583, Singapore
M. Hu, P. Servati
Department of Electrical and Computer Engineering
University of British Columbia
Vancouver, BC V6T1Z4, Canada
H. Shi
China–Japan Union Hospital of Jilin University
Changchun 130033, P. R. China
Adv. Sci. 2023,10, 2206982 © 2023 The Authors. Advanced Science published by Wiley-VCH GmbH
2206982 (1 of 10)
www.advancedsciencenews.com www.advancedscience.com
early stages.[16,17 ] Various efforts have been made in the past to
develop suitable flexible data gloves for continuous and objective
monitoring and assessment of hand function.[18–21 ]
Wearable sensors hold attractive features such as wearing com-
fort, excellent compliance, low cost, and light weight.[21–25 ] The
combination of wearable sensors and textiles with outstanding
air permeability could provide favorable conditions for flexible
gloves to track human movement states (joint motion, muscle
strength, physiological tremor, and pathological tremor).[18,26-31]
With the support of machine learning feature extraction, sub-
tle features hidden in complex signals can be recognized, and
utilized to implement gesture recognition, and hand function
assessment.[19,25,31-40 ] Existing wearable gloves offer high accu-
racy in monitoring hand movements (Table S1, Supporting
Information),[19,21,41-54 ] yet diverse challenges remain to be re-
solved. On the one hand, since the majority of research only uti-
lizes a single sensor to capture limited indexes to characterize
hand function, this situation is farfetched to characterize the di-
verse motion characteristics of the hand.[18-20,22,34,44 ] On the other
hand, researchers were more interested in capturing hand move-
ment information and neglected to incorporate scales for clinical
assessment.[34,38,44,47,48 ] Combining objective measurements and
clinical scales can promote the assessment of hand dysfunction,
and guide physicians in the development of treatment plans for
hand rehabilitation.[43,45,46 ]
Herein, we report a multimodal sensor-based textile glove for
monitoring and assessing PD patients’ hand function. The glove
comprehensively obtains information of various hand kinemat-
ics through a built-in flexible sensor system. Specifically, flexible
sensors measure the bending angle of the fingers and the force
of fingers and palm applied on objects. Combined with hand
tremor signals fed back by accelerometers, our glove analyzes the
participants’ hand functions in an objective and exhaustive man-
ner with comfortable wearable experiences for patients. Further-
more, a large scale of data-containing hand kinematics was cap-
tured from 40 participants using the multimodal sensor glove,
which was shown capable of quantitatively assessing finger flex-
ibility, hand muscle strength, and hand stability with the aid of
filtering, normalization, cluster analysis, and neural network. Fi-
nally, we proved the consistency of the data by comparing the as-
sessment results and clinical evaluation with Kappa values, and
we gathered a visualization of the results with human–machine
interaction (HCI) equipment. Our remarkable system that com-
bines comfort experience and comprehensive functions, offers a
distinctive universal approach to long-existing challenges in hand
function assessment in PD patients. In addition to aiding in for-
mulating rehabilitation treatments, our glove can objectively as-
sess patients’ progress following hand rehabilitation training and
direct physicians in making prompt treatment adjustments.
2. Results and Discussions
2.1. Design of the Multimodal Sensor Glove
The primary components of the system are the sensor system,
the primary control module, and the PC terminal. A proper se-
lection of sensors is crucial for achieving comprehensive hand
function assessments. The chosen sensor system must be capa-
ble of accurately capturing various hand kinematic information
while meeting requirements for comfort and flexibility. Embed-
ding too many rigid sensors not only increases the cost of the
glove but may also limit patients’ freedom of movement, thus re-
ducing comfort and convenience. Flexible sensors can effectively
solve this problem by sensing multiple physical quantities within
a small area and better adapting to the shape and movement of
the hand. Flexible bending sensors, with a resistance range of
10–110 kΩand a resistance tolerance of ±30%, were selected
to measure finger bending angle information. Flexible thin-film
pressure sensors, with a sensitivity of 10 g and an accuracy of
<5%, were chosen to measure changes in hand muscle strength.
Inertial Measurement Unit (IMU) sensors were chosen to mea-
sure the acceleration signal generated by hand tremors due to
their high precision and programmable acceleration range char-
acteristics (ranging from ±2g,±4g,±8g,to±16 g, with gde-
noting gravity acceleration). In addition, the microcontroller unit
(MCU) selected can meet the interface and processing speed re-
quirements of the system.
The proposed multimodal sensor glove features a two-layer
structure with an outer layer protecting the sensors from the en-
vironment and an inner layer offering the sensors positioning
and support. The layout of the sensors is based on pathological
investigations of the PD patient’s hands. To quantify hand mus-
cle strength values, we situated the flexible thin-film pressure
sensors in the palm of the multimodal sensor glove, correspond-
ing to specific human hand positions like palm, purlicue, distal
phalanx, and middle phalanx (Figure 1a,b). Five bending sensors
are placed on the back of the glove’s fingers (Figure 1a,b) to cap-
ture the bending variations of fingers. In addition, the IMU on
the main control circuit board (Figure 1b; Figure S1, Support-
ing Information) is fitted to the back of the multimodal sensor
glove (Figure 1a,b) to measure the acceleration caused by hand
tremors.
Figure 1c depicts the main control circuit board for the mul-
timodal sensor glove system. The main board incorporates the
minimum system of the MCU, the bend sensor module circuit,
the pressure sensor module circuit, the IMU sensor and its pe-
ripheral circuit, and the power supply module (Figures S2–S6,
Supporting Information). In addition, physical interfaces are re-
served on the main control circuit board for bending and pres-
sure sensors to facilitate data transfer and subsequent data pro-
cessing. Figure 1d demonstrates the hand function assessment.
Quantitative processing and analysis are performed based on the
collected hand kinematics data. After appropriate preprocessing
(amplification and filtering) steps, each signal is quantitatively
analyzed. Bending signals are normalized to simplify the grading
mechanism, while hand muscle strength signals are sorted using
the cluster analysis method. Then, a Back Propagation Neural
Network (BPNN) is applied to identify and classify tremor sig-
nals. The hand function assessment results are then determined
by comparing the assessment data to the clinical scale, followed
by visualization through HCI interface (Figure S7, Supporting
Information).
2.2. Finger Flexibility Assessments
Hand flexibility is determined by the flexion of phalanges, which
are influenced by the mobility of synovial joints within the
Adv. Sci. 2023,10, 2206982 © 2023 The Authors. Advanced Science published by Wiley-VCH GmbH
2206982 (2 of 10)
www.advancedsciencenews.com www.advancedscience.com
Figure 1. Multisensor-based hand function assessment multimodal sensor glove. a,b) A depiction of the inner layering of the multimodal sensor glove
and the layout of each sensor. Flexible film pressure sensors are located on the palmer side of the glove, with detection points corresponding to the palm
of the hand, purlicue, middle phalanges, and distal phalanges. Flexible bending sensors are located on each of the dorsal phalanges. IMU and MCU are
integrated onto the dorsal side of the glove. c) The 3D diagram of the main control circuit board, including the physical interfaces connected to MCU,
IMU, bending sensors, and thin film pressure sensors. d) Hand function assessment process and HCI. The hand function assessment process includes
collecting, processing, and analyzing hand kinematic signals.
metacarpophalangeal and proximal/distal interphalangeal joints.
Disorders of the nervous system caused by PD can lead to symp-
toms such as joint rigidity and muscle disuse atrophy, which
results in decreased flexibility and range of motion of the fin-
gers due to inflammation and fibrosis of the synovial joints. We
collected participants’ finger bend signals via the flexible bend-
ing sensor module and quantitatively analyzed the differences in
hand flexibility.
During the hand flexibility assessment (Figure 2a), each sub-
ject wore the multimodal sensor glove and completed nine ges-
tures “Flat hand”,“Fist”,“OK”,“Orchid fingers”,“A”,“W”,“B”,“U”,
and “V” in sequence according to the prescribed protocol. Each
gesture was performed ten times with a 3-s interval to reduce
measurement errors caused by finger fatigue. The multimodal
sensor glove collected the finger-bending data once each gesture
was completed. Figure 2b and Table S2 (Supporting Information)
show gesture completion by two subjects (one patient and one
healthy subject), indicating a significant difference in the bend-
ing angle of fingers between the two subjects. It can be seen
that the patient with PD suffers hand dysfunction in both flex-
ion and stretching of the fingers. For the “Flat hand” gesture, the
patient’s finger curvature is 100 times than that of the healthy
subject’s, indicating that the patient’s fingers face certain obsta-
cles in stretching. For the “Fist” gesture, the patient’s finger cur-
vature is 0.5 times than that of the healthy subject’s, indicat-
ing that the patient’s fingers have dysfunction in flexion. For the
“OK” gesture, the bending angles of the patient’s thumb and in-
dex fingers are 0.5 times than that of the healthy subject’s, and
the bending angles of the patient’s middle, ring, and little fingers
are about ten times than that of the healthy subject’s, indicating
that the patient’s fingers have dysfunction in flexibility and coor-
dination. The difference in finger flexibility between the two sub-
jects is as apparent when comparing other gestures. Therefore,
it can be concluded that the patient suffers from dysfunction in
hand flexibility compared to the healthy subject.
Figure 2c illustrates a schematic diagram of the multimodal
sensor glove system, with the hand flexibility grades of each
subject given in Table S3 (Supporting Information). To compare
the consistency of the flexibility assessment results with clinical
observations, the clinical scale used by doctors for hand flex-
ibility assessments was also applied to the subjects (Table S3,
Supporting Information). The distribution of grading results
Adv. Sci. 2023,10, 2206982 © 2023 The Authors. Advanced Science published by Wiley-VCH GmbH
2206982 (3 of 10)
www.advancedsciencenews.com www.advancedscience.com
Figure 2. Finger flexibility assessment. a) The process of the flexibility assessment. b) The finger bending angle of two subjects (patient and healthy
subject) when completing nine assessment gestures (“Flat hand”, “Fist”, “OK”, “Orchid fingers”, “A”, “W”, “B”, “U”, “V”). c) Schematic diagram of
the hand function assessment for 12 subjects. d) Heat map of hand flexibility assessment grades distribution given by the system and the doctor,
respectively. The row lists grades given by the system, while the column lists grades given by the doctor.
Adv. Sci. 2023,10, 2206982 © 2023 The Authors. Advanced Science published by Wiley-VCH GmbH
2206982 (4 of 10)
www.advancedsciencenews.com www.advancedscience.com
of 12 subjects given by our system and doctors is analyzed and
further illustrated in a heat map (Figure 2d), indicating that the
grade F4 is the largest, accounting for 66.7%. About 8.33% of
total cases reports differences between the grading results given
by our system and doctors. The results indicate that the flexibility
grades of subjects 1 to 8 (healthy subjects) are distributed in the
grade F4, and the flexibility grades of subjects 9 to 12 (patients)
are distributed between F3 and F2. The final Kappa value calcu-
lated by the Kappa consistency test is 0.833 (Z-score of 3.906, *P
<0.05, **P<0.01), indicating that the flexibility grades given by
the system and clinical observations are highly consistent.
2.3. Hand Muscle Strength Assessments
Early symptoms of PD such as hand muscle weakness, rigid-
ity, and muscle pain, are often mistaken as common represen-
tations of aging. However, with the aggravation of the diseases,
the nerve conduction velocity of the patient’s upper extremity de-
creases due to demyelination of the nerves, gradually weakening
the sensory function of the median and ulnar nerves. Therefore,
the hand muscle strength test is necessary for judging the extent
of damage to hand muscles and nerves, which is significant in
monitoring the progression of PD. To assess the patient’s muscle
strength status, we collected muscle signals during hand touch-
ing objects via the flexible thin-film pressure sensor, followed by
a series of quantitative analyses to measure the damage level of
muscle strength.
During the muscle strength assessment, each subject wore
the multimodal sensor glove and completed four groups of cus-
tomized actions (Figure 3a,b). We required all subjects to perform
each action ten times and ensured that they had a 3-s rest period
to improve the accuracy of the measurement results. Figure 3c
shows the corresponding muscle strength values of 12 subjects
who completed these actions. Significant differences in hand
muscle strength values are shown between patients and healthy
subjects. However, the action of “grasp the cylinder” requires the
most physical contact between the hand and the sensors, demon-
strating the maximum values of measured muscle force. Table S4
(Supporting Information) indicates the specific muscle strength
of each subject when completing each action. Subject 9 exhibited
weakness when performing the actions of “grasp”, “pinch”, and
“click”, showing significantly lower corresponding muscle force
values. Subjects 10, 11, and 12 demonstrated different extents of
strength dysfunction when completing the actions.
Combined with the interval partitioning results of cluster anal-
ysis (Figure 3d) and the Lovett grading standard, the hand muscle
strength of the subjects was assessed with the maximum mus-
cle strength values collected. Table S5 (Supporting Information)
shows the muscle strength value of each subject. To compare the
consistency of the hand muscle strength assessment results with
clinical observations, the clinical scale was applied for hand mus-
cle strength assessment by doctors (Table S5, Supporting Infor-
mation). The distribution and proportion of grading results were
analyzed and further illustrated in a heat map (Figure 3e), show-
ing that the number of subjects with the hand muscle strength
grade M4 is the largest (50%). About 8.33% of the cases report
differences between the grading results. The results show that
the hand muscle strength grades of subjects 1 to 8 (healthy sub-
jects) are distributed between M4 and M5, and the hand muscle
strength grades of subjects 9 to 12 (patients) are distributed be-
tween M0 and M2. The final Kappa value calculated by the Kappa
consistency test is 0.867 (Z-score of 4.637, *P<0.05, **P<0.01),
indicating highly consistent hand muscle strength grades given
by our system and clinical observations.
2.4. Hand Stability Assessments
The most prevalent tremor in early PD is a unilateral finger
rolling movement, which then evolves into uncontrollable, regu-
lar tremors of the ipsilateral or contralateral limbs in a stationary
condition. Although hand tremors will not cause direct harm
from a physiological standpoint, they create significant disrup-
tion in everyday living. As a result, measuring tremor signals
is critical in rehabilitating patients’ hand functions. Recently,
accelerometers have been one of the most common methods
for measuring tremor signals, and machine learnings are also
frequently applied to identify and classify tremors.[55,56 ] To
quantify the tremor degree of the patients’ hands, we collected
acceleration signals via IMU sensor, followed by extracting
characterizations (amplitude, standard deviation, frequency,
etc.). The tremor data were then classified by using BPNN[57]
and tenfold cross-validation,[58] which was used to assess the
patients’ hand stability.
Figure 4a shows the stability assessment. In the test, based
on the Static Tremor section in UPDRS, each of the 40 subjects
was required to wear the multimodal sensor glove and complete
a palm-down motion to remain the hand horizontally for 5 s.
The doctors scored the grade of hand tremor of each subject.
We labeled the samples as “0”, “1”, and “2” for “No tremor”,
“Mild tremor”, and “Severe tremor”, respectively. Furthermore,
to avoid the overfitting phenomenon, the original sample of 40
groups was expanded to 120 groups (Table S6, Supporting In-
formation). Specifically, we designed a simulated tremor exper-
iment in which 32 healthy subjects were required to randomly
simulate different grades of tremor at frequencies ranging from
3 to 8 Hz according to the tremor motion characteristics. In ad-
dition, the acceleration data generated by the hand at different
stages were also recorded in eight PD patients.
We performed feature extraction on 120 sets of samples. And
the obtained feature values and the labels labeled by the doctors
were used to train the machine-learning model. Then, we parti-
tioned all experimental samples into training and test sets with
8:2, and used tenfold cross-validation to determine the optimal
hyperparameters of BPNN, Support Vector Machine, K Nearest
Neighbors, and Decision Tree classifiers (Tables S7–S10, Sup-
porting Information). Compared with other models above, BPNN
with the number of hidden nodes of six obtained a higher accu-
racy of 94.78% (Table S7 and Figure S8, Supporting Information).
Finally, we verified the generalization ability of each classi-
fier on the test sets, and the comparison results showed that
the BPNN got the highest accuracy (95.83%), which is shown
in Figure 4b and Table S11 (Supporting Information). In addi-
tion, Figure 4d shows the loss function of the BPNN training,
the final trained loss function value is approaching 0.1. And we
evaluated the classification effectiveness of the classifier using
receiver operating characteristic (ROC) curves (Figure 4e). The
Adv. Sci. 2023,10, 2206982 © 2023 The Authors. Advanced Science published by Wiley-VCH GmbH
2206982 (5 of 10)
www.advancedsciencenews.com www.advancedscience.com
Figure 3. Hand muscle strength assessment. a) The process of the muscle strength assessment. b) Four sets of customized assessment actions: i) grasp
the cylinder, ii) pinch objects with fingertip, iii) grasp the ball, and iv) click objects with finger. c) The muscle strength status of 12 subjects completing
customized actions. d) Interval partitioning results of cluster analysis. The ordinate represents grades, while the abscissa represents the range of the
interval, given the specific values of the interval boundaries. e) Heat map of muscle strength assessment grades distribution given by the system and
the doctor, respectively. The row lists grades given by the system, while the column lists grades given by the doctor.
results show that the classifier performs well. The trained confu-
sion matrix is shown in Figure S9 (Supporting Information). We
analyzed the values of precision, recall, and F1 score. Experimen-
tal results show a good classifier performance in precision and
recall in the “No tremor”, “Mild tremor,” and “Severe tremor” cat-
egories. Specifically, the precision and recall of the “No tremor”
category are both 100%; the precision of the “Mild tremor” cate-
gory is 100%, and the recall is 90%; the precision of the “Severe
tremor” category is 85.71%, the recall is 100%. Considering both
recall and precision, we calculated the F1 scores of each category.
The F1 scores for the categories “No tremor”, “Mild tremor”, and
“Severe tremor” were 100%, 94.74%, and 92.31%, respectively,
indicating a good classifier performance across all classification
categories. The classifier is very accurate in classifying samples
into the correct category. Furthermore, a higher Kappa value is
obtained by the Kappa consistency test, which is 0.937 (Z-score
of 6.473, *P<0.05, **P<0.01), indicating a strong consistency
between the stability assessment and the doctor’s assessment.
2.5. System Reliability Analysis
We conducted three repeated reliability analysis experiments to
comprehensively validate the reliability of the multimodal sen-
sor glove for assessing hand dysfunction. The reliability analysis
experiments include finger flexibility assessment, hand muscle
strength assessment, and hand stability assessment. 12 subjects
(8 healthy subjects and 4 patients) were required to complete
Adv. Sci. 2023,10, 2206982 © 2023 The Authors. Advanced Science published by Wiley-VCH GmbH
2206982 (6 of 10)
www.advancedsciencenews.com www.advancedscience.com
Figure 4. Hand stability assessment. a) The process of the stability assessment. b) Comparison of different methods in terms of accuracy. c) Schematic
diagram of the BPNN: nine input layer nodes, six hidden layer nodes, and three output layer nodes. d) Loss function of BPNN training. e) ROC curves
for stability assessment based on BPNN. The horizontal coordinate of the ROC curve is not correlated with the vertical coordinate, and the closer the
ROC curve is to the (0, 1) point, the better the model is represented. Area under curve (AUC) is the area enclosed by the ROC curve and the x-axis. The
value of AUC can be used to measure the goodness of the classifier, and the higher the AUC means the better the classification effect.
three experiments sequentially, each repeating ten times in the
same time period. To avoid subject fatigue during experiments,
we provided a 1-min rest period between each experiment, and
the rest duration was adjusted according to the subjects’ condi-
tion. The intraclass correlation coefficient (ICC) model calculated
the ICC value and 95% confidence interval of each repeated ex-
periment. The results, presented in Table S12 (Supporting Infor-
mation), showed that the ICC values for finger flexibility, hand
muscle strength, and hand stability assessments were 0.923, 0.91,
and 0.946, respectively. Overall, the ICC values of the multimodal
sensor glove system were greater than 0.9, indicating its high re-
liability in evaluating hand dysfunctions. The multimodal sensor
glove provides an objective, effective, and comprehensive tool for
clinical evaluation and objective quantitative data for subsequent
rehabilitation treatments.
3. Conclusion
A multisensor-based multimodal glove for PD patients has been
fabricated, with high comfort, low cost, and variable size charac-
teristics. The multimodal sensor glove captures and analyzes the
motion signs of rigidity, muscle weakness, and tremor in the pa-
tient’s hand, which bring a breakthrough in assessing the hand
function of PD patients. To examine the subjects’ hand flexibil-
ity and coordination levels, different gestures were designed for
one-fingered, two-fingered, and five-fingered situations. We de-
signed hand grasping, pinching, and clicking actions for specific
hand parts like finger pulp, fingertips, palm, and purlicue to il-
lustrate the hand muscle strength condition. The hand tremor
grade was examined by analyzing the Static Tremor section in
UPDRS. Our results indicated that the multimodal sensor glove
recorded the signals on bending angles, muscle strength values,
and acceleration generated by tremors when completing desig-
nated gestures/actions. Graded results of hand dysfunction as-
sessment were generated after analyzing the above signals, which
can be used to assist in assessing the development stages of the
disease. Furthermore, the reliability of the system for finger flex-
ibility, hand muscle strength and hand stability assessment was
confirmed by repeated reliability evaluation experiments, respec-
tively, indicating that the system has excellent reliability.
The proposed multimodal sensor glove demonstrates the abil-
ity to monitor and assess signals on hand bending angles, muscle
Adv. Sci. 2023,10, 2206982 © 2023 The Authors. Advanced Science published by Wiley-VCH GmbH
2206982 (7 of 10)
www.advancedsciencenews.com www.advancedscience.com
strength values, and tremor grades in real-world scenarios by
utilizing a multisensor system. The assessment techniques and
methods are well aligned with current clinical rehabilitation
needs, addressing the noncomprehensive nature of current
hand function assessment devices. Various useful signals and
graded results on the patient’s hand function are provided,
assisting doctors in formulating rehabilitation treatment plans
for the disease. In addition, the future topic on improving the
functionality of the multimodal sensor glove includes developing
procedures combining hand function assessment and the reha-
bilitation process. A detailed study on the relationship between
various kinematic parameters of each finger joint during hand
movements is required to further facilitate a comprehensive
hand function assessment.
4. Experimental Section
Fabrication of Electronics:Flexible printed circuit (FPC) was patterned
using a UV laser system (LPKF; Protolaser U4) for mechanical support
and electrical interconnections of electronic components. The FPC has
dimensions of 43 mm * 38 mm and contains passive components (re-
sistors and capacitors), microcontroller (STM32F103C8T6), DC–DC con-
verter (AMS1117-3.3 V), operational amplifier (LM358), and IMU (MPU-
9250). The IMU is connected to the FPC through a low-temperature reflow
process with a soldering paste and a heat gun. Flex 4.5flexible bending
sensors and ZNS-01 flexible thin-film pressure sensor are electrically con-
nected by soldering thin copper wires with the FPC reserved interface, re-
spectively.
Ethics Information and Study Design:This study was approved by the
ethics committees of China–Japan Union Hospital of Jilin University (Clin-
ical Research Review No. 20221124002). All participants gave written in-
formed consent for the research to track and analyze information on vari-
ous hand kinematics. Inclusion criteria of patients: 1) diagnosed with PD,
2) age above 60, and 3) has enough cognitive and verbal ability to under-
stand experiments and follow instructions. Exclusion criteria: 1) behavioral
and cognitive impairment and/or low adherence, and 2) diagnosed with
other neurological diseases. Eight eligible patients with different severity
of hand dysfunctions were recruited in this study. Their symptoms contain
but were not limited to rigidity, inflexibility, and tremor, with an average
nine-year illness time. In addition, 32 healthy subjects with no neurologi-
cal or other related diseases were control groups (Table S13, Supporting
Information). Eight healthy subjects and four patients with PD were cho-
sen for hand flexibility and muscle strength assessment tests. Based on
the sample size, all 40 subjects took the stability assessment test. Follow-
ing the test requirement or instruction, each subject was asked to perform
corresponding gestures or actions, which the system then collected, pro-
cessed, and analyzed the gesture/action data to complete the hand func-
tion assessment.
Finger Flexibility Assessment:As bend signals were analog, low-
frequency signals, they must first be processed by analog to digital conver-
sion (ADC) and amplified to acquire target data for the system to analyze.
To eliminate the noise mixed in with the target signal, the Exponentially
Weighted Average algorithm was applied to filter signals. The algorithm
banished the negative effect over responseof electronic devices and greatly
neutralized the beating of the data caused by the environmental distur-
bance (Figure S10, Supporting Information). Furthermore, the min–max
standard method was applied to normalize collected bending data, which
varied by the individual difference in a range of 0°to 180°. After mapping
the bending data to the region of (0, 1), the primary dimension of the
signals was eliminated, and the data were dimensionless, which simpli-
fied subsequent rating work. Table S14 (Supporting Information) shows
the normalized bending angle range with its corresponding grade. A to-
tal of five grades (F0 to F4, where F0 indicates the worst flexibility) were
refined based on the four grades of the TAM assessment standard (Ta-
ble S15, Supporting Information). The fingers’ ability of flexion and exten-
sion was then explored through the designed gestures (“Flat hand”, “Fist”,
“OK”, “Orchid fingers”, “A”, “W”, “B”, “U”, “V”). It is worth noting that
the maximum bending value obtained through several measurements and
computations would be used for the following ranking task. Based on the
traditional TAM scale, after comparing the maximum bending values with
Table S14 (Supporting Information), the severity of patients’ fingers flexi-
bility and corresponding grades were obtained, which were displayed over
HCI interface, simultaneously (Figure S7b, Supporting Information).
Hand Muscle Strength Assessment:During quantitative analysis, the
collected muscle signals were processed through ADC, amplification, and
filtration. In the filter process, the Exponentially Weighted Average algo-
rithm effectively diminished noise that was caused by factors such as
hand tremors or long-term bending of the sensor (Figure S11, Supporting
Information). Additionally, the experiment designed four sets of actions
(“Grasp the cylinder”, “Pinch objects with fingertip”, “Grasp the ball”,
and “Click objects with finger”) to investigate the pinch strength and grip
strength as well as the muscle strength of the finger pulp, fingertips, palm,
and purlicue (the muscle between forefinger and thumb). Next, the idea
of Cluster Analysis was put forward, which calculates the scalability and
efficiency of the operation while keeping the data distance, classifying the
grade of muscle strength at a data-based angle, and obtaining the opti-
mal solution in the local range by using the distance from the measured
muscle strength data to the set a classification center point. K-Means Clus-
ter Analysis was chosen for dividing the muscle strength grades.[59] The k
points were selected as the initial centers (𝜇1,𝜇2,…, 𝜇kRn) of the clus-
ters c(k), and then the distance of sample x(i) from the nearest center point
𝜇(j) was calculated, as shown in Formula (1). Based on the Lovett scale,
during the experiment the cluster kwas assigned a value of six
c(j)=argminj
x(i)𝜇(j)
2,j1,,k (1)
Formula (2) was used to recalculate the value of the cluster center 𝜇(j),
where nc(j)was the number of data in the jth cluster
𝜇(j)=xc(j)xj
nc(j)
(2)
After iterations until convergence, an accurate assortment of processed
muscle strength signals is obtained.
Finally, to assess the patients’ hand muscle strength status more accu-
rately, integrating the Lovett grading standard (Table S16, Supporting In-
formation), hand muscle strength grades (M0 to M5, where M0 indicates
the worst muscle strength) were provided together with visual results (Fig-
ure S7c, Supporting Information).
Hand Stability Assessment:To comprehensively assess hand stability,
it is necessary to fuse the acceleration values measured by the IMU sensor
on the three axes (i.e., x,y,andz). This is because interpreting accelera-
tion data separately on each axis can be intricate and arduous. Fusing the
triaxial acceleration into a single indicator can present and explain hand
motion stability more conveniently. This fused indicator is commonly re-
ferred to as “combined acceleration”, calculated as follows
a(k)=ax(k)2+ay(k)2+az(k)2,k=0,1,,n1(3)
In the equation, a(k) represents the combined acceleration signal of
the tremor, while ax(k), ay(k), and az(k) represent acceleration signals on
the x-, y-, and z-axis, respectively. nrepresents the length of the collected
acceleration data. Figure S12 (Supporting Information) shows the original
triaxial acceleration signals and their combined acceleration signal. It is
worth noting that the tremor signal processed subsequently refers to the
combined acceleration signal obtained by fusing the acceleration signals
on the three axes.
As the frequency of the static tremor mainly focuses within the range
of 3–8 Hz, the sampling frequency was set as 100 Hz and the acquisition
range as ±4g. the signal band was then filtered around to acquire accu-
rate tremor signals and avoid errors such as temperature drift and dark
Adv. Sci. 2023,10, 2206982 © 2023 The Authors. Advanced Science published by Wiley-VCH GmbH
2206982 (8 of 10)
www.advancedsciencenews.com www.advancedscience.com
current which was introduced during data collection. Then, the median fil-
tering (window length =5) and bandpass filtering (cut-off frequencies are
3 and 8 Hz) algorithms eliminated noises from systematic error, power-
line interference, and human intention. Finally, Kalman Filter was adopted
to remove random errors inside the signal band[60] (Figure S13, Support-
ing Information). Specifically, based on Kalman Filter, the first two terms
of the Taylor series expansion were taken from Formula (4), and Adaptive
Noise Model Parameters were then added to filter errors inside the tremor
signal band.
f(x)=f(x0)
0!+f(x0)
1!(xx0)+f′′(x0)
2!(xx0)2++f(n)(x0)
n!(xx0)n
+Rn(x)(4)
According to the specificity of the tremors, characterizations of pro-
cessed signals in both the time domain and frequency domain were ex-
tracted to improve the grading accuracy. From the time domain, the peak-
to-peak value, logarithmic peak-to-peak value, standard deviation, loga-
rithmic standard deviation, the root-mean-square (RMS), and the logarith-
mic RMS of the triaxial acceleration were extracted. While, the frequencies
of the x-, y-, and z-axis and the combined frequency were extracted from
the frequency domain. Then, the tremor signals based on the machine-
learning model were identified and classified and tenfold cross-validation
was used to determine the optimal hyperparameters of the model to avoid
underfitting or overfitting problems of the model. Compared with the orig-
inal UPDRS grades (0 to 4, where 4 indicates the worst stability), 0 was
divided as “No tremor”, 1 and 2 as “Mild tremor”, and 3 and 4 as “Severe
tremor”. The visualized grading results were also displayed on the HCI
interface (Figure S7d, Supporting Information).
Statistical Analysis:To verify the consistency between the multimodal
sensor glove system and clinical assessment, the Kappa consistency test
model was used. The Kappa coefficient value determines the degree of
consistency, which is classified as excellent (0.8 <Kappa 1.0), good (0.6
<Kappa 0.8), moderate (0.4 <Kappa 0.6), or poor otherwise. Z-score
and P-value were used to test for significant consistency in the Kappa co-
efficient. If the Z-score is greater than 1.96 or less than 1.96, and the
P-value is less than the significance level, there is significant consistency.
The sample size for finger flexibility and hand muscle strength is the same
(n=12), while the samples for hand stability assessment were divided
into a training set and test set in an 8:2 ratio. A consistency analysis of
predictions was conducted on the test set (n=24) with doctor’s assess-
ments.
Furthermore, the ICC consistency model was employed to determine
the test-retest reliability of the multimodal sensor glove. Reliability was
classified as excellent (ICC >0.9), good (0.75 <ICC 0.9), moderate
(0.5 <ICC 0.75), or poor otherwise. Through three repeated measure-
ment reliability experiments, the ICC values and 95% confidence intervals
were calculated for the results of 10 hand function assessments performed
by 12 subjects, and comprehensively validated the reliability of the multi-
modal sensor glove for assessing hand dysfunctions. The Kappa and ICC
consistency tests were conducted using IBM SPSS Statistics software (Ver-
sion 26.0).
Supporting Information
Supporting Information is available from the Wiley Online Library or from
the author.
Acknowledgements
Y.L. and J.Y. contributed equally to this work. X.J. thanks Science and
Technology Development Plan Project of Jilin Province (Grant No.
20230401088YY) for supporting this study. X.X. acknowledges National
Research Foundation, Singapore and A*STAR supporting this study un-
der its RIE2020 Industry Alignment Fund Industry Collaboration Projects
(IAF-ICP) grant call (Grant No. I2001E0059). X.X. also acknowledges sup-
port from the N.1 Institute for Health, the Institute for Health Innovation
and Technology, and the SIA-NUS Digital Aviation Corporate Laboratory.
J.C. acknowledges the Henry Samueli School of Engineering & Applied
Science and the Department of Bioengineering at the University of Cal-
ifornia, Los Angeles for the startup support. J.C. also acknowledges the
Hellman Fellows Research Grant, the UCLA Pandemic Resources Program
Research Award, and the Research Recovery Grant by the UCLA Academic
Senate, and the Brain & Behavior Research Foundation Young Investigator
Grant (Grant No. 30944), and the Catalyzing Pediatric Innovation Grant
(Grant No. 47744) from the West Coast Consortium for Technology & In-
novation in Pediatrics, Children’s Hospital Los Angeles.
Conflict of Interest
The authors declare no conflict of interest.
Data Availability Statement
The data that support the findings of this study are available from the cor-
responding author upon reasonable request.
Keywords
finger flexibility, hand muscle strength, hand stability, Parkinson’s disease,
smart glove, wearable bioelectronics
Received: November 27, 2022
Revised: April 16, 2023
Published online: May 7, 2023
[1] W. Poewe, K. Seppi, C. M. Tanner, G. M. Halliday, P. Brundin, J. Volk-
mann, A. E. Schrag, A. E. Lang, Nat. Rev. Dis. Primers 2017,3, 17013.
[2] T. Pringsheim, K. Fiest, N. Jette, Neurology 2014,83, 1661.
[3] W. Yang, J. L. Hamilton, C. Kopil, J. C. Beck, C. M. Tanner, R. L. Albin,
E. R. Dorsey, N. Dahodwala, I. Cintina, P. Hogan, T. Thompson, NPJ
Parkinson’s dis. 2020,6, 15.
[4] H. Zach, M. Dirkx, B. R. Bloem, R. C. Helmich, J. Parkinson’s Dis. 2015,
5, 471.
[5] J. Jankovic, J. Neurol., Neurosurg. Psychiatry 2008,79, 368.
[6] S. M. Fereshtehnejad, Y. Zeighami, A. Dagher, R. B. Postuma, Brain
2017,140, 1959.
[7] P. Svenningsson, E. Westman, C. Ballard, D. Aarsland, Lancet Neurol.
2012,11, 697.
[8] K. R. Chaudhuri, A. Schrag, D. Weintraub, A. Rizos, C. Rodriguez-
Blazquez, E. Mamikonyan, P. Martinez-Martin, Mov. Disord. 2020,35,
116.
[9] E. Tolosa, A. Garrido, S. W. Scholz, W. Poewe, Lancet Neurol. 2021,
20, 385.
[10] M. Lu, Q. Zhao, K. L. Poston, E. V. Sullivan, A. Pfefferbaum, M.
Shahid, M. Katz, L. M. Kouhsari, K. Schulman, A. Milstein, J. C.
Niebles, V. W. Henderson, L. Fei-Fei, K. M. Pohl, E. Adeli, Med. Image
Anal. 2021,73, 102179.
[11] K. Moriya, T. Yoshizu, Y. Maki, J. Orthop. Sci. 2021,26, 792.
[12] H. Gustafsson, J. Aasly, S. Stråhle, A. Nordström, P. Nordström, Neu-
rology 2015,84, 1862.
[13] V. C. De Cock, P. Dodet, S. Leu-Semenescu, C. Aerts, G. Castelnovo,
B. Abril, S. Drapier, H. Olivet, A. G. Corbillé, L. Leclair-Visonneau,
M. Sallansonnet-Froment, M. Lebouteux, M. Anheim, E. Ruppert, N.
Vitello, A. Eusebio, I. Lambert, A. Marques, M. L. Fantini, D. Devos,
C. Monaca, N. Benard-Serre, S. Lacombe, M. Vidailhet, I. Arnulf, M.
Doulazmi, E. Roze, Lancet Neurol. 2022,21, 428.
Adv. Sci. 2023,10, 2206982 © 2023 The Authors. Advanced Science published by Wiley-VCH GmbH
2206982 (9 of 10)
www.advancedsciencenews.com www.advancedscience.com
[14] R. B. Postuma, D. Berg, M. Stern, W. Poewe, C. W. Olanow, W. Oertel,
J. Obeso, K. Marek, I. Litvan, A. E. Lang, G. Halliday, C. G. Goetz, T.
Gasser, B. Dubois, P. Chan, B. R. Bloem, C. H. Adler, G. Deuschl,
Mov. Disord. 2015,30, 1591.
[15] J. L. Adams, K. Dinesh, C. W. Snyder, M. Xiong, C. G. Tarolli, S.
Sharma, E. R. Dorsey, G. Sharma, NPJ Parkinson’s dis. 2021,7, 106.
[16] K. A. Jellinger, G. Logroscino, G. Rizzo, M. Copetti, S. Arcuti, D. Mar-
tino, A. Fontana, Neurology 2016,87, 237.
[17] C. H. Adler, T. G. Beach, J. G. Hentz, H. A. Shill, J. N. Caviness, E.
Driver-Dunckley,M.N.Sabbagh,L.I.Sue,S.A.Jacobson,C.M.
Belden, B. N. Dugger, Neurology 2014,83, 406.
[18] K. Meng, X. Xiao, Z. Liu, S. Shen, T. Tat, Z. Wang, C. Lu, W. Ding, X.
He, J. Yang, J. Chen, Adv. Mater. 2022,34, 36.
[19] A. Moin, A. Zhou, A. Rahimi, A. Menon, S. Benatti, G. Alexandrov, S.
Tamakloe, J. Ting, N. Yamamoto, Y. Khan, F. Burghardt, L. Benini, A.
Arias, J. Rabaey, Nat. Electron. 2021,4, 54.
[20] S. Sundaram, P. Kellnhofer, Y. Li, J. Y. Zhu, A. Torralba, W. Matusik,
Nature 2019,569, 698.
[21] Z. Zhou, K. Chen, X. Li, S. Zhang, Y. Wu, Y. Zhou, K. Meng, C. Sun,
Q.He,W.Fan,E.Fan,Z.Lin,X.Tan,W.Deng,J.Yang,J.Chen,Nat.
Electron. 2020,3, 571.
[22] Y. Gao, L. Yu, J. C. Yeo, C. T. Lim, Adv. Mater. 2020,32, 1902133.
[23] Y. Zhou, X. Zhao, J. Xu, Y. Fang, G. Chen, Y. Song, S. Li, J. Chen, Nat.
Mater. 2021,20, 1670.
[24] J. Yang, J. Mun, S. Y. Kwon, S. Park, Z. Bao, S. Park, Adv. Mater. 2019,
31, 1904765.
[25] M. Zhu, T. He, C. Lee, Appl. Phys. Rev. 2020,7, 031305.
[26] X. Xiao, J. Yin, G. Chen, S. Shen, A. Nashalian, J. Chen, Matter 2022,
5, 1342.
[27] X. Zhao, Y. Zhou, J. Xu, G. Chen, Y. Fang, T. Tat, X. Xiao, Y. Song, S.
Li, J. Chen, Nat. Commun. 2021,12, 6755.
[28] X. Xiao, X. Xiao, Y. Zhou, X. Zhao, G. Chen, Z. Liu, Z. Wang, C. Lu,
M. Hu, A. Nashalian, S. Shen, K. Xie, W. Yang, Y. Gong, W. Ding,
P. Servati, C. Han, S. X. Dou, W. J. Li, J. Chen, Sci. Adv. 2021,7,
eabl3742.
[29] A. Libanori, G. Chen, X. Zhao, Y. Zhou, J. Chen, Nat. Electron. 2022,
5, 142.
[30] X. Xiao, J. Yin, S. Shen, Z. Che, X. Wan, S. Wang, J. Chen, Curr. Opin.
Solid State Mater. Sci 2022,26, 101042.
[31] G. Chen, X. Xiao, X. Zhao, T. Tat, M. Bick, J. Chen, Chem. Rev. 2022,
122, 3259.
[32] Y. Tang, H. Zhou, X. Sun, N. Diao, J. Wang, B. Zhang, C. Qin, E. Liang,
Y. Mao, Adv. Funct. Mater. 2019,30, 1907893.
[33] S. Shen, X. Xiao, J. Yin, X. Xiao, J. Chen, Small Methods 2022,6,
2200830.
[34] J. Hughes, A. Spielberg, M. Chounlakone, G. Chang, W. Matusik, D.
Rus, Adv. Intell. Syst. 2020,2, 2000002.
[35] D. Liu, P. Zhu, F. Zhang, P. Li, W. Huang, C. Li, N. Han, S. Mu, H.
Zhou, Y. Mao, Nano Res. 2023,16, 1196.
[36] M. Wang, Z. Yan, T. Wang, P. Cai, S. Gao, Y. Zeng, C. Wan, H. Wang,
L. Pan, J. Yu, S. Pan, K. He, J. Lu, X. Chen, Nat. Electron. 2020,3, 563.
[37] K. K. Kim, I. Ha, M. Kim, J. Choi, P. Won, S. Jo, S. H. Ko, Nat. Commun.
2020,11, 222.
[38] Z. Sun, M. Zhu, X. Shan, C. Lee, Nat. Commun. 2022,13, 5224.
[39] F. Wen, Z. Zhang, T. He, C. Lee, Nat. Commun. 2021,12, 5378.
[40] H. Zhou, W. Huang, Z. Xiao, S. Zhang, W. Li, J. Hu, T. Feng, J. Wu, P.
Zhu, Y. Mao, Adv. Funct. Mater. 2022,32, 2208271.
[41] W. Navaraj, R. Dahiya, Adv. Intell. Syst. 2019,1, 1900051.
[42] Y. Luo, Z. Wang, J. Wang, X. Xiao, Q. Li, W. Ding, H. Y. Fu, Nano Energy
2021,89, 106330.
[43] M.Amit,L.Chukoskie,A.J.Skalsky,H.Garudadri,T.N.Ng,Adv.
Funct. Mater. 2020,30, 1905241.
[44] R. Ranzani, F. Viggiano, B. Engelbrecht, J. P. O. Held, O. Lambercy,
R. Gassert, presented at 2019 IEEE 16th In. Con. on Rehabilita-
tion Robotics (ICORR), Method for Muscle Tone Monitoring Dur-
ing Robot-Assisted Therapy of Hand Function: A Proof of Concept,
Toronto, ON, Canada, June, 2019.
[45] S. Balasubramanian, J. Klein, E. Burdet, Curr. Opin. Neurobiol. 2010,
23, 661.
[46] M. Germanotta, V. Gower, D. Papadopoulou, A. Cruciani, C. Pecchi-
oli, R. Mosca, G. Speranza, C. Falsini, F. Cecchi, F. Vannetti, A. Mon-
tesano, S. Galeri, F. Gramatica, I. Aprile, J. Neuroeng. Rehabil. 2020,
17,1.
[47] V. Majhi, S. Paul, G. Saha, J. K. Verma, presented at 2020 In. Con.
on Computational Performance Evaluation (ComPE), Sensor based
Detection of Parkinson’s Disease Motor Symptoms, Shillong, India,
July, 2020.
[48] L. Lonini, A. Dai, N. Shawen, T. Simuni, C. Poon, L. Shimanovich,
M. Daeschler, R. Ghaffari, J. A. Rogers, A. Jayaraman, NPJ Digit Med.
2018,1, 64.
[49] Y. Zheng, Y. Peng, G. Wang, X. Liu, X. Dong, J. Wang, Measurement
2016,93,1.
[50] S. Jiang, B. Lv, W. Guo, C. Zhang, H. Wang, X. Sheng, P. B. Shull, IEEE
Trans. Ind. Inf. 2017,14, 3376.
[51] H. B. Kim, W. W. Lee, A. Kim, H. J. Lee, H. Y. Park, H. S. Jeon, S. K.
Kim, B. Jeon, K. S. Park, Comput. Biol. Med. 2018,95, 140.
[52] Q. Fang, S. S. Mahmoud, X. Gu, J. Fu, IEEE J. Biomed. Health Inf. 2018,
23, 758.
[53] Y. Zou, D. Wang, S. Hong, R. Ruby, D. Zhang, K. Wu, IEEE Internet
Things J. 2020,7, 7377.
[54] X. Chen, L. Gong, L. Wei, S.-C. Yeh, L. Xu, L. Zheng, Z. Zou, IEEE
Trans. Ind. Inf. 2020,17, 943.
[55] V. K. Legaria-Santiago, L. P. Sánchez-Fernández, L. A. Sánchez-
Pérez, A. Garza-Rodríguez, Comput. Biol. Med. 2022,140,
105059.
[56] S. Tadse, M. Jain, P. Chandankhede, presented at 2021 5th In. Con.
on Intelligent Computing and Control Systems (ICICCS), Parkinson’s
Detection Using Machine Learning, Madurai, India, May, 2021.
[57] S. R. Schreglmann, D. Wang, R. L. Peach, J. Li, X. Zhang, A. Latorre,
E. Rhodes, E. Panella, A. M. Cassara, E. S. Boyden, M. Barahona, S.
Santaniello, J. Rothwell, K. P. Bhatia, N. Grossman, Nat. Commun.
2021,12, 363.
[58] P. Ortelli, D. Ferrazzoli, V. Versace, V. Cian, M. Zarucchi, A. Gus-
meroli, M. Canesi, G. Frazzitta, D. Volpe, L. Ricciardi, R. Nardone,
I. Ruffini, L. Saltuari, L. Sebastianelli, D. Baranzini, R. Maestri,
NPJ Parkinson’s dis. 2022,8, 42.
[59] M. E. Celebi, H. A. Kingravi, P. A. Vela, Expert Syst. Appl. 2013,40, 200.
[60] E. de Bézenac, S. S. Rangapuram, K. Benidis, M. Bohlke-Schneider, R.
Kurle, L. Stella, H. Hasson, P. Gallinari, T. Januschowski, Adv. Neural
Inf. Process Syst. 2020,33, 2995.
Adv. Sci. 2023,10, 2206982 © 2023 The Authors. Advanced Science published by Wiley-VCH GmbH
2206982 (10 of 10)
... Most of the wearable solutions in the literature rely either on IMUs [20, 63-65, 70, 71] to detect hand movements or surface EMG sensors[18, 66,72] for muscular activity measurements. These sensors may be integrated into smart gloves or simple exoskeletons, alongside additional components such as pressure and force measurements systems and microcontrollers [67,73,74]. ...
... Machine learning. Machine learning represents a more dynamic and data-driven approach in biosignal post-processing [166][167][168] . Instead of relying on predefined rules, machine learning models are trained on labelled or unlabelled data sets to learn patterns and relationships within data 169 . ...
Article
Full-text available
Soft bioelectronic devices can be interfaced with anatomically curved organs, such as the heart, brain and skin, to provide continuous analysis of physiological information. However, body movements and physiological activities may induce motion artefacts, which can adversely affect signal accuracy and stability. Importantly, motion artefact management is key to promoting the clinical translation of soft bioelectronics to ensure that soft bioelectronic devices can selectively detect target biological signals with high accuracy. In this Review, we discuss how body activities can affect the soft bioelectronic–tissue interface and result in motion artefact signals, including interface impedance instability motion artefacts, biopotential motion artefacts and mechanical motion artefacts. We then investigate different motion artefact management strategies, including materials engineering, device and circuit design, and algorithmic intervention, to reduce the contribution of motion artefacts to signal acquisition, processing and interpretation.
... The glove assessed flexibility, muscle strength, and stability, showing high consistency with clinical observations. The system's reliability was confirmed through repeated experiments, with intraclass correlation coefficients exceeding 0.9 [51]. ...
Article
Full-text available
This review investigates the convergence of artificial intelligence (AI) and personalized health monitoring through wearable devices, classifying them into three distinct categories: bio-electrical, bio-impedance and electro-chemical, and electro-mechanical. Wearable devices have emerged as promising tools for personalized health monitoring, utilizing machine learning to distill meaningful insights from the expansive datasets they capture. Within the bio-electrical category, these devices employ biosignal data, such as electrocardiograms (ECGs), electromyograms (EMGs), electroencephalograms (EEGs), etc., to monitor and assess health. The bio-impedance and electro-chemical category focuses on devices measuring physiological signals, including glucose levels and electrolytes, offering a holistic understanding of the wearer’s physiological state. Lastly, the electro-mechanical category encompasses devices designed to capture motion and physical activity data, providing valuable insights into an individual’s physical activity and behavior. This review critically evaluates the integration of machine learning algorithms within these wearable devices, illuminating their potential to revolutionize healthcare. Emphasizing early detection, timely intervention, and the provision of personalized lifestyle recommendations, the paper outlines how the amalgamation of advanced machine learning techniques with wearable devices can pave the way for more effective and individualized healthcare solutions. The exploration of this intersection promises a paradigm shift, heralding a new era in healthcare innovation and personalized well-being.
... Subsequently, modern smart systems have inculcated gesture and voice controls into HCI systems, leading to greater convenience and efficiency of usage. [27][28][29] While eye movement control for HCI has been the subject of several studies, particularly for applications associated with assistive technology, it remains a relatively untapped resource in commercial systems. [30][31][32] Among the major advantages of eye-controlled HCI lies its requirement for significantly lower energy and effort from the user as compared to other human-body-based HCI systems. ...
Article
With the emergence of new communication technologies, smart devices have been playing an increasingly important role in our daily lives. Such devices, typically controlled through voice or hand gestures, serve as disability assistance or in the enhancement of convenience in daily tasks. Eye movement represents a relatively untapped resource for assistive human-computer interaction (HCI) systems with considerable utility, particularly for passive sensing applications and as assistance for people with disabilities. Triboelectric nanogenerators (TENGs) are a compelling alternative to conventional sensors, owing to their high sensitivity, short response time, wearability, and low cost. This review delves into various TENG-based wearable sensors that have demonstrated applications in eye movement monitoring and the broader HCI domain. We also provide a perspective on the challenges that have inhibited the commercialization of these sensors and possible applications of such technology for eye healthcare.
Article
Wearable strain sensors (WSSs) have found widespread applications, where the key is to optimize their sensing and wearing performances. However, the intricate material designs for developing WSSs often rely on costly reagents and/or complex processes, which bring barriers to their large-scale production and use. Herein, a facile and affordable (material cost of < $0.002/cm2) method is presented for fabricating conductive bandage (CB)-based WSSs by electrospraying a carbon nanotube (CNT) layer on commercial self-adhesive bandages with excellent biosafety, stretchability, mechanical compliance, breathability and cost effectiveness. The wrinkled and fibrous structures of self-adhesive bandages were rationally leverage to control the geometry of CNT layer, thereby ensuring tunable mechanoelectrical sensitivities (gauge factors of 2 ~ 850) of CBs. Moreover, a strain-sensing mechanism directly mediated by the highly wrinkled microstructure is unveiled, which can work in synergy with a training-loosened-fibrous microstructure. The excellent performance of CBs for monitoring full-range strain signals in human bodies was further demonstrated. CBs would possess great potential for being developed into WSSs because of their outstanding cost-performance ratio.
Article
Full-text available
Triboelectric nanogenerators (TENGs) have recently gained attention as a compelling platform technology for building wearable bioelectronics. Aside from being self‐powered, TENGs are lightweight, low in cost, rich in material choice, comfortable to wear, and increasingly versatile with advances in sensitivity and efficiency. Due to these features, TENGs have become appealing in biomedical sensing applications, especially for human respiration monitoring. A wealth of information can be collected by breath‐induced electrical signals, which are crucial in the analysis of a patient's respiratory condition and the early detection of harmful respiratory‐linked diseases. TENGs have thus been used to continuously collect important respiratory data, from the breathing patterns, flow rate, and intensity of an individual's respiratory cycle to the chemicals that may be present in their breath. This review paper provides an overview of recent developments in TENG‐based wearable respiratory monitoring as well as future opportunities and challenges for respiratory healthcare.
Article
Full-text available
The gut‐brain axis has recently emerged as a crucial link in the development and progression of Parkinson's disease (PD). Dysregulation of the gut microbiota has been implicated in the pathogenesis of this disease, sparking growing interest in the quest for non‐invasive biomarkers derived from the gut for early PD diagnosis. Herein, an artificial intelligence‐guided gut‐microenvironment‐triggered imaging sensor (Eu‐MOF@Au‐Aptmer) to achieve non‐invasive, accurate screening for various stages of PD is presented. The sensor works by analyzing α‐Syn in the gut using deep learning algorithms. By monitoring changes in α‐Syn, the sensor can predict the onset of PD with high accuracy. This work has the potential to revolutionize the diagnosis and treatment of PD by allowing for early intervention and personalized treatment plans. Moreover, it exemplifies the promising prospects of integrating artificial intelligence (AI) and advanced sensors in the monitoring and prediction of a broad spectrum of diseases and health conditions.
Article
Full-text available
Human skin perceives external environmental stimulus by the synergies between the subcutaneous tactile corpuscles. Soft electronics with multiple sensing capabilities by mimicking the function of human skin are of significance in health monitoring and artificial sensation. The last decade has witnessed unprecedented development and convergence between multimodal tactile sensing devices and soft bioelectronics. Despite these advances, traditional flexible electronics achieve multimodal tactile sensing for pressure, strain, temperature, and humidity by integrating monomodal sensing devices together. This strategy results in high energy consumption, limited integration, and complex manufacturing process. Various multimodal sensors and crosstalk-free sensing mechanisms have been proposed to bridge the gap between natural sensory system and artificial perceptual system. In this review, we provide a comprehensive summary of tactile sensing mechanism, integration design principles, signal-decoupling strategies, and current applications for multimodal tactile perception. Finally, we highlight the current challenges and present the future perspectives to promote the development of multimodal tactile perception.
Article
Full-text available
Human‐machine interfaces (HMIs) play important role in the communication between humans and robots. Touchless HMIs with high hand dexterity and hygiene hold great promise in medical applications, especially during the pandemic of coronavirus disease 2019 (COVID‐19) to reduce the spread of virus. However, current touchless HMIs are mainly restricted by limited types of gesture recognition, the requirement of wearing accessories, complex sensing platforms, light conditions, and low recognition accuracy, obstructing their practical applications. Here, an intelligent noncontact gesture‐recognition system is presented through the integration of a triboelectric touchless sensor (TTS) and deep learning technology. Combined with a deep‐learning‐based multilayer perceptron neural network, the TTS can recognize 16 different types of gestures with a high average accuracy of 96.5%. The intelligent noncontact gesture‐recognition system is further applied to control a robot for collecting throat swabs in a noncontact mode. Compared with present touchless HMIs, the proposed system can recognize diverse complex gestures by utilizing charges naturally carried on human fingers without the need of wearing accessories, complicated device structures, adequate light conditions, and achieves high recognition accuracy. This system could provide exciting opportunities to develop a new generation of touchless medical equipment, as well as touchless public facilities, smart robots, virtual reality, metaverse, etc.
Article
Full-text available
The hands are used in all facets of daily life, from simple tasks such as grasping and holding to complex tasks such as communication and using technology. Finding a way to not only monitor hand movements and gestures but also to integrate that data with technology is thus a worthwhile task. Gesture recognition is particularly important for those who rely on sign language to communicate, but the limitations of current vision‐based and sensor‐based methods, including lack of portability, bulkiness, low sensitivity, highly expensive, and need for external power sources, among many others, make them impractical for daily use. To resolve these issues, smart gloves can be created using a triboelectric nanogenerator (TENG), a self‐powered technology that functions based on the triboelectric effect and electrostatic induction and is also cheap to manufacture, small in size, lightweight, and highly flexible in terms of materials and design. In this review, an overview of the existing self‐powered smart gloves will be provided based on TENGs, both for gesture recognition and human‐machine interface, concluding with a discussion on the future outlook of these devices.
Article
Full-text available
Advancements of virtual reality technology pave the way for developing wearable devices to enable somatosensory sensation, which can bring more comprehensive perception and feedback in the metaverse-based virtual society. Here, we propose augmented tactile-perception and haptic-feedback rings with multimodal sensing and feedback capabilities. This highly integrated ring consists of triboelectric and pyroelectric sensors for tactile and temperature perception, and vibrators and nichrome heaters for vibro- and thermo-haptic feedback. All these components integrated on the ring can be directly driven by a custom wireless platform of low power consumption for wearable/portable scenarios. With voltage integration processing, high-resolution continuous finger motion tracking is achieved via the triboelectric tactile sensor, which also contributes to superior performance in gesture/object recognition with artificial intelligence analysis. By fusing the multimodal sensing and feedback functions, an interactive metaverse platform with cross-space perception capability is successfully achieved, giving people a face-to-face like immersive virtual social experience.
Article
Full-text available
The assessment of cognitive deficits is pivotal for diagnosis and management in patients with parkinsonisms. Low levels of correspondence are observed between evaluations assessed with screening cognitive tests in comparison with those assessed with in-depth neuropsychological batteries. A new tool, we named CoMDA (Cognition in Movement Disorders Assessment), was composed by merging Mini-Mental State Examination (MMSE), Montreal Cognitive Assessment (MoCA), and Frontal Assessment Battery (FAB). In total, 500 patients (400 with Parkinson’s disease, 41 with vascular parkinsonism, 31 with progressive supranuclear palsy, and 28 with multiple system atrophy) underwent CoMDA (level 1–L1) and in-depth neuropsychological battery (level 2–L2). Machine learning was developed to classify the CoMDA score and obtain an accurate prediction of the cognitive profile along three different classes: normal cognition (NC), mild cognitive impairment (MCI), and impaired cognition (IC). The classification accuracy of CoMDA, assessed by ROC analysis, was compared with MMSE, MoCA, and FAB. The area under the curve (AUC) of CoMDA was significantly higher than that of MMSE, MoCA and FAB ( p < 0.0001, p = 0.028 and p = 0.0007, respectively). Among 15 different algorithmic methods, the Quadratic Discriminant Analysis algorithm (CoMDA-ML) showed higher overall-metrics performance levels in predictive performance. Considering L2 as a 3-level continuous feature, CoMDA-ML produces accurate and generalizable classifications: micro-average ROC curve, AUC = 0.81; and AUC = 0.85 for NC, 0.67 for MCI, and 0.83 for IC. CoMDA and COMDA-ML are reliable and time-sparing tools, accurate in classifying cognitive profile in parkinsonisms. This study has been registered on ClinicalTrials.gov (NCT04858893).
Article
Powering wearable bioelectronics with decent skin conformability and wearing comfort is highly desired. Fiber batteries could provide an attractive alternative to traditional rigid ones and present a compelling solution to this problem. In this review, we will discuss the various classes of fiber batteries, including lithium batteries, zinc batteries, and other types of fiber batteries. We will then report the latest research progress on each battery category through its working mechanism, materials usage, structure design, and wearable applications. Finally, we provide insights into current challenges and future applications of fiber batteries, aiming to promote the development of low-cost and high-performance fiber battery technologies for wearable bioelectronics.
Article
As a stretchable seamless device, electronic skin (E-skin) has drawn enormous interest due to its skin-like sensing capability. Besides the basic perception of force and temperature, multiple perception that is beyond existing functions of human skin is becoming an important direction for E-skin developments. However, the present E-skins for multiple perceptions mainly rely on different sensing materials and heterogeneous integration, resulting in a complex device structure. Additionally, their stretchability is usually achieved by the complicated microstructure design of rigid materials. Here, we report an intrinsically stretchable polymer semiconductor based E-skin with a simple structure for multiple perceptions of force, temperature, and visible light. The E-skin is on the basis of poly(3-hexylthiophene) (P3HT) nanofibers percolated polydimethylsiloxane (PDMS) composite polymer semiconductor, which is fabricated by a facile solution method. The E-skin shows reliable sensing capabilities when it is used to perceive strain, pressure, temperature, and visible light. Based on the E-skin, an intelligent robotic hand sensing and controlling system is further demonstrated. Compared with conventional E-skins for multiple perceptions, this E-skin only has a simple monolayer sensing membrane without the need of combining different sensing materials, heterogeneous integration, and complicated microstructure design. Such a strategy of utilizing intrinsically stretchable polymer semiconductor to create simple structured E-skin for multiple perceptions will promote the development of E-skins in a broad application scenario, such as artificial robotic skins, virtual reality, intelligent gloves, and biointegrated electronics.
Article
The smart textile research community is advancing rapidly with the aim of changing the way we wear clothes in the near future. An eardrum-inspired acoustic textile was recently developed that can function as a sensitive audible microphone. The acoustic textile is woven from yarns with a high Young’s modulus and thermal drawing composite piezoelectric fibers that can concentrate mechanical stress in the piezoelectric composite layer during the heat drawing process, endowing the fiber with high sensitivity. The acoustic textile can convert even very slight 10-7 atmospheric pressure waves at audible frequencies into mechanical vibrations with nano�meter amplitudes, which further induce the electrical signals through the fibers to detect the direction of sound pulses, broadcast human speech, and capture heartbeats, making it suitable as a wear�able platform for long-term biomonitoring
Article
Background Insomnia is a frequent complaint of patients with Parkinson's disease, and it negatively affects quality of life. Drugs that improve both sleep and parkinsonism would be of major benefit to patients with Parkinson's disease-related insomnia. We aimed to test the safety and efficacy of subcutaneous night-time only apomorphine infusion in patients with Parkinson's disease and insomnia. Methods We did a randomised, multicentre, double-blind, placebo-controlled, crossover trial in 11 expert centres in Parkinson's disease and sleep centres in France. Participants aged 35–90 years with fluctuating Parkinson's disease and moderate to severe insomnia (Insomnia Severity Index score ≥15) were randomly assigned to either first receive night-time subcutaneous apomorphine (up to 5 mg/h) or matching placebo. Randomisation was done using a computer-generated plan in blocks of four, stratified by centre. This first intervention was followed by a 14-night washout period, then crossover to the other intervention. The treatment periods consisted of a 10-night titration phase followed by a 7-night fixed-dose phase. The dose was adjusted during the titration phase on the basis of a daily telephone call assessing sleep quality and treatment tolerability. The primary efficacy endpoint was the difference in Parkinson's disease sleep scale (PDSS) scores from the beginning to the end of each treatment period. Analysis was done on an intention-to-treat basis. This trial is registered with ClinicalTrials.gov, NCT02940912. Findings Between Jan 31, 2017, and Jan 29, 2021, 46 participants were enrolled. 25 (54%) patients were randomly assigned to receive apomorphine first and 21 (46%) patients to receive placebo first. Mean change in PDSS score was significantly greater with night-time apomorphine infusion (15·18 [SD 24·34]) compared with placebo (5·23 [21·52]; treatment effect 9·95 [95% CI 0·88–19·03]; p=0·041). Adverse events were reported in 25 (54%) participants during the apomorphine period and in 17 (37%) participants during the placebo period (p=0·16). Apomorphine was associated with more frequent dizziness than was placebo (seven [15%] vs 0; p=0·041). Interpretation Subcutaneous night-time only apomorphine infusion improved sleep disturbances according to difference on PDSS score, with an overall safety profile consistent with previous studies in Parkinson's disease. This treatment might be useful to manage sleep disturbances in patients with advanced Parkinson's disease and moderate to severe insomnia. Funding Orkyn and Aguettant Pharma. Translation For the French translation of the abstract see Supplementary Materials section.
Article
Incorporating sensing and therapeutic capabilities into everyday textiles could be a powerful approach in the development of personalized healthcare. The creation of such smart textiles has been driven by the fabrication of various miniaturized platform technologies, and has led to the construction of compact, autonomous and interconnected functional textiles. Here we review the development of smart textiles for application in personalized healthcare. We examine the different platform technologies, the various fabrication strategies and the range of clinical scenarios in which they are used. We also explore the current commercial and regulatory landscapes, and consider issues of data management. Finally, we highlight the key steps required to transition these technological platforms to commercial applications. This Review examines the development of smart textiles for application in personalized healthcare, examining the different platform technologies, fabrication strategies and clinical scenarios, as well as the current commercial and regulatory landscape.