Fig 2 - uploaded by Jian Huang
Content may be subject to copyright.
Block diagram of the control architecture of the cane robot. The robot collects information of the user's walking based on the perception system, and estimates the user's state and determines essential support. Different control strategies are adopted based on different working modes.
Source publication
Assisting and supervising daily long-durational walking is very crucial for patients with lower extremity dysfunction, especially in the stage of recovery towards a state of walking independently. However, due to the shortage of caregivers and high-cost of nursing, long-term manual assistance and supervision is costly. Thus, in this paper, we propo...
Context in source publication
Context 1
... the walking-aid mode shown in Fig. 2, the user intuitively grasps the handle during walking. Based on the Intention Based Admittance Control Algorithm (IBAC) as proposed in our former research [2], the robot can estimate the human walking intention from the interaction force obtained by a force sensor and provide the compliant physical support through the physical human ...
Citations
... 机器人系统的一项核心技术，其功能是控制机器人 稳定可靠地自动跟随人体目标并执行协作任务 [5] ， 实现人机共融环境下安全友好的交互协作。 近年来，人机跟随策略方面的研究主要关注的 是后跟随策略 [6] 。 然而，有关人类结伴行走行为的 研究 [7] 表明，人类更希望机器人尽可能在其视野 范围内运动。 相较于机器人在目标人后方跟随给人 类带来心理压力，机器人在目标人侧方陪伴运动是 一种更为舒适、自然的跟随方式 [8] 。 目前只有少数 研究关注并实现了与人体目标并排的陪伴跟随，但 效果并不理想。 Manawadu 等 [9] 提出了一种人机并 排运动模型实现了机器人的陪伴跟随，但采用的比 例控制算法鲁棒性较差。 Yao 等 [10] 设计了一种基 于 PID 法的跟随策略，机器人通过转角时从后跟 随模式切换到并排跟随模式以扩大机器人的视野， 避免丢失被跟随目标，然而该策略只适用于特定场 景。 Xue 等 [11] 提出了一种基于虚拟目标的跟随策 略，该策略将被跟随目标人的位置转换到机器人正 前方从而得到一个虚拟目标，但该方法无法适应目 标人的运动意图临时突变引起的速度变化。 此外， 上述方法下人机相对位置控制主要是基于单一的位 置跟踪策略，未考虑人机交互过程的柔顺性和用户 在伴随任务中的舒适感受。 为实现人机共融系统，近年来有学者尝试将跟 随策略集成到机器人地图导航框架中。 Ferrer 等 [12] 提出了基于社会力模型的地图导航框架，Repiso 等 [13] 对其进行改进，使机器人能够遵守人类社交 规则。 Kästner 等 [14] 采用强化学习算法，在栅格地 图中通过环境和任务的语义信息训练智能体策略。 然而，基于地图导航框架的方法数据量大，平台算 力要求较高，系统响应速度不稳定，参数的整定也 较为复杂，难以在不同机器人之间移植。 为了提升控制的实时性，近年来无地图自主导 航方法也被应用于机器人的跟随任务中。 Yan 等 [2] 提出基于模型预测控制的跟随策略，提升跟随控 制的精度，但未考虑机器人在非结构化场景下的避 障。 Yan 等 [15] [20] ： I I I(q q q)q q q +C C C(q q q,q q q)q q q = B B B(q q q)τ τ τ ...
目前社交机器人与目标人体并排行走（人体伴随与避障）时的运动控制策略的研究相对缺乏。为提升陪伴任务中用户的舒适度、机器人的运动柔顺度及其安全避障能力，本文提出了一种基于导纳控制的人体伴随与避障控制策略。首先，基于人机交互空间理论设计了交互力模型以构建人机动态交互关系，避免机器人侵犯目标人的亲密区域以提升目标人的舒适度；其次，将导纳控制模型与交互力模型结合，通过设置合理的导纳参数提升机器人运动控制的柔顺度；最后，引入行为动力学模型模拟人类的避障行为，以保障人体伴随任务的安全性。此外，提出了一组评价指标以验证伴随控制器的性能。根据仿真实验结果，在柔顺度方面，相较于 PID 法和虚拟弹簧模型（VSM）法，本方法下速度变化量分别降低 69.6% 和 67.1%；在舒适度方面，机器人未给目标人带来不适；在安全性方面，本方法的避障失败率仅为 10%，优于人工势场（APF）法和 VSM 法的 40% 和 50%。实物实验中，机器人的柔顺度和舒适度指标均较好，避障失败率仅为 5%，有效实现了安全友好的人体伴随与避障控制。
Currant research on the motion control strategies of the social robots walking side-by-side with a target human body (i.e. human accompanying and obstacle avoidance) is deficient. To improve the accompanying comfort level, and the motion compliance and safe obstacle-avoidance performances of the robot, a human-accompanying and obstacle-avoidance control strategy is proposed based on admittance control. Firstly, an interaction force model is designed based on the theory of human-robot interaction space, which describes the dynamic human-robot interaction relationship to prevent the robot from infringing into the companion’s intimate area, thereby enhancing the comfort level of the companion. Secondly, the admittance control model is combined with the interaction force model to improve the motion compliance of the robot through optimal admittance parameters. Finally, a behavioral dynamics model is introduced to simulate the human obstacleavoidance behavior, thereby ensuring the safety of the human-accompanying task. Additionally, a set of evaluation indexes are proposed for human-accompanying performance. The simulation results show that the robot velocity change under the proposed method is reduced by 69.6% and 67.1% respectively compared with PID (proportional-integral-derivative) and VSM (virtual spring model) methods, demonstrating its advantages in terms of compliance; in terms of comfort, the robot doesn’t cause discomfort to humans; in terms of safety, the obstacle-avoidance failure rate of the proposed method is only 10%, better than the 40% and 50% of APF (artificial potential field) and VSM methods. In the physical experiment, the robot exhibits better compliance and comfort performances, and the obstacle-avoidance failure rate is only 5%. Therefore, the proposed method effectively achieves safe and friendly human accompanying and obstacle-avoidance control.
... Additionally, the forward orientation of the human body, denoted as hm , is determined based on either the vertical direction of the line connecting the legs or the speed direction of the midpoint of the line connecting the legs. This orientation varies in accordance with different walking modes [17]. ...
For people with lower limb muscle weakness, effective and timely rehabilitation intervention is essential for assisting in daily walking and facilitating recovery. Numerous studies have been conducted on rehabilitation robots; however, some critical issues in the field of human-following remain unaddressed. These include potential challenges related to the loss of sensory signals for intention recognition and the complexities associated with maintaining the relative pose of robots during the following process. A human-following surveillance robot is introduced as the basis of the research. To address potential interruptions in motion signals, such as data transmission blockages or body occlusion, we propose a human walking intention estimation algorithm based on set-membership filtering with incomplete observation. To ensure uninterrupted user walking and maintain an effective aid and detection range, we propose a human-following control algorithm based on prescribed performance. The experiment verifies the effectiveness of the proposed methods. The proposed intention estimation algorithm achieves continuous and accurate intention recognition under incomplete observation. The control algorithm presented in this paper achieves constrained robot following with respect to the relative pose.
... To ensure adaptability and stability, such HRIs usually perform their behaviors fully collaboratively. In this regard, collaborative behaviors of HRIs have been investigated in recent research, including co-behaviors [220] [221], sit-tostand [222], and human-following [223], etc. ...
Gait coordination (GC), meaning that one leg moves in the same pattern but with a specific phase lag to the other, is a spontaneous behavior in the walking of a healthy person. It is also crucial for unilateral amputees with the robotic leg prosthesis to perform ambulation cooperatively in the real world. However, achieving the GC for amputees poses significant challenges to the prostheses’ dynamic modeling and control design. Still, there has not been a clear survey on the initiation and evolution of the detailed solutions, hindering the precise decision of future explorations. To this end, this paper comprehensively reviews GC-oriented dynamic modeling and adaptive control methods for robotic leg prostheses. Considering the two representative environments concerned with adaptive control, we first classify the dynamic models into the deterministic model for structured terrain and the constrained stochastic model for stochastically uneven terrain. Inspired by the concept of synchronization, we then emphasize three typical problems for the GC realization, i.e., complete coordination on structured terrain, stochastic coordination on stochastically uneven terrain, and finite-time delayed stochastic coordination. Finally, we conclude with a discussion on the remaining challenges and opportunities in controlling robotic leg prostheses.
... Shunki et al. used a cane-type assistive mobile robot for clinical gait training in clinical settings and verified the efficacy of their proposed gait rehabilitation strategy with the robot [11], [12]. Yan et al. implemented a user intention estimation model and successfully achieved human following for the safety and supervision of users' independent walking during rehabilitation training [13]. ...
... At this range, the localization of humans can be affected by the limited perceptible information obtained by sensors, as well as occlusion caused by the swinging of lower limbs during normal walking. Huang et al. have proposed a laser-based cane-type robot that can follow a human user during rehabilitation training, which has demonstrated satisfactory human following performance in close range [13], [23]. Although lidars can provide accurate and high-resolution depth information for object detection [13], [24], they can be expensive and have limitations in terms of size, weight, and power consumption. ...
... Huang et al. have proposed a laser-based cane-type robot that can follow a human user during rehabilitation training, which has demonstrated satisfactory human following performance in close range [13], [23]. Although lidars can provide accurate and high-resolution depth information for object detection [13], [24], they can be expensive and have limitations in terms of size, weight, and power consumption. Moreover, they may require complex processing and calibration, particularly when used in outdoor scenarios. ...
Cane-type robots have been utilized to assist and supervise the mobility-impaired population. One essential technique for cane-type robots is human following control, which allows the robot to follow the user. However, the limited perceptible information of humans by sensors at close range, combined with the occlusion caused by lower limb swing during normal walking, affect the localization of users. These limitations make it difficult to achieve human following at close range.To address these challenges, this study developed a new cane-type wheeled robot and proposed a novel human-following control with multi-camera fusion. This control system mainly consists of two parts: 1) a human following controller that locates a user by multi-camera fusion and generates control signals to follow the user. 2) a cane robot controller designed to steer the cane robot to a target position. The proposed strategy's effectiveness has been validated in outdoor experiments with six healthy subjects. The experimental scenarios included different terrains (i.e., straight, turning, and inclined paths), road conditions (i.e., flat and rough roads), and walking speeds. The obtained results showed that the average tracking error for position and orientation was less than 5 cm and 15{\deg} respectively across all scenarios. Moreover, the cane robot can effectively adapt to a wide range of individual gait patterns and achieve stable human following at daily walking speeds (0.75 m/s - 1.45 m/s).
... One way to predict human motion intention is to observe human movements and make use of human kinematics information such as body segment positions and velocities. Usually, human kinematics can be obtained through vision sensors (Chalvatzaki et al. (2019), Zhu et al. (2022)), Laser ranger finder (LRFs) Yan et al. (2021), Ultra-Wideband (UWB) Xue et al. (2022) and wearable sensors Yuan et al. (2019). However, these sensors are not cost-effective or convenient to put on for daily use in the home and community scenarios. ...
Numerous diseases and aging can cause degeneration of people's balance ability resulting in limited mobility and even high risks of fall. Robotic technologies can provide more intensive rehabilitation exercises or be used as assistive devices to compensate for balance ability. However, With the new healthcare paradigm shifting from hospital care to home care, there is a gap in robotic systems that can provide care at home. This paper introduces Mobile Robotic Balance Assistant (MRBA), a compact and cost-effective balance assistive robot that can provide both rehabilitation training and activities of daily living (ADLs) assistance at home. A three degrees of freedom (3-DoF) robotic arm was designed to mimic the therapist arm function to provide balance assistance to the user. To minimize the interference to users' natural pelvis movements and gait patterns, the robot must have a Human-Robot Interface(HRI) that can detect user intention accurately and follow the user's movement smoothly and timely. Thus, a graceful user following control rule was proposed. The overall control architecture consists of two parts: an observer for human inputs estimation and an LQR-based controller with disturbance rejection. The proposed controller is validated in high-fidelity simulation with actual human trajectories, and the results successfully show the effectiveness of the method in different walking modes.
... The robot kept the human pressure center within the support polygon of the human-robot system by adjusting the position and posture of the support rod. In order to ensure the human following performance of the walking-aid cane robot to the human, Huang et al. proposed in 2021 to use the human walking intention estimation method to effectively monitor the user's walking [11]. The proposed system can provide different walking modes according to the needs of different users. ...
... (11) is obtained after simplification. * * sin( + ) = * * sin (10) = * * sin * sin( + ) (11) Secondly, the second objective "the friction of the robot chassis will not make the robot chassis slip" is analyzed. The force analysis diagram of the robot chassis is shown in Fig. 3(c). ...
... In our previous study, a single LRF mounted cane-type omni-directional robot is designed for providing walking assistance and companionship based on the human-following control method [16]. In this paper, to ensure reliable gait analysis and rehabilitation evaluation under both the walking-aid and accompanying circumstances, two LRFs (URG-04LX-UG01, Hokuyo Automatic Co., Ltd.) are attached to the proposed cane robot and used to scan the environment in two horizontal planes, as shown in Fig. 1. ...
... Human's orientation θ h is defined as the person's positive facing direction, which can be estimated by the intention estimation algorithm proposed in [16]. The swing angles of the legs can be obtained in the sagittal and coronal planes of the human body, as shown in (4), ...
... The diagram of the gait analysis and evaluation system. Based on the human-following control method proposed in our former research [16], the cane robot can maintain a fixed relative posture D = [D x , D y , β] T with the target human, allowing the LRFs to obtain the accurate gait data. The fixed relative posture D can be customized under the rehabilitation physician's suggestion. ...
Gait analysis and evaluation are vital for disease diagnosis and rehabilitation. Current gait analysis technologies require wearable devices or high-resolution vision systems within a limited usage space. To facilitate gait analysis and quantitative walking-ability evaluation in daily environments without using wearable devices, a mobile gait analysis and evaluation system is proposed based on a cane robot. Two laser range finders (LRFs) are mounted to obtain the leg motion data. An effective high-dimensional Takagi-Sugeno-Kang (HTSK) fuzzy system, which is suitable for high-dimensional data by solving the saturation problem caused by softmax function in defuzzification, is proposed to recognize the walking states using only the motion data acquired from LRFs. The gait spatial-temporal parameters are then extracted based on the gait cycle segmented by different walking states. Besides, a quantitative walking-ability evaluation index is proposed in terms of the conventional Tinetti scale. The plantar pressure sensing system records the walking states to label training data sets. Experiments were conducted with seven healthy subjects and four patients. Compared with five classical classification algorithms, the proposed method achieves the average accuracy rate of 96.57%, which is improved more than 10%, compared with conventional Takagi-Sugeno-Kang (TSK) fuzzy system. Compared with the gait parameters extracted by the motion capture system OptiTrack, the average errors of step length and gait cycle are only 0.02 m and 1.23 s, respectively. The comparison between the evaluation results of the robot system and the scores given by the physician also validates that the proposed method can effectively evaluate the walking ability.
... This method has become an indispensable part of intelligent medical services [15]. The authors in [16] propose a walking assistant robot with walking stick to follow the user, which can assist and supervise patients with lower limb dysfunction to walk long distance through the estimation of human walking intention. However, this robot is only suitable for patients who recover to the stage of independent walking, and cannot provide patients with body weight support and more reliable safety protection. ...
In response to the current problem of low intelligence of mobile lower limb motor rehabilitation aids. This paper proposes an intelligent control scheme based on human movement behavior in order to control the rehabilitation robot to follow the patient’s movement. Firstly, a multi-sensor data acquisition system is designed according to the rehabilitation needs of the patient and the movement characteristics of the human body. A mathematical model of movement behavior is then established. By analyzing and processing motion data, the change in the center of gravity of the human body and the behavior intention signal are derived and used as a control command for the robot to follow the human body’s movement. Secondly, in order to improve the control effect of rehabilitation robot following human motion, an adaptive radial basis function neural network sliding mode controller (ARBFNNSMC) is designed based on the robot dynamic model. The adaptive adjustment of switching gain coefficient is performed by radial basis function neural network. The controller can overcome the influence caused by the change of robot control system parameters due to the fluctuation of the center of gravity of human body, enhance the adaptability of the system to other disturbance factors, and improve the accuracy of following human body motion. Finally, the motion following experiment of the rehabilitation robot is performed. The experimental results show that the robot can recognize the motion intention of human body and perform the training goal of following different subjects to complete straight lines and curves. The correctness of human motion behavior model and robot control algorithm is verified, which shows the feasibility of the intelligent control method proposed in this paper.
... Several methods have been put forward to minimize the influence of arm position changes in past studies, such as the multi-position classifier (Geng et al., 2012), cascade classifier (Geng et al., 2017), dynamic training (Shahzad et al., 2019), position-invariant features (Asogbon et al., 2020), and other classification algorithms. Multi-modal signals are usually needed for many methods, such as the accelerometry (ACC) signal, the near-infrared spectroscopy (NIRS) (Guo et al., 2015(Guo et al., , 2017, electroencephalography (EEG) (Leeb et al., 2011), FMG (Ferigo et al., 2017;Prakash et al., 2020a,b;Huang et al., 2021), and some industry sensors used for human-centered robotic systems Yan et al., 2021). Among these signals, the ACC is the most commonly used signal for the complementary of EMG (Geng et al., 2012;Huang et al., 2015;Shahzad et al., 2019), since the tri-axis ACC signal could provide information about arm position. ...
Robust classification of natural hand grasp type based on electromyography (EMG) still has some shortcomings in the practical prosthetic hand control, owing to the influence of dynamic arm position changing during hand actions. This study provided a framework for robust hand grasp type classification during dynamic arm position changes, improving both the “hardware” and “algorithm” components. In the hardware aspect, co-located synchronous EMG and force myography (FMG) signals are adopted as the multi-modal strategy. In the algorithm aspect, a sequential decision algorithm is proposed by combining the RNN-based deep learning model with a knowledge-based post-processing model. Experimental results showed that the classification accuracy of multi-modal EMG-FMG signals was increased by more than 10% compared with the EMG-only signal. Moreover, the classification accuracy of the proposed sequential decision algorithm improved the accuracy by more than 4% compared with other baseline models when using both EMG and FMG signals.
... In order to estimate the disturbance and realize the control stability of the mobile wheeled inverted pendulum systems, a novel sliding mode control based on high-order disturbance observer was developed (Huang et al., 2020). Further a cane type walkingaid robot is proposed (Yan et al., 2021), which follow a human user for the safety and supervision of independent walking during rehabilitation training. The autonomous positioning and navigational capabilities of mobile robots are critical in walking robots, and an accurate map is the basis of these capabilities in walking robots. ...
In this paper, a simultaneous localization and mapping algorithm based on the weighted asynchronous fusion of laser and vision sensors is proposed for an assistant robot. When compared to the synchronous fusion algorithm, this method can effectively use the redundant data in the vision sensor and improve the tracking accuracy of the algorithm. At the same time, the attitude estimation of the visual sensor is taken as a prior of the attitude estimation of the laser sensor to reduce the number of iterations and improve the efficiency of the algorithm. Further, according to the running state of the robot, a weighting coefficient based on angle is introduced to improve the confidence of the measurement. Experimental results show that the algorithm is robust and can work in a degraded environment. When compared to the synchronous fusion method, the asynchronous fusion algorithm has a more accurate prior, faster operation speed, higher pose estimation frequency, and more accurate positioning accuracy.
