Fig 2 - uploaded by Hongyi Li
Content may be subject to copyright.
Source publication
In teleoperation tasks, information about the relative posture between end-effector and the object to be grasped is of key importance for human operators. Although visual information plays a major role in monitoring collision and fun-tuning the end-effector towards the object, the operator should make strict observations about the video images to t...
Contexts in source publication
Context 1
... section goes into details about the hardware implementation of the experimental system. Since both client system and robotic system are located at Shenyang Institute of Automation, an intermediate note which is at Shijiazhuang Railway Institute is adopted to simulate the network environment. Both control data and feedback information will pass through that note. The system is illustrated in Fig. 1. After connecting to the robot control module, the client sends velocity commands to the remote robot and displays force on the master device once the relative posture between the end-effecter and the object is received. Communication between the client PC and the master device is achieved with the IEEE 1394 interface. The screen display interface consists of two sections. The upper gives two video streams from the aspectant camera and the lateral camera respectively. It should be noticed that although force/haptic feedback plays an important role in improving performance of telerobotic systems, visual feedback can not be replaced by force/haptic feedback. Since video stream can be used to provide global information about the work space. The lower is a control panel that is used to display the control mode of the remote robot, including position and attitude control mode. In position control mode, position of the end-effecter will be controlled and the attitude is invariant. While in attitude control mode, attitude of the end-effecter will be controlled and the position is fixed. The interface is depicted in Fig. 2 (a). In the client side, a PHANToM Omni model from SensAble Technologies is selected as the master device. The PHANToM Omni can be operated under standard operating system such as Linux, Windows NT, 2000 and XP and provides high fidelity 3D force feedback. On the other hand, PHANToM Omni models are high-precision instruments with large workspace and high force. Furthermore, it has optical position encoders to read the gimbal position, as well as three DC actuators to provide translating forces to the user’s hand. As opposed to position-controlled manipulators PHANToM Omni models are fully back-drivable, such that the user will not feel any forces as long as there is no interaction in the virtual world [2]. The low inertia and friction of this gravity-compensated arm can bring a very crisp, high quality haptic feedback effect. The PHANToM Omni model is illustrated in Fig. 2 (b). At the remote site, a 6-DOF MOTOMAN industrial robot, which is a typical robotic system and is widely equipped in industry and laboratories, is used as the slave robot. The robot controller runs on a PC platform under QNX real-time operating system. The control softtware is composed of several processes among which there is a particular process which is responsible for communication with the client. Commands coming from the client is received by the communicating process and sent to the servo to actuate motors of the robot after manipulation by other processes. In order to obtain the posture of the end-effecter relative to the object, a MINTRON CCD camera is fixed on the wrist of the robot. A PC (Pentium III, Windows XP) is used as the image processing computer which has access to the client computer through Internet. It is noted that the speed for image processing and computing plays an important role in enhancing the performance of force feedback in the client module. Consequently, the multi-thread technology is used in the image processing unit. As mentioned before, force/haptic information feedback can not take the place of video feedback completely. We need to help the operator to observe the current situation of the remote work site. For this reason, two video cameras are mounted in the work site and used to provide lateral view and aspectant view of the environment respectively, as depicted in Fig. 3 (a) and (b). In this module, the VIC (Video Conferencing Tool) is used to transmit video streams to the client. This video conferencing application was developed by the network Research Group at the Lawrence Berkeley National Laboratory in collaboration with the University of California, Berkeley [9]. The operation of the system is described as follows: once relative posture between the end-effecter and the object is received, the force obtained according to visual-to-force transformation method will be displayed on the PHANToM Omni device. At the same time, state data is converted to the control command and transmitted to the remote robot through the Internet. As a result, acquisition of the relative posture and the client interface are two important components of the system. The image processing application is written in Visual C++ and produces relative posture data at a rate of 10 Hz. The client application is also developed with Visual C++ on a Windows XP platform with the help of HDAPI [2]. To improve performance of force feedback, the PHANToM Omni maintains a high priority thread called servo loop. This thread is executed at a consistent 1kHz. IV. A CQUISITION OF THE R ELATIVE P OSTURE BETWEEN E ND - EFFECTER AND THE O BJECT TO B E G RASPED In this section, the relative posture obtained based on a model-based posture estimation method will be introduced. The PnP problem is stated as follows: given a set of points with known coordinates in an object-centered frame and their corresponding projections onto an image plane and given the intrinsic camera parameters, finding the transformation matrix. The P3P problem is the smallest subset of control points that yields a finite number of solutions. In Fig. 4, O is the perspective center, the side length of the triangle is a , b ,and c respectively. Let OA x , OB y , OC z . According to cosine theorem, we have b 2 x 2 z 2 2 xz cos ...
Context 2
... section goes into details about the hardware implementation of the experimental system. Since both client system and robotic system are located at Shenyang Institute of Automation, an intermediate note which is at Shijiazhuang Railway Institute is adopted to simulate the network environment. Both control data and feedback information will pass through that note. The system is illustrated in Fig. 1. After connecting to the robot control module, the client sends velocity commands to the remote robot and displays force on the master device once the relative posture between the end-effecter and the object is received. Communication between the client PC and the master device is achieved with the IEEE 1394 interface. The screen display interface consists of two sections. The upper gives two video streams from the aspectant camera and the lateral camera respectively. It should be noticed that although force/haptic feedback plays an important role in improving performance of telerobotic systems, visual feedback can not be replaced by force/haptic feedback. Since video stream can be used to provide global information about the work space. The lower is a control panel that is used to display the control mode of the remote robot, including position and attitude control mode. In position control mode, position of the end-effecter will be controlled and the attitude is invariant. While in attitude control mode, attitude of the end-effecter will be controlled and the position is fixed. The interface is depicted in Fig. 2 (a). In the client side, a PHANToM Omni model from SensAble Technologies is selected as the master device. The PHANToM Omni can be operated under standard operating system such as Linux, Windows NT, 2000 and XP and provides high fidelity 3D force feedback. On the other hand, PHANToM Omni models are high-precision instruments with large workspace and high force. Furthermore, it has optical position encoders to read the gimbal position, as well as three DC actuators to provide translating forces to the user’s hand. As opposed to position-controlled manipulators PHANToM Omni models are fully back-drivable, such that the user will not feel any forces as long as there is no interaction in the virtual world [2]. The low inertia and friction of this gravity-compensated arm can bring a very crisp, high quality haptic feedback effect. The PHANToM Omni model is illustrated in Fig. 2 (b). At the remote site, a 6-DOF MOTOMAN industrial robot, which is a typical robotic system and is widely equipped in industry and laboratories, is used as the slave robot. The robot controller runs on a PC platform under QNX real-time operating system. The control softtware is composed of several processes among which there is a particular process which is responsible for communication with the client. Commands coming from the client is received by the communicating process and sent to the servo to actuate motors of the robot after manipulation by other processes. In order to obtain the posture of the end-effecter relative to the object, a MINTRON CCD camera is fixed on the wrist of the robot. A PC (Pentium III, Windows XP) is used as the image processing computer which has access to the client computer through Internet. It is noted that the speed for image processing and computing plays an important role in enhancing the performance of force feedback in the client module. Consequently, the multi-thread technology is used in the image processing unit. As mentioned before, force/haptic information feedback can not take the place of video feedback completely. We need to help the operator to observe the current situation of the remote work site. For this reason, two video cameras are mounted in the work site and used to provide lateral view and aspectant view of the environment respectively, as depicted in Fig. 3 (a) and (b). In this module, the VIC (Video Conferencing Tool) is used to transmit video streams to the client. This video conferencing application was developed by the network Research Group at the Lawrence Berkeley National Laboratory in collaboration with the University of California, Berkeley [9]. The operation of the system is described as follows: once relative posture between the end-effecter and the object is received, the force obtained according to visual-to-force transformation method will be displayed on the PHANToM Omni device. At the same time, state data is converted to the control command and transmitted to the remote robot through the Internet. As a result, acquisition of the relative posture and the client interface are two important components of the system. The image processing application is written in Visual C++ and produces relative posture data at a rate of 10 Hz. The client application is also developed with Visual C++ on a Windows XP platform with the help of HDAPI [2]. To improve performance of force feedback, the PHANToM Omni maintains a high priority thread called servo loop. This thread is executed at a consistent 1kHz. IV. A CQUISITION OF THE R ELATIVE P OSTURE BETWEEN E ND - EFFECTER AND THE O BJECT TO B E G RASPED In this section, the relative posture obtained based on a model-based posture estimation method will be introduced. The PnP problem is stated as follows: given a set of points with known coordinates in an object-centered frame and their corresponding projections onto an image plane and given the intrinsic camera parameters, finding the transformation matrix. The P3P problem is the smallest subset of control points that yields a finite number of solutions. In Fig. 4, O is the perspective center, the side length of the triangle is a , b ,and c respectively. Let OA x , OB y , OC z . According to cosine theorem, we have b 2 x 2 z 2 2 xz cos ...
Similar publications
The goal of the CoSy project is to create cognitive robots to serve as a testbed of theories on how humans work (13), and to identify problems and techniques relevant to producing general-purpose human- like domestic robots. Given the constraints on the resources available at the robot's disposal and the complexity of the tasks that the robot has t...
Citations
The most important tool for controlling an industrial robotic arm is a teach pendant, which controls the robotic arm movement in work spaces and accomplishes teaching tasks. A good teaching tool should be easy to operate and can complete teaching tasks rapidly and effortlessly. In this study, a new teaching system is proposed for enabling users to operate robotic arms and accomplish teaching tasks easily. The proposed teaching system consists of the teach pen, optical markers on the pen, a motion capture system, and the pen tip estimation algorithm. With the marker positions captured by the motion capture system, the pose of the teach pen is accurately calculated by the pen tip algorithm and used to control the robot tool frame. In addition, Fitts' Law is adopted to verify the usefulness of this new system, and the results show that the system provides high accuracy, excellent operation performance, and a stable error rate. In addition, the system maintains superior performance, even when users work on platforms with different inclination angles.
Human motor and cognitive behavior has been considered as an important research content and received increased attention from academia. As a suitable platform, the inverted pendulum under manual control has been studied extensively, since the inverted pendulum is essentially a nonlinear system and strong coupling exists between the human operator and pendulum system. However, performance of the system when feedback force is provided has not been reported. This paper presents a virtual pendulum system which is built based on OpenGL, while control inputs are provided with a joystick. Dynamical model has been obtained though mathematical manipulation. In order to obtain more realistic effect, dynamical model is solved with Runge-Kutta method. The joystick interface enables operators to control the pendulum manually, providing a fun experience. Comparative experiments have been carried out and the preliminary results confirm the performance difference when force/haptic information is provided to the operator. This study establishes the foundation for further research on cognitive behavior of human operators in human-machine interaction systems and neural control systems.
Operability/transpancy is of vital importance in telerobotic systems. For fear of collision between the end-effecter and the objects, human operator must make strict observation on the visual image and carefully plan small pieces of motion of the robot at a time. This makes operability worse. Good force telepresence can reduce the workload of human operator, overcome the limitation of vision information, and improve the operability performance of teleoperation system. In this paper, we describe a novel virtual force feedback control approach to enhance force telepresence and then make operability better. Based on accurate measurement of relative posture with a monocular vision, we first perform camera calibration to detect the relationship between pixel points in the image and points in the real object. Secondly, with the information transformation technique, the virtual reflective force is generated. Finally, virtual force feedback is used to improve the performance of operability. Experiments are made to demonstrate the effectiveness of posture-based virtual force feedback guided control method in telerobotic system.
Telepresence is a critical problem in robotic teleoperation systems. For the sake of collision avoidance between the end-effecter and the object, the human operator has to make close observations on the visual information. This overloads the visual sense of the human operator. On the other hand, there are some potential obstacles in gathering vision information. As a result, the advance notice of collision might be missed or neglected. In order to settle this issue, a virtual force guided control method for robotic teleoperation system is proposed in this paper. This approach combines position-force command strategy and virtual force feedback to perform proper operations, where the virtual force is generated by transforming visual information into reflective force. This control approach improves the operability of teleoperation system and enhances the effectiveness. Experiments are conducted and demonstrate the effectiveness of the virtual force guided control method.
