Figure 2 - available from: Signal Image and Video Processing
This content is subject to copyright. Terms and conditions apply.
Source publication
Lack of eye gaze in the video conferencing system severely hinders immersive and comfortable communication. In this paper, we propose an eye gaze correction system for immersive video conferencing. Our system consists of a full HD display and two cameras mounted on the top and bottom of the display. In order to correct the eye gaze, we warp two cap...
Similar publications
The concept of supporting ad hoc or dynamic membership tele-present meetings through pulling up a chair is novel. In real world business situations, people pull up a chair after catching the eye of someone already seated. Telethrone is a situated display on a chair which allows multiple correct views of a remote collaborator. The system has been ex...
Gaze redirection aims at manipulating a given eye gaze to a desirable direction according to a reference angle and it can be applied to many real life scenarios, such as video-conferencing or taking groups. However, the previous works suffer from two limitations: (1) low-quality generation and (2) low redirection precision. To this end, we propose...
Gaze correction is a type of video re-synthesis problem that trains to redirect a person's eye gaze into camera by manipulating the eye area. It has many applications like video conferencing, movies, games and has a great future in medical fields such as to experiment with people having autism. Existing methods are incapable of gaze redirection of...
Citations
... For instance, work done by Baek and Ho [2] requires a high definition setup of a display and a set of four cameras two mounted on top and two mounted at the bottom of the display. The camera interpolates between the viewpoints obtained by the cameras on top and bottom to generate a morphed face which renders a mutual gaze aligned view to the remote viewer. ...
... Work done by Ford D.A. and Silbermam [9] involves extracting user's facial features such as eyes and substituting a corrected version of the feature to render a gaze corrected perspective of an individual participant. Likewise, work done by Eu-Tteum Baek and Yo-Sung Ho [2] involves morphing facial features to render eye-contact between interacting participants during video conferencing. Work done by Ruigang Yang and Zhengyou Zhang [32] involves a graphics hardware that syntheses eye contact based on stereo analysis combined with rich domain knowledge to synthesize a video that maintains eye contact. ...
Existing live tele-teaching systems enable eye-contact between interacting participants, however, they are often incomplete as they neglect finer levels of adherence to gaze such as gaze awareness and gaze following. A multilocation eLearning classroom setting often does not preserve relative neighborhood i.e., displays showing videos of remote participants at each location might not be congruent with their actual seating positions. This leads to incoherent gaze patterns during interactions. We present a media-rich distributed classroom architecture with multiple cameras and displays in each classroom. During interaction changes, cameras capturing appropriate perspectives of participants are streamed to displays in other classrooms. Hence for all interactions, the physical participants of a classroom are presented with appropriate perspectives of remote participants resembling gaze patterns during conventional-classroom interactions. We also present a framework to systematically analyze gaze patterns with its dependencies. The framework dictates optimal placement of media devices ensuring minimal deviation in capturing appropriate perspectives for a given set of resources. Evaluation results on a three classroom test-bed indicates a marked reduction in viewer cognitive load in discerning the entity-at-focus in an eLearning classroom environment.
Although live video communication is widely used, it is generally less engaging than face-to-face communication because of limitations on social, emotional, and haptic feedback. Missing eye contact is one such problem caused by the physical deviation between the screen and camera on a device. Manipulating video frames to correct eye gaze is a solution to this problem. In this article, we introduce a system to rotate the eyeball of a local participant before the video frame is sent to the remote side. It adopts a warping-based convolutional neural network to relocate pixels in eye regions. To improve visual quality, we minimize the L2 distance between the ground truths and warped eyes. We also present several newly designed loss functions to help network training. These new loss functions are designed to preserve the shape of eye structures and minimize color changes around the periphery of eye regions. To evaluate the presented network and loss functions, we objectively and subjectively compared results generated by our system and the state-of-the-art, DeepWarp, in relation to two datasets. The experimental results demonstrated the effectiveness of our system. In addition, we showed that our system can perform eye-gaze correction in real time on a consumer-level laptop. Because of the quality and efficiency of the system, gaze correction by postprocessing through this system is a feasible solution to the problem of missing eye contact in video communication.
