Nick C. Tang

Academia Sinica · Institute of Information Science

Video-based group behavior analysis is drawing attention to its rich applications in sports, military, surveillance and biological observations. The recent advances in tracking techniques, based on either computer vision methodology or hardware sensors, further provide the opportunity of better solving this challenging task. Focusing specifically o...

The recent advances in imaging devices have opened the opportunity of better solving the tasks of video content analysis and understanding. Next-generation cameras, such as the depth or binocular cameras, capture diverse information, and complement the conventional 2D RGB cameras. Thus, investigating the yielded multi-modal videos generally facilit...

With the aim at accurate action video retrieval, we firstly present an approach that can infer the implicit skeleton structure for a query action, an RGB video, and then propose to expand this query with the inferred skeleton for improving the performance of retrieval. It is inspired by the observation that skeleton structures can compactly and eff...

We present a novel two-pass framework for counting the number of people in an environment where multiple cameras provide different views of the subjects. By exploiting the complementary information captured by the cameras, we can transfer knowledge between the cameras to address the difficulties of people counting and improve the performance. The c...

The recent advances in RGB-D cameras have allowed us to better solve increasingly complex computer vision tasks. However, modern RGB-D cameras are still restricted by the short effective distances. The limitation may make RGB-D cameras not online accessible in practice, and degrade their applicability. We propose an alternative scenario to address...

The recent advances in imaging devices have opened the opportunity of better solving computer vision tasks. The next-generation cameras, such as the depth or binocular cameras, capture diverse information, and complement the conventional 2D RGB cameras. Thus, investigating the yielded multi-modal images generally facilitates the accomplishment of r...

We propose a human motion extrapolation algorithm that synthesizes new motions of a human object in a still image from a given reference motion sequence. The algorithm is implemented in two major steps: contour manifold construction and object motion synthesis. Contour manifold construction searches for low-dimensional manifolds that represent the...

We present a framework to count the number of people in an environment where multiple cameras with different angles of view are available. We consider the visual cues captured by each camera as a knowledge source, and carry out cross-camera knowledge transfer to alleviate the difficulties of people counting, such as partial occlusions, low-quality...

In this paper, we propose a new framework to synthesize human motions based on only one single posture given in the input image. To generate visually pleasing motion sequences, the proposed framework consists of two key techniques. One is motion retrieval, which retrieves reference motions from a human motion database on a low-dimensional motion ma...

Video forgery, also referred as video falsifying, is a technique for generating fake videos by altering, combining, or creating new video contents. For instance, the outcome of a 100 m race in the olympic game is forged as an example in this paper. We track objects and segment motions using a modified mean shift mechanism. The resulting video layer...

Video inpainting is an important video enhancement technique used to facilitate the repair or editing of digital videos. It has been employed worldwide to transform cultural artifacts such as vintage videos/films into digital formats. However, the quality of such videos is usually very poor and often contain unstable luminance and damaged content....

Storytelling and narrative creation are very popular research issues in the field of interactive media design. In this paper, we propose a framework for generating video narrative from existing videos which user only needs to involve in two steps: (1) select background video and avatars; (2) set up the movement and trajectory of avatars. To generat...

In this section, we discuss about the technology of video forgery. There are three technologies we will discuss. The first is inpainting, inpainting is an algorithm developed for a long time. It includes image inpainting, video inpainting, and motion inpainting. Exemplar-Based Image Inpainting[1] is a technique which be proposed in 2004. It complet...

In this paper, we discuss whether there is any possibility of hiding data in video. Testing our theory, we propose a novel video hiding scheme based on LSB, and we use three-dimensional array to auxiliary our scheme, adaptive LSB data hiding technique, which is a typical method used to achieve information hiding in images, and three-dimensional arr...

Video inpainting is often used as a tool to assist user in removing objects or repairing damaged areas in a video. To deal with different kinds of video, several techniques such as object segmentation and temporal continuity maintenance are commonly adopted. In this paper, we extend the concept of exemplar-based inpainting to propose a new video in...

Face image partially occluded or damaged can be repaired automatically. We proposed a new inpainting algorithm, based on patch guidance deduced from an existing face database, to recover the damaged portions. This newly proposed concept of guided inpainting method produces seamless faces which are hardly seen drawbacks. Examples of our results can...

Image inpainting or image completion is the technique that automatically restores/completes removed areas in an image. When dealing with a similar problem in video, not only should a robust tracking algorithm be used, but the temporal continuity among video frames also needs to be taken into account, especially when the video has camera motions suc...

Images with human faces comprise an essential part in the imaging realm. Occlusion or damage in facial portions will bring a remarkable discomfort and information loss. We propose an algorithm that can repair occluded or damaged facial images automatically, named ‘facial image inpainting’. Inpainting is a set of image processing methods to recover...

Video Forgery is a technique for generating fake video by altering, combining, or creating new video contents. The procedure of producing fake video involves several challenge image and video processing techniques. We discuss these techniques as follows: The first challenge to alter the behavior of actors (or objects) in the original video involve...

Special effect production is an important technology for movie industry. Although sophisticated equipments can be used to produce scenery such as fire and smoke, usually, the approach is expensive and perhaps dangerous. Can sceneries be re-produced based on existing videos using digital technology? The answer is yes but it is difficult. In this vid...

In this paper, a novel image-based method of distance measurement is proposed. Traditionally, the parallax method is frequently used to measure the distance between the camera and the measured surface. But for diminishing the computational complexity and inaccuracy cause by image processing procedure, the proposed system only using two parallel las...

In recent years, facial recognition techniques can be applicable in varies fields and has significant results. In obtaining facial images, if the face image were covered or damaged, the degree of error and inaccuracy will increase significantly. Therefore, the purpose of this paper is not only for extracting facial features, but also to recover the...

Image inpainting technique is that repairs damaged area or remove areas in an image. In order to deal with this kind of problems, not only a robust image inpainting algorithm should be used, technique of structure generation is also needs to adopt in inpainting procedure. In this paper, we extend an exemplar-based image inpainting method by incorpo...

This survey paper discusses research issues of digital inpainting and video falsifying. Image inpainting /completion methods use textural synthesis and pixel interpolation techniques to remove defects or user selected bjects. Some image inpainting techniques also can be applied to 3D surfaces, to cope with holes not covered by a 3D scanner. On the...

In this paper, an adaptive approach to skeleton detection is presented.First, we proposed a method to combine thinning and tracking algorithm which can find the skeleton tracking points, and the points would be reconstructed by using multiple cameras to obtain the 3D coordinates the X, Y, and Z-axis coordinates. The coordinates can be converted int...

We propose a novel motion interpolation algorithm by using the mean-shift segmentation and motion analysis technique. Mean shift segmentation is frequently used to extract objects from video according to its efficiency and robustness of non-rigid object tracking. For diminishing the computational complexity in motion estimation and object tracking...

Video inpainting is the unique technique that can automatically restore damaged or partially removed image.It is also the tool for filling in the missing part in a video sequence. Exploration of more advanced concepts in video painting, this paper is intended to develop a new video algorithm based on temporal continuations and exemplar-based image...

In this paper, we propose a new inpainting algorithm based on color segmentation to preserve the structure of image. We use color segmentation to construct contour lines, and estimate the slope of contour lines and how they should extend. We also use the advantage of Bezier curves that fit the curves to the real world. The algorithm also fit to mor...

We change the behavior of actors in a video. For instance, the outcome of a 100-meter race in the Olympic game can be falsified. We track objects and segment motions using a modified mean shift mechanism. The resulting video layers can be played in different speeds and at different reference points with respect to the original video. In order to ob...

We create video scenes using existing videos. A panorama is generated from background video, with foreground objects removed by video inpainting technique. A video planning script is provided by the user on the panorama with accurate timing of actors. Actions of these actors are extensions of existing cyclic motions, such as walking, tracked and ex...

Video Falsifying is a technique for generating fake video by altering, combining, or creating new video contents. For instance, the outcome of a 100-meter race can be changed. Alternatively, the winner of a horse race can be created from combined videos. The technique is for special effect production in movie industry. We demonstrate how to use our...

We present a hyper-interaction platform which integrates several techniques, including motion tracking, motion recognition, a motion reaction control mechanism, and a high precision 3-D model of the International Space Station. The goal is to build an interaction platform for trainers to navigate in a virtual reality. The platform can be applied to...

Image in-painting or image completion removes objects from a photo and automatically produces a visually pleasant result. However, to remove objects from a video, the resulting video may have ghost shadows even each individual frame is in-painted properly. We use motion estimation algorithm to separate objects and backgrounds into several layers. O...

This video program presents several video inpainting/implant mechanisms implemented under different types of videos with diversified temporal continuations. We use an improved image inpainting method to remove foreground objects. Another set of objects are implanted into the inpainted video. Detailed algorithms are presented in a paper in the 2006...

Recent interesting issues in video inpainting are defect removal and object removal. We take one more step to replace the removed objects in a video sequence by implanting objects from another video. Before implant, we improve an exemplar-based image inpainting algorithm by using a new patch matching strategy which incorporates edge properties. The...

The adaptive urban traffic signal control (TSC) system became a development trend of intelligent transportation system (ITS). We investigated the vision based surveillance and to keep sight of the unpredictable and hardly measurable disturbances may perturb the traffic flow. We integrated and performed the vision based methodologies that include th...

Hand gesture recognition is a new challenge to convey information such as input data or to control devices and applications such as computers, games, PDAs, browsers, cell phones, information appliances and MP3 audio players, etc. Image-based techniques detect a gesture by capturing pictures of a user's hand signs during the course of a gesture, and...