Sketch drawing examples of Chinese Kungfu forms. 

Context in source publication

Context 1

... thumbnails, through its images and annotations, describes how various joints move in relation to one an- other in the overall figures. The earlier work on computer- based motion choreography can be traced back to the end of 1970. Brown and Smoliar developed a graphical tool to assist the editing and creation of Labanotation sequence for human choreographer [BS76]. In light of the iterative and interactive nature of the choreographic process, Calvert et al investigated how to support the choreographer in compositional process, and developed a visual idea generator with intuitive interface of Stage-, Body-, and Stance- Ed- ∗ itors [SCLG90] [CBM 93]. Most of these existing efforts focus on movement notating, and intuitive interfaces to de- vise and create a notation-based choreography. They often require the end user to newly learn a notating system for character movement choreographing. However, sketch-drawings are a natural choice for the creation of graphical media such as character movement, as the artists can quickly and easily sketch 2D figures, and traditional animators often begin work by quickly sketching thumbnails of a character in key poses to capture the charac- ter’s overall motion [DAC 03]. Our goal here is to provide design support such as movement exploring, reconciling, juxtaposing and synthesizing during the process of creating character motions. We choose Chinese Kungfu, a popular theme in feature film and video game industry, as motion domain. Chinese Kungfu motions are natural and aesthetic movements of attack and defense, such as the punch, strike, hook, block, chop and various kicks, and these martial art techniques have been handed down by drawings, or pictures with text explanation for hundreds of years. As shown in Figure 2, a typical martial art form is made of an initial posture, intermediate movements (attack and/or defense) and a closing posture. The initial and closing postures are often represented as hand-drawn human figures. The trajectories of hit joints within a Kung-fu form are often indicated by cursive arrowed lines. In order to assist the conception and development of Kungfu forms, a sketch-based motion authoring and choreographing system, MotionMaster, is proposed. Its input is solely based on sketch-drawings. The principle behind it is that MotionMaster should be able to rough out realistic character motions at the early stage of motion design, as the rapid-prototyped 3D movements helps the motion composer not only to evaluate the choreography work, but also to fol- low and develop the intent of movement idea in an animation sequence. Such a Kung-fu motion authoring problem can be formalized as follows: P T is T-pose skeleton of a specific character, P A and P B are the initial and closing postures of a Kungfu forms respectively(see Figure 1), S is a set of cursive lines that indicate the trajectory of joints during the intermediate movements in current Kung-fu form. Based on the reference posture P T , the user interactively specifies the joint correspondence relationship of P A and P B accordingly. The system will automatically infer the desired 3D motion sequence that matches the input P A , P B and S. The remainder of this paper is organized as follows. We first review the related work in Section 2, and explain the pipeline and scheme of sketch-based motion authoring and choreographing in Section 3. Then we describe how to generate the best-fit motion segments from a motion database in Section 4. Section 5 presents how to iteratively optimize and refine the motion segments in question. We show the experimental results and conclude with a discussion of our approach in Section 6 and 7. The understanding of Kung-fu motions via sketch drawings involves the recognition of initial and closing postures, and the trajectory recovery of moving joints in the intermediate movements within the Kungfu form. The relevant work in computer vision and animation community can be described by three main categories as follows. 3D posture recovery from a single view is ill-defined due to insufficient spatial information. Its primary challenge is that many 3D poses may be consistent with a given 2D stick figure. In computer vision community, the early work on the problem of reconstructing 3D poses from a monocular video sequence can be traced back to 1980s [LC85]. Several recent surveys on these techniques have been explored in [Gav99] [MG01]. In general, there are two main approaches, namely, model-based and learning-based, to the 3D pose reconstruction problem. The model-based approach assumes that a 3D skeleton is known a priori, and uses an articulated human model to generate possible 3D postures that match the 2D human figure. In order to obtain the best 3D posture, a set of physical, environmental, or dynamic constraints is then applied to cull invalid 3D postures generated ini- tially [Tay00] [DCR01]. In learning-based approaches, they automatically learn a mapping between features in the 2D image and that in the 3D posture through stochastic learning processes [RSR01]. It requires a large set of training data to learn prior correspondence knowledge of posture features in 2D image and specific 3D motion. By taking the guiding data set in the learning-based approach and a priori knowledge of human model and constraints in the model-based approach, Chiu et al exploits a posture library and constraints to guide the reconstruction work. This hybrid approach will retrieve an appropriate candidate from the library in terms of the similarity of the 3D posture’s 2D projection and the human figure in the image ∗ [CWW 04]. The 3D pose recovery techniques used here is in spirit similar to the one used in our system. However, our system goes significantly beyond theirs. we rapidly reconstruct the entire 3D motion sequence expressed in the 2D sketches, in stead of the 3D postures. A useful and common approach to interfacing 2D-3D animations is to provide an intuitive sketch-based interface, as hand drawn animation sketches provides a flexible and simple means of "roughing out" expressive animated characters. In the sketch-based motion generation, Sabiston attempted to pose stick skeleton figure from a series of 2D thumbnail sketches using a simple "flipbook" analogy. Assuming the animator has a reasonable knowledge of perspective and foreshortening, the system is able to determine the joint angles for the 3D figure by calculating the amount of foreshortening on hand-drawn limbs [SAB91]. Davis et al provides a simple sketching interface for articulated figure animation. The user draws the skeleton on top of the 2D sketch, and then the system constructs the set of poses that exactly match the drawing, It also allows the user ...