^{1}

^{2}

^{1}

^{2}

Example-based mesh deformation techniques produce natural and realistic shapes by learning the space of deformations from examples. However, skeleton-based methods cannot manipulate a global mesh structure naturally, whereas the mesh-based approaches based on a translational control do not allow the user to edit a local mesh structure intuitively. This paper presents an example-driven mesh editing framework that achieves both global and local pose manipulations. The proposed system is built with a surface deformation method based on a two-step linear optimization technique and achieves direct manipulations of a model surface using translational and rotational controls. With the translational control, the user can create a model in natural poses easily. The rotational control can adjust the local pose intuitively by bending and twisting. We encode example deformations with a rotation-invariant mesh representation which handles large rotations in examples. To incorporate example deformations, we infer a pose from the handle translations/rotations and perform pose space interpolation, thereby avoiding involved nonlinear optimization. With the two-step linear approach combined with the proposed multiresolution deformation method, we can edit models at interactive rates without losing important deformation effects such as muscle bulging.

Editing and animation of character models is an important task in computer graphics. The user, in general, demands the editing procedure to be interactive and intuitive. However, the ability to pose a model naturally and realistically is equally important for a character modeling system. Example-based techniques are a promising approach to improve the realism of interactive editing techniques. Pose space deformation (PSD) [

In contrast, the mesh-based inverse kinematics (MeshIK) technique [

In this paper, we present an example-driven mesh deformation technique that allows us not only to edit a global pose naturally but also to adjust a local pose precisely. Our system provides both translational and rotational controls to deform a model. By translating handle vertices, the user can produce a model in natural poses easily. By applying rotations to handle triangles, the user can bend and twist the model as if manipulating a joint. To this end, we formulate the problem by inferring a pose from handle translations/rotations and using a two-step linear surface optimization technique that is based on a linear rotation-invariant mesh representation. The benefits of this formulation are as follows: (1) it is straightforward to incorporate translational and rotational constraints; (2) it achieves rotation invariance and thus is applicable to examples that involves large deformations (rotations and bending); (3) it avoids time-consuming nonlinear optimization. In combination with the proposed multiresolution approach, we can edit high-resolution meshes at interactive rates while retaining important deformation effects such as muscle bulging.

The rest of the paper is organized as follows. In Section

Pose space deformation (PSD) [

To create realistic animation, some methods [

Linear differential surface deformation methods (see, e.g., [

In contrast, nonlinear methods solve these problems by iteratively optimizing both positions and transformations and thus work for large deformations at the cost of additional computations [

The mesh-based inverse kinematics (MeshIK) system edits a mesh by manipulating handle points while constraining deformation in example space [

Deformation transfer [

Our method is most similar to the variational PSD proposed by Weber et al. [

Our goal is to develop a mesh editing system that can manipulate a pose of the model both globally and locally. An overview of our method is depicted in Figure

Overview of our method. (a) Global pose editing is achieved with the translational control. (b) Local pose editing is done using the rotational control. (c) Flow of deformation.

Our method is based on pose space interpolation and a two-step surface optimization method. The workflow is depicted in Figure

Inputs to our system are the base mesh and example meshes. The base mesh

Our system is based on the pose space deformation framework that is thought of as a combination of surface deformation and shape interpolation. To integrate these two effectively, we use a two-step optimization method based on a rotation-invariant mesh representation similar to those proposed in [

Consider triangle

The deformation gradient is translation invariant but not rotation invariant, which means that it is affected by global rotations. To address this issue, we first decompose the deformation gradient into rotation and scale/shear components by polar decomposition, that is,

Rotation-invariant encoding. We first compute the deformation gradients. We then decompose these into absolute rotations and scale/shear components. While the scale/shear components are rotation invariant, the absolute rotations are not. Thus, we compute a relative rotation of two adjacent triangles to remove a global rotation.

To reconstruct vertex positions from rotation-invariant encodings, we solve two sparse linear systems. We use the penalty method [

To compute the new absolute rotation

Having obtained

Once the deformation gradients

In this section, we introduce two basic surface manipulation tools for formulating our pose space deformation.

To deform a surface, we specify handle vertices/triangles on the mesh and apply translations/rotations to them (Figure

Handle-based surface deformation. We manipulate handle vertices/triangles on the surface by applying translations/rotations to them. The mesh deforms according to handle movements while preserving the original details. Note that, because we optimize rotations and vertices separately, rotation constraints and translation constraints should be compatible with each other, otherwise, it produces distortions. Without incorporating examples, our method deforms the model smoothly regardless of anatomical structures.

Comparison of interpolation results. We perform interpolation on two meshes in different orientations:

The advantage of performing interpolation using the rotation-invariant encoding is that it is robust to large rotations. However, when simply interpolating the components of

We concatenate

The rotation-invariant encoding is robust-to-extreme orientation differences of input models [

Now, we are ready to formulate our pose space surface manipulation technique by unifying the handle-based surface deformation and shape interpolation methods described in the previous section. To integrate these two, rotation-invariant encodings must be altered according to handle movements, that is, a technique to associate handle translations/rotations with interpolation weights must be developed.

To obtain interpolation weights, the original MeshIK [

To evaluate the pose of the model, we need the measure that is rotation invariant. To this end, we compute relative positions and rotations from handles (Figure

We infer a pose of a model from relative positions and rotations computed from handle translations/rotations.

When editing with the translational control, the user places a coordinate frame and handle vertices on the mesh. The frame is placed on a triangle that exists around the trunk of the model and is used to form a local coordinate to compute relative positions of handles. This way, we can eliminate the effect of global rotations and can evaluate the pose successfully. In addition, our system allows the user to manipulate global orientation and position of the model by rotating and translating the frame.

When editing with the rotational control, the user places a pair of handle triangles called

Next, we compute the interpolation weights from the pose using a scattered data interpolation method. The interpolation weights are obtained by evaluating the pose computed from handle translations/rotations, in the pose space formed by example poses. We would like to compute the interpolation weights such that they satisfy the following criteria.

At an example point, the weight for that example must be one, and all other weights must be zero.

The weights must add up to 1.

The weight must change continuously to produce smooth animation.

The absolute value of the weight must be small to avoid excessive exaggeration of deformation.

For this task, we use

Let

After computing interpolation weights, we use (

By incorporating examples, our method can edit the model naturally and capture deformation effects such as muscle bulging. However, solving two large linear systems is still relatively time consuming, mainly due to the rotation optimization, which requires performing factorization for every frame. In addition, it requires a relatively large memory space (

Our observation is that, if we layer a surface with a coarse mesh and fine details, then deformation induced by pose changes mainly affects the underlying coarse mesh and has little effect on the fine details. In fact, Weber et al. [

Inspired by Sumner et al. [

To compute the weight

To construct a coarse mesh, we use a variant of the quadratic mesh simplification method (Qslim) [

Incorporating the above technique into the pose space surface manipulation framework is straightforward. We prepare coarse examples and store their rotation-invariant encodings. We also store harmonic weights and fine details that are the vectors emanating from the vertex of a coarse mesh to that of a dense mesh,

We implemented the pose space surface manipulation technique on an Intel Core2 Quad Q9400 2.66 GHz machine. We used the sparse Cholesky solver provided with the CHOLMOD [

In the accompanying video, we show use cases of the pose space surface manipulation framework for interactive editing of mesh models. Figure

Editing a horse (1k triangles) with translations and rotations. By translating the handles on the feet, we can pose the horse naturally (a). Once the overall pose is determined, the user edit the pose locally with the rotational control, if required (b). The blue region is frozen so that deformations remain local.

Using our translational editing method, we can create animation sequences. As shown in Figure

With 6 coarse examples (1k triangles) (a), we can create walking elephant (30k triangles) by translating handles on the feet (b).

Posing of the performer model (dense: 20k triangles, coarse: 1k triangles). We place a frame on the back and handles on the feet (a). We first adjust the global orientation of the performer (b). We can pose the performer in a running sequence by translating the handles (c).

Figure

Editing of highly nonrigid objects. (a) We provide 14 examples (1k triangles) to edit a cloth. (b) By just dragging the lower right corner of the model to apply translations, we can animate the cloth realistically.

Figure

Twisting of the shoulder. We provide 9 examples (a) and are able to obtain natural results using the rotational control (b). The rotational control is more intuitive for controlling shoulder movements than the translational control.

In Figure

Locations of triangles. Our method works even when the handle triangle is placed distally (b), proximally (c), or on the region where a muscle bulges (d).

In Figure

Comparison with the transformation propagation approach [

Comparison with the material modulation method [

Next, we compared our method using translational controls with MeshIK [

Edits of a bar model with the translational control. (a) We first provide two examples (gray). The shape in-between can be created meaningfully. However, it causes shearing distortions when we deform the bar in the vertical direction. (b) By providing another example in this direction, we can produce a natural result. (c) Because the distance-based interpolation method lacks the extrapolation capability, local distortions around handles occur when the pose is outside of the interpolation range.

If we use deformation gradient (Def Grad) that was used in MeshIK [

The effect of using a small weight for position constraints. Using a small value for

Storing rotation-invariant encodings requires a relatively large amount of memory. To solve this problem, we propose a multiresolution approach. In Figure

Interpolation results using coarse meshes. We perform interpolation on several mesh resolutions. Top: interpolation of the original model (17k triangles). Middle: the interpolation results of coarse meshes (4k, 2k and 1k triangles). Bottom: comparison of the dense results. The results are nearly identical for all the resolutions.

In Table

Timing comparisons of each step with and without the multiresolution method (in seconds).

Mesh size | No. Ex. | Interp. Rot Inv | Rotation optimization | Vertex optimization | Matrix ortho. | Coarse-to-Dense | |||||

Const. | Factor | Solve | Const. | Factor | Solve | ||||||

w/o multires. | 17k | 10 | 0.12 | 0.21 | 0.25 | 0.027 | [0.78] | [0.023] | 0.003 | 0.50 | — |

With multires. | 1k | 10 | 0.01 | 0.01 | 0.01 | 0.001 | [0.06] | [0.002] | 0.001 | 0.03 | 0.008 (17k), 0.075 (140k) |

From left to right: Mesh size, number of examples, interpolation of rotation-invariant encodings, rotation optimization (construction and factorization of

Using the multiresolution method, we can edit a mesh in an interactive rate. Our method requires approximately 0.1 sec to edit a 17k mesh and 0.2 sec to edit a 140k mesh. This is a significant speedup over the original MeshIK that requires approximately 1 sec to solve a

Our method has limitations that need to be overcome. Although the use of the multiresolution method helps achieving interactive rates, fine details remain static. It is, therefore, difficult for our method to model complex deformation, for example, facial expressions generated by muscle contractions. Our method is probably suitable for large- and medium-scale deformation, and it is rather unfit for modeling small scale surface-detail deformations. Also, because we assign one interpolation weight for each example, our method do not accept the user to place many handles, for example, more than 10 or 20 handles. We believe that our global-to-local editing strategy based on translational/rotational controls works well for interactive posing of the model without using many handles, which can create a new shape in a wide range of poses. However, in the case where simultaneous manipulations of a large number of handles are required, for example, deforming the model using motion-capture data points, our method would fail to approximate deformations. This problem could be alleviated by using the weighted pose space deformation framework [

Pose space surface manipulation is a novel example-based mesh editing technique that can deform the model naturally and that is able to adjust a pose locally. We provide both translational and rotational constraints to achieve direct manipulation of a surface to pose the model. Our method is reasonably efficient because, by inferring a pose from handles and by performing pose space interpolation, we are able to solve the problem with two linear systems, which avoids involved nonlinear optimization. With this two-step linear approach combined with the multiresolution deformation method, we achieve interactive rates without losing important deformation effects such as muscle bulging. The performance of our method would be further improved by a GPU implementation [

The authors would like to thank Brett Allen for the Arm and Shoulder datasets, Robert W. Sumner for the Elephant, Lion, Horse, and Cloth models, and Daniel Vlasic for the Performer model.