Automatic Non-Rigid Registration of 3D Dynamic Data for Facial Expression Synthesis and Transfer
CVPR 2008
Sen Wang, Xianfeng Gu and Hong Qin
Automatic non-rigid registration of 3D time-varying
data is fundamental in many vision and graphics applications
such as facial expression analysis, synthesis, and
recognition. Despite many research advances in recent
years, it still remains to be technically challenging, especially
for 3D dynamic, densely-sampled facial data with a
large number of degrees of freedom (necessarily used to
represent rich and subtle facial expressions). In this paper,
we present a new method for automatic non-rigid registration
of 3D dynamic facial data using least-squares conformal
maps, and based on this registration method, we
also develop a new framework of facial expression synthesis
and transfer. Nowadays more and more 3D dynamic,
densely-sampled data become prevalent with the advancement
of novel 3D scanning techniques. To analyze and utilize
such huge 3D data, an efficient non-rigid registration
algorithm is needed to establish one-to-one inter-frame correspondences.
Towards this goal, a non-rigid registration
algorithm of 3D dynamic facial data is developed by using
least-squares conformal maps with additional feature
correspondences detected by employing active appearance
models (AAM). The proposed method with additional, interior
feature constraints guarantees that the non-rigid data
will be accurately registered. The least-squares conformal
maps between two 3D surfaces are globally optimized with
the least angle distortion and the resulting 2D maps are stable
and one-to-one. Furthermore, by using this non-rigid
registration method, we develop a new system of facial expression
synthesis and transfer. Finally, we perform a series
of experiments to evaluate our non-rigid registration
method and demonstrate its efficacy and efficiency in the
applications of facial expression synthesis and transfer.