projects people publications datasets

Shape Modeling and Analysis

Early work on deformable models involved illumination constraints for shape estimation. A recent result in 3D shape reconstruction based on images that we worked on was a new PDE-based methodology for deformable surfaces that is capable of automatically evolving its shape to capture the geometric boundary of the data and simultaneously discover its underlying topological structure.

Recent technology advances have enabled the capture of high quality dense 3-D data samples undergoing motions at video speeds. Even though such data cannot be captured as easily as 2D digital images they open new ways to model and analyze deformable motion especially in the case of human motion. Such improved motion understanding (together with similarly improved appearance models based on extensive 3D data) will undoubtedly be useful even for improved 2D image analysis. However in order to build such databases, two related problems must be solved first. Matching between two different data sets in order to be able to combine and in the case of motion, accurate registration between different frames. We have been introduced a novel framework, based on conformal geometry, that allows to map both 3D geometric features and appearance to 2D representations on which well-known 2D techniques can be used. For the case of accurate intraframe 3D registration we have already developed a method based on harmonic mapping, which we have successfully applied to facial expressions. More details are given in the Expression Modeling section.

  • Former students:
    • Liu Yang

    • Shape Reconstruction from 3D and 2D Data Using PDE-Based Deformable Surfaces

    [publications & media]

    We propose a new PDE-based methodology for deformable surfaces that is capable of automatically evolving its shape to capture the geometric boundary of the data and simultaneously discover its underlying topological structure. Our model can handle multiple types of data (such as volumetric data, 3D point clouds and 2D image data), using a common mathematical framework. The deformation behavior of the model is governed by partial differential equations (e.g. the weighted minimal surface flow). Unlike the level-set approach, our model always has an explicit representation of geometry and topology. The regularity of the model and the stability of the numerical integration process are ensured by a powerful Laplacian tangential smoothing operator. By allowing local adaptive refinement of the mesh, the model can accurately represent sharp features. We have applied our model for shape reconstruction from volumetric data, unorganized 3D point clouds and multiple view images. The versatility and robustness of our model allow its application to the challenging problem of multiple view reconstruction. Our approach is unique in its combination of simultaneous use of a high number of arbitrary camera views with an explicit mesh that is intuitive and easy-to-interact-with. Our model-based approach automatically selects the best views for reconstruction, allows for visibility checking and progressive refinement of the model as more images become available. The results of our extensive experiments on synthetic and real data demonstrate robustness, high reconstruction accuracy and visual quality.

    Publications
    Media