A Theory of Stabilization by Skull Carving

This presentation was given at SIGGRAPH Asia 2024, which was held 3–6 December, 2024 in Tokyo, Japan. Authors: Mathieu Lamarre, Patrick Anderson, and Étienne Danvoye.

Watch the presentation above and download the paper below.

A Theory of Stabilization by Skull Carving

Accurately stabilizing facial motion is essential for creating photo-real avatars for 3D games, virtual reality, movies, and machine-learning training data. In that last case, facial stabilization needs to work quickly and automatically when the source data is a population of people with varying morphology. It is critical to be able to distinguish rigid skull motions from facial expressions, since misalignment between skull motion and facial expressions can lead to an animation model that is hard to control and unusable for natural-looking motion.

Existing stabilization methods struggle to work with sparse sets of very different expressions, such as when combining multiple units from a facial action coding system (FACS). Other methods are not sufficiently robust, depend on motion data to find stable points, or make one-for-all invalid physiological assumptions.

In this paper, we leverage recent advances in neural signed distance fields and differentiable isosurface meshing to compute skull stabilization rigid transforms directly on unstructured triangle meshes or point clouds. This significantly enhances accuracy and robustness.

We introduce the concept of a “stable hull” as the surface of the boolean intersection of stabilized scans, analogous to the “visual hull” (in shape-from-silhouette) and the “photo hull” (in space carving). This stable hull resembles a skull overlaid with minimal soft tissue thickness, and upper teeth are automatically included. Our skull carving algorithm simultaneously optimizes the stable hull shape and rigid transforms to get accurate stabilization of complex expressions for a diverse set of people, outperforming existing methods.

Download the paper (PDF 17.8 MB)