From Photo to Expression: Generating Photorealistic Facial Rigs

This presentation was part of the Machine Learning Summit at the Game Developers Conference on Mar 18–22, 2024.

From Photo to Expression: Generating Photorealistic Facial Rigs

In this presentation, SEED’s Hau Nghiep Phan discusses how machine learning can improve facial animation in the gaming industry.

Watch the full presentation at the GDC Vault.

The presentation showcases EA’s in-house facial rig called FaceRig, and demonstrates how blendshapes can be automatically created to suit various character identities. 

The presentation covers: FaceMixer, which can generate blendshapes from a neutral input and FaceOptim, which automates the creation of blendshapes and animations from scanned expressions or 4D animation. 

Nhgiep also introduces FaceBot, a solution that generates photorealistic face textures and meshes with expressions that can then be fed back into the previous system. By combining FaceBot with FaceOptim or FaceMixer, users can generate high-quality rigged faces for game characters with minimal effort. 

Finally, the presentation includes a photo-to-mesh project from SEED’s Hector Leon, which converts single 2D photos into 3D meshes and textures.

Future developments include incorporating FaceAnim for animation, allowing for more organic and realistic facial animation.