Face Adapter
for Pre-Trained Diffusion Models
with Fine-Grained ID and Attribute Control

1Zhejiang University, 2Tencent Youtu Lab, 3VIVO, 4Nanyang Technological University

Motivation

Face-Adapter aims to address the unsatisfactory performance of current SD adapters
in performing face reenactment/ swapping.

⭐Face Reenactment

Current SD adapters for face editing struggle to follow fine-grained target structure with text-based attribute control.

Interpolate start reference image.

⭐Face Swapping

Current SD adapters for face editing struggle to generate facial detail and handle face shape changes in face swapping.

Interpolate start reference image.

Adapter Design

            ⭐ Fully disentangled ID, target structure and attribute control enable 'one-model-two-tasks'.
            ⭐ Addressing overlooked issues.
            ⭐ Simple yet effective, plug and play.

More Comparison Results

            Beside SD Adapters, Face-Adapter sets itself apart from

            ✨ Diffusion-based reenactment/ swapping methods,
                  by effectively using the prior of SD and improving generation quality.
            ✨ GAN-based reenactment/ swapping methods,
                  by exceled in accommodating large pose and face shape variations
                  and demonstrates exceptional performance in background generation.

⭐Face Reenactment

Interpolate start reference image.

⭐Face Swapping


=== Comparison Under Large Face Shape Variations ===

Interpolate start reference image.


=== Comparison Under Large Pose Variations ===

Interpolate start reference image.


Check out our paper and mess around with our code!

Interpolate start reference image.