Unifying Multi-Modal Hair Editing via Proxy Feature Blending

View at publisher
Authors: Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Can Wang, Weiming Zhang, Gang Hua, Nenghai Yu
Cited by: 1
References: 60

Abstract

Hair editing is a long-standing problem in computer vision that demands both fine-grained local control and intuitive user interactions across diverse modalities. Despite the remarkable progress of GANs and diffusion models, existing methods still lack a unified framework that simultaneously supports arbitrary interaction modes (e.g., text, sketch, mask, and reference image) while ensuring precise editing and faithful preservation of irrelevant attributes. In this work, we introduce a novel paradigm that reformulates hair editing as proxy-based hair transfer. Specifically, we leverage the dense and semantically disentangled latent space of StyleGAN for precise manipulation and exploit its feature space for disentangled attribute preservation, thereby decoupling the objectives of editing and preservation. Our framework unifies different modalities by converting editing conditions into distinct transfer proxies, whose features are seamlessly blended to achieve global or local edits. Beyond 2D, we extend our paradigm to 3D-aware settings by incorporating EG3D and PanoHead, where we propose a multi-view boosted hair feature localization strategy together with 3D-tailored proxy generation methods that exploit the inherent properties of 3D-aware generative models. Extensive experiments demonstrate that our method consistently outperforms prior approaches in editing effects, attribute preservation, visual naturalness, and multi-view consistency, while offering unprecedented support for multimodal and mixed-modal interactions.

Fields of Study & Subjects

Generative Adversarial Networks and Image Synthesis Computer Graphics and Visualization Techniques 3D Shape Modeling and Analysis

Publication details

Journal / source
IEEE Transactions on Pattern Analysis and Machine Intelligence
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Volume
48
Issue
6
Pages
6407–6424
Publication date
21 January 2026
Language
EN