Instruction Guided Multi Object Image Editing with Quantity and Layout Consistency

Read full text
Authors: Jiaqi Tan, Fangyu Li, Yang Liu
Cited by: 0
References: 25

Abstract

Instruction driven image editing with standard CLIP text encoders often fails in complex scenes with many objects. We present QL-Adapter, a framework for multiple object editing that tackles two challenges: enforcing object counts and spatial layouts, and accommodating diverse categories. QL-Adapter consists of two core modules: the Image-Layout Fusion Module (ILFM) and the Cross-Modal Augmentation Module (CMAM). ILFM fuses layout priors with ViT patch tokens from the CLIP image encoder to strengthen spatial structure understanding. CMAM injects image features into the text branch to enrich textual embeddings and improve instruction following. We further build QL-Dataset, a benchmark that spans broad category, layout, and count variations, and define the task of quantity and layout consistent image editing (QL-Edit). Extensive experiments show that QL-Adapter achieves state of the art performance on QL-Edit and significantly outperforms existing models.

Fields of Study & Subjects

Advanced Image and Video Retrieval Techniques Advanced Vision and Imaging Medical Image Segmentation Techniques

Publication details

Journal / source
ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Publisher
IEEE
Pages
8912–8916
Publication date
21 April 2026
Language
EN