Encrypted Masked Modeling and Gated Fusion for privacy-preserving content-based image retrieval

This paper presents Encrypted Masked Modeling and Gated Fusion (EMF), a privacy-preserving image retrieval framework for JPEG-compliant encrypted images. EMF supports cloud retrieval without plaintext access by extracting syntax-invariant cues from encrypted entropy streams, including variable-length integer categories, block zero counts, run-length profiles, and Huffman symbol statistics. VLI and TZCPB cues are tokenized in parallel and combined through block-level convex gating. DC and AC histogram tokens add global entropy context. Self-supervised pretraining couples Encrypted Masked Modeling with momentum contrastive learning, allowing context-aware prediction of encrypted-domain targets without plaintext reconstruction. For scalable use, EMF adds cancelable indexing through keyed orthonormal rotation, product quantization, and inverted file indexing, with optional Merkle-based verification. Experiments on Corel10K and ImageNet-100 show higher mAP, Precision@100, and Recall@100 than prior encrypted-domain baselines. These results show that fused syntax-invariant cues and cipher-tailored pretraining improve retrieval under encrypted-domain constraints across tested retrieval benchmark settings.