Exemplar-based image colorization with awareness of object co-saliency

Abstract While exemplar-based colorization colorizes a target image according to the chromatic information of a given reference image, we observed that existing methods were easily disturbed by colors of multiple objects in a reference image. To tackle the issue, we propose enhancing the exemplar-based colorization process according to the correlation among salient objects within the target and reference images. The proposed framework predicts the regions of common objects that occur in both the target and reference images. It first focuses on colorizing the co-salient objects and the remaining region, respectively. Based on the above preliminary features, our region-aware module with attention mechanism, which can moderately tolerate imperfect predicted regions, then progressively colorizes the whole image. Experiments show that the proposed framework can substantially alleviate the color disturbance from unrelated objects and generate more precise colors, especially for salient common objects. An extended metric is presented to complement the limitation of existing metrics for exemplar-based colorization. Moreover, while we extend our framework to colorize only specific salient regions of an image, it can become an intelligent editing tool for exemplar-based colorization.