Restoration of archival film with large areas of structural damage

Abstract The existing models for end-to-end repair of archival films can only repair small structural-type damage, not large structural-type damage. In this study, a structural damage repair framework was designed by imitating the process of observing damage with the human eye. Structural-type damage in archival films is well known to be content-independent, and this work represents the first attempt to exploit the difference between structural-type damage and content in time correlation by modeling multi-scale features. Specifically, our framework consists of a structural damage detection model and a repair model. The detection model first generates channel attention maps for multi-channel coding features. It then performs matrix multiplication with the encoded features of each channel separately. The temporal correlation differences of different channels are modeled. In addition, the source reference attention module is used to supplement the spatial detail information. The repair model considers the sparsity and temporal uncorrelation of structural damage and implements damage repair by inputting a single reference frame and a damage mask image through multiscale fusion. This framework was compared with baselines using the Iskakov dataset. The proposed detection model with a strong baseline improves the mIoU by 47.43%, and the developed repair model with a strong baseline improves the PSNR by 28.52%. In terms of time cost control, our model restores 720P video sequences in an average time of only 0.415s, outperforming baselines. Qualitative experiments were also conducted to verify the effectiveness of the model in repairing large damage in different scenes. The experimental results show that modeling differences based on temporal correlation is a promising solution for large-area damage in archival films and that the effect of composite degradation of archival films on feature learning can be effectively mitigated by multi-scale fusion and attention mechanisms.