Infrared and visible image fusion with convolutional neural networks

作者：Yü Liu, Xun Chen, Juan Cheng, Hu Peng, Zengfu Wang · 发表于：International Journal of Wavelets Multiresolution and Information Processing · 年份：2017 · DOI：10.1142/s0219691318500182 · 被引用次数：481 · 研究领域：Advanced Image Fusion Techniques、Infrared Target Detection Methodologies、Remote-Sensing Image Classification

The fusion of infrared and visible images of the same scene aims to generate a composite image which can provide a more comprehensive description of the scene. In this paper, we propose an infrared and visible image fusion method based on convolutional neural networks (CNNs). In particular, a siamese convolutional network is applied to obtain a weight map which integrates the pixel activity information from two source images. This CNN-based approach can deal with two vital issues in image fusion as a whole, namely, activity level measurement and weight assignment. Considering the different imaging modalities of infrared and visible images, the merging procedure is conducted in a multi-scale manner via image pyramids and a local similarity-based strategy is adopted to adaptively adjust the fusion mode for the decomposed coefficients. Experimental results demonstrate that the proposed method can achieve state-of-the-art results in terms of both visual quality and objective assessment.