Panoramic Arbitrary Style Transfer with Deformable Distortion Constraints

作者：Wujian Ye, Yue Wang, Yijun Liu, Wenjie Lin, Xiang Xin · 发表于：Journal of Visual Communication and Image Representation · 年份：2024 · DOI：10.1016/j.jvcir.2024.104344 · 被引用次数：3 · 研究领域：Generative Adversarial Networks and Image Synthesis、Human Motion and Animation、Advanced Vision and Imaging

Neural style transfer is a prominent AI technique for creating captivating visual effects and enhancing user experiences . However, most current methods inadequately handle panoramic images, leading to a loss of original visual semantics and emotions due to insufficient structural feature consideration. To address this, a novel panorama arbitrary style transfer method named PAST-Renderer is proposed by integrating deformable convolutions and distortion constraints . The proposed method can dynamically adjust the position of the convolutional kernels according to the geometric structure of the input image, thereby better adapting to the spatial distortions and deformations in panoramic images. Deformable convolutions enable adaptive transformations on a two-dimensional plane, enhancing content and style feature extraction and fusion in panoramic images. Distortion constraints adjust content and style losses, ensuring semantic consistency in salience, edge, and depth of field with the original image. Experimental results show significant improvements, with the PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index Measure) of stylized panoramic images’ semantic maps increasing by approximately 2–4 dB and 0.1–0.3, respectively. Our method PAST-Renderer performs better in both artistic and realistic style transfer, preserving semantic integrity with natural colors, realistic edge details, and rich thematic content.