A continual learning-guided training framework for pansharpening

Supervised learning-based methods for pansharpening were criticized since their appearance because they rely on scale-shift assumption, i.e. those methods usually perform much better at reduced resolution than at full resolution. To address this issue, here we propose a general training framework for supervised learning-based pansharpening. Our training process consists of two stages: the first one is a conventional supervised method, which is applied to the reduced resolution dataset to obtain the converged model, while in the second model, obtained from stage one, is trained through an unsupervised learning scheme. Moreover, we developed a novel loss function made up of two terms that guarantees model high performance, both at reduced and full resolution. To the best of our knowledge, this is the first attempt to introduce the continual learning concept into pansharpening. The proposed framework is general and can be applied to any supervised learning-based pansharpening network. Also, the proposed method shows robustness with respect to the changing of the satellite sensor used to provide the data to be fused. Extensive tests on images from QuickBird, GaoFen-2, WorldView-3, and WorldView-2 show the effectiveness of the proposed methodology.