Fine‐Tuning a Weather Foundation Model With Lightweight Decoders for Unseen Physical Processes

作者：Fanny Lehmann, Fırat Özdemir, Benedikt Soja, Torsten Hoefler, Siddhartha Mishra, Sebastian Schemm · 发表于：Journal of Geophysical Research Machine Learning and Computation · 年份：2026 · DOI：10.1029/2025jh000837 · 被引用次数：1 · 研究领域：Computer Graphics and Visualization Techniques

Abstract Recent advances in AI weather forecasting have led to the emergence of so‐called “foundation models”, typically defined by expensive pretraining and minimal fine‐tuning for downstream tasks. However, in the natural sciences, a desirable foundation model should also encode meaningful statistical relationships between the underlying physical variables. This study evaluates the performance of the state‐of‐the‐art Aurora foundation model in predicting hydrological variables, which were not included during pretraining. We introduce a lightweight approach using shallow decoders trained on the latent representations of the pretrained model to predict these new variables. As a baseline, we compare this to fine‐tuning the full model, which allows further optimization of the latent space while incorporating new variables into both inputs and outputs. The decoder‐based approach requires 43% less training time and 53% less memory, while achieving strong accuracy across various hydrological variables and preserving desirable properties of the foundation model, such as autoregressive stability. Notably, decoder accuracy depends on the physical correlation between the new variables and those used during pretraining, indicating that Aurora's latent space captures meaningful physical relationships. The decoders outperform UNet, climatology, and persistence baselines on variables related to pretraining, showing the benefits of the latent space. In this sense, we argue that an importan...