Why evaluate AI weather model performance for Antarctic winds?
Antarctic near-surface winds are critical for the climate: they drive sea ice formation, boundary layer stability, precipitation reaching the ice sheet, and ice shelf dynamics [1]. Katabatic winds are channelled through narrow coastal valleys at scales of 1–10 km. At these scales, winds may be invisible to global weather models, if they represent the atmosphere on a grid of 0.25° cells (such as GraphCast forecast and ERA5 reanalysis). Many AI weather models, such as GraphCast, are not trained on real observations. They learn on ERA5 – a global reconstruction of historical weather which is also a model. Then AI forecast model is verified against ERA5. If ERA5 is wrong, the AI model may look correct anyway. GraphCast AI weather forecast model has been shown to outperform classic numerical weather models on global average metrics [2]. However, existing verification benchmarks are dominated by mid-latitude data and may not reflect AI model performance in polar regions. Studies show ERA5 underestimates wind speed at Antarctic coastal stations [3]. No published study has yet tested on Antarctic in-situ wind observations whether AI models make the same errors.
Error against stations vs against reanalysis
Bias and RMSE at each station computed three ways — GraphCast vs ERA5 vs in-situ observations.
25 stations
Locations of the low-elevation Antarctic stations used as ground truth, on a polar projection.