Webinar: “Embedding Space Metrics Constrained Fine-Tuning”

While fine-tuning Geospatial Foundation Models (GFMs) has become the standard for domain adaptation, it often leads to catastrophic forgetting or ‘representation drift,’ where the model’s fundamental understanding of semantic relationships is warped. This webinar introduces Embedding Space Metrics Constrained Fine-Tuning (ESM-CFT), an optimisation method in the TerraTorch framework designed to preserve the structural integrity of the latent space during supervised fine-tuning.

Unlike traditional methods that rely solely on cross-entropy loss, ESM-CFT incorporates a geometric constraint based on distance metrics to ensure that the relative positioning of representations in the embedding space remains stable.

The webinar presents promising preliminary experimental results and opens the stage for discussion of future work.