Learning Representations of Satellite Images From Metadata Supervision - Réseau télédétection INRAE
Communication Dans Un Congrès Année : 2024

Learning Representations of Satellite Images From Metadata Supervision

Résumé

Self-supervised learning is increasingly applied to Earth observation problems that leverage satellite and other remotely sensed data. Within satellite imagery, metadata such as time and location often hold significant semantic information that improves scene understanding. In this paper, we introduce Satellite Metadata-Image Pretraining (SatMIP), a new approach for harnessing metadata in the pretraining phase through a flexible and unified multimodal learning objective. SatMIP represents metadata as textual captions and aligns images with metadata in a shared embedding space by solving a metadata-image contrastive task. Our model learns a non-trivial image representation that can effectively handle recognition tasks. We further enhance this model by combining image self-supervision and metadata supervision, introducing SatMIPS. As a result,SatMIPS improves over its image-image pretraining baseline, SimCLR, and accelerates convergence. Comparison against four recent contrastive and masked autoencoding-based methods for remote sensing also highlight the efficacy of our approach. Furthermore, our framework enables multimodal classification with metadata to improve the performance of visual features, and yields more robust hierarchical pretraining. Code and pretrained models will be made available at: https://github.com/preligens-lab/satmip.
Fichier principal
Vignette du fichier
base-main-supp.pdf (3.36 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
licence

Dates et versions

hal-04709749 , version 1 (25-09-2024)

Licence

Identifiants

  • HAL Id : hal-04709749 , version 1

Citer

Jules Bourcier, Gohar Dashyan, Karteek Alahari, Jocelyn Chanussot. Learning Representations of Satellite Images From Metadata Supervision. ECCV 2024 - 18th European Conference on Computer Vision, Sep 2024, Milano, Italy. pp.1-30. ⟨hal-04709749⟩
0 Consultations
0 Téléchargements

Partager

More