Introducing NeuCo-Bench: Benchmarking Compact Earth Observation Embeddings with Purpose 

We are pleased to announce the release of NeuCo-Bench, a lightweight, task-driven benchmarking framework for evaluating compact Earth Observation (EO) embeddings. Originally developed to support submissions for the EARTHVISION 2025 Challenge at CVPR, NeuCo-Bench is now available as a standalone toolkit for local evaluation and research use. 

What is NeuCo-Bench? 
NeuCo-Bench is a model-agnostic evaluation framework designed to test how well compressed, fixed-size EO embeddings preserve task-relevant semantics. In contrast to traditional pixel-level reconstruction metrics, NeuCo-Bench focuses on downstream utility—measuring what embeddings can do in real-world EO tasks. 

Key Features 

• Compact and Efficient: Evaluates fixed-size embeddings (e.g. 1024-dimensional vectors) suitable for resource-constrained pipelines. 
• Task-Driven Evaluation: Applies linear probes to practical EO tasks including land-cover classification, cloud detection, and biomass estimation. 
• Model-Agnostic Comparison: Supports embeddings from foundation models, compression techniques, or handcrafted baselines. 
• Robust Metrics: Includes signal-to-noise ratios and dynamic rank aggregation for comprehensive benchmarking. 

Why NeuCo-Bench?
As EO applications move toward real-time and edge processing, embedding efficiency matters. NeuCo-Bench helps researchers and practitioners systematically evaluate semantic retention under compression and standardize comparison across approaches. 

Useful link
Explore the repo and get started (with example scripts): github.com/embed2scale/NeuCo-Bench 

We welcome your feedback and contributions to accelerate EO embedding research together! 

CVPR Challenge: https://www.grss-ieee.org/events/earthvision-2025/?tab=challenge 
NeuCo-Bench Repo: https://github.com/embed2scale/NeuCo-Bench