NVIDIA C-RADIOv4: Unifying Vision AI for Scalable, Multi-Task Performance

Revolutionizing Computer Vision with a Unified Backbone

The field of artificial intelligence often necessitates specialized models for distinct computer vision tasks, leading to complex deployments. NVIDIA addresses this challenge with the introduction of C-RADIOv4, an innovative agglomerative vision backbone designed to streamline AI perception workloads. This new architecture effectively distills the strengths of SigLIP2-g-384, DINOv3-7B, and SAM3 into a singular student encoder, providing a versatile solution for classification, retrieval, dense prediction, and segmentation tasks at scale.

The Power of Agglomerative Distillation

C-RADIOv4 builds upon the foundation of earlier RADIO models, utilizing agglomerative distillation to train a single Vision Transformer (ViT)-style student. This student network learns to emulate both the dense feature maps and summary tokens produced by multiple heterogeneous teacher models. While previous iterations combined models like DFN CLIP, DINOv2, and SAM, C-RADIOv4 significantly upgrades its teacher ensemble, incorporating:

• SigLIP2-g-384: For superior image-text alignment.
• DINOv3-7B: To generate high-quality self-supervised dense features.
• SAM3: Providing segmentation-centric features and ensuring compatibility with the SAM3 decoder.

This strategic selection allows the student encoder to concurrently support various vision tasks by matching DINOv3 and SAM3 for dense features, and SigLIP2 and DINOv3 for summary tokens.

Advanced Training for Enhanced Robustness

Several key innovations underpin C-RADIOv4's robust performance:

Stochastic Multi-Resolution Training

Unlike models trained on a fixed set of input sizes, C-RADIOv4 employs stochastic multi-resolution training. It samples input sizes across a broad spectrum, from low resolutions (e.g., 128-432 pixels) to high resolutions (e.g., 512-1152 pixels). This approach, combined with FeatSharp upsampling for SigLIP2 features, ensures consistent performance across varying input scales, closely mirroring DINOv3-7B's scaling trends with significantly fewer parameters.

Noise Suppression through Shift Equivariance

Large vision models can introduce artifacts into distilled students. C-RADIOv4 mitigates this by integrating two shift-equivariant mechanisms: a shift-equivariant dense loss and shift-equivariant MESA regularization. These techniques ensure that the student learns input-dependent structures rather than memorizing positional noise from teachers, by presenting independently shifted crops of images and aligning features before loss calculation. Additionally, DAMP (Differentiated Adaptive Multi-Scale Patching) injects multiplicative noise, further boosting robustness.

Balanced Multi-Teacher Distillation

To prevent certain teachers from dominating the optimization process, C-RADIOv4 introduces an angular dispersion-aware summary loss. This mechanism normalizes the squared angle between student and teacher embeddings by the teacher's angular dispersion. This equalization ensures that both vision-language semantics from SigLIP2 and dense representation quality from DINOv3 receive balanced influence during training.

Performance and Deployment Advantages

C-RADIOv4 demonstrates impressive performance across a range of benchmarks. On ImageNet-1k zero-shot classification, it achieves approximately 83.09% top-1 accuracy. For k-NN classification, it improves upon prior RADIO versions and maintains stable or improving performance at higher resolutions where DINOv3 may degrade. Key dense prediction metrics include 55.20 mIoU on ADE20k and 87.24 mIoU on PASCAL VOC, proving competitive with DINOv3-7B.

Designed for practical application, C-RADIOv4 serves as a direct drop-in replacement for the Perception Encoder backbone within SAM3. Its integration preserves segmentation behavior and, in some instances, resolves failure cases observed with the original encoder. For efficient deployment, C-RADIOv4 offers a ViTDet-mode configuration, utilizing windowed attention for faster inference on high-resolution dense tasks, making it a viable solution where global attention at all layers proves too computationally expensive. The model is released under the NVIDIA Open Model License, facilitating its adoption and further development.

With C-RADIOv4, NVIDIA pushes the boundaries of unified vision AI, offering a scalable, robust, and versatile backbone for the next generation of intelligent systems.