Cohere's Tiny Aya: A 70-Language SLM Redefines On-Device AI

Cohere's Tiny Aya: A 70-Language SLM Redefines On-Device AI

Cohere AI Labs has unveiled Tiny Aya, a groundbreaking family of small language models (SLMs) poised to redefine multilingual artificial intelligence. Challenging the notion that larger models inherently mean superior performance, Tiny Aya delivers exceptional translation and text generation across 70 distinct languages with a compact 3.35 billion parameters. Notably, it is optimized for efficient execution directly on smartphones and other edge devices.

Diverse Models and Core Architecture

The Tiny Aya collection features five distinct models: Tiny Aya Base (pretrained), Tiny Aya Global (balanced instruction-tuned), and three specialized regional variants—Earth (Africa/West Asia), Fire (South Asia), and Water (Asia-Pacific/Europe).

At its core, Tiny Aya employs a dense, decoder-only Transformer architecture. It integrates 3.35 billion total parameters (2.8 billion non-embedding) across 36 layers, utilizing a 262,000-token tokenizer designed for balanced language representation. Key architectural features include interleaved sliding window and full attention, Grouped Query Attention (GQA), and an 8192-token context window. The model was pretrained on 6 trillion tokens using a Warmup-Stable-Decay schedule, with SwiGLU activations and bias removal ensuring stability.

Innovative Training: FUSION and Regional Merging

Cohere utilized sophisticated post-training techniques, particularly for low-resource languages. Fusion-of-N (FUSION) is a synthetic data generation pipeline where prompts are submitted to a "team of teachers" (advanced language models). A judge LLM, the Fusor, intelligently extracts and combines the most robust elements from their responses, creating high-quality training signals.

For regional refinement, models underwent finetuning on five distinct clusters. To prevent "catastrophic forgetting" of global safety principles, Cohere implemented SimMerge, merging regional checkpoints with the global model using similarity signals to ensure both specialized performance and overall integrity.

Setting New Multilingual Performance Standards

Tiny Aya Global has consistently demonstrated superior performance against competitors across a range of multilingual tasks:

• Translation Excellence: On WMT24++, it surpassed GEMMA3-4B in translation quality for 46 of 61 languages.
• Enhanced Reasoning: In the GlobalMGSM mathematical reasoning benchmark for African languages, Tiny Aya achieved 39.2% accuracy, significantly outperforming GEMMA3-4B (17.6%) and QWEN3-4B (6.25%).
• Robust Safety: The model recorded the highest mean safe response rate on MultiJail, at 91.1%.
• Language Integrity: Maintaining 94% language accuracy, Tiny Aya effectively avoids defaulting to English when instructed to respond in another language.

On-Device Deployment: AI in Your Pocket

Designed for edge computing, Tiny Aya is highly optimized for mobile deployment. Utilizing 4-bit quantization (Q4_K_M), the model requires only a modest 2.14 GB memory footprint. This enables it to run at approximately 10 tokens per second on an iPhone 13 and 32 tokens per second on an iPhone 17 Pro. Crucially, this quantization results in only a minimal 1.4-point reduction in generation quality, positioning Tiny Aya as a powerful solution for offline, private, and localized AI applications directly on personal devices.

Impact on the AI Landscape

Tiny Aya challenges conventional wisdom regarding model scale, proving that thoughtful architecture and innovative data curation can yield state-of-the-art multilingual performance in a compact package. Its combination of diverse language capabilities and efficient on-device execution marks a significant step forward for ubiquitous, accessible artificial intelligence, fostering new possibilities for global communication and localized applications.