Underdog Saluki 27B: Compact AI Agent Model Excels in Tool Calling, Outperforming Larger Original

Conway Research Introduces Saluki 27B: A Lean Agent AI Prioritizing Tool Calling

Conway Research, creators of the Underdog on-device assistant, has publicly released Saluki 27B under an Apache 2.0 license. This model represents a significant advancement, as it is a 2-bit GGUF quantization of the Qwen3.8-27B, fitting into a mere 7.89 GB file. In stark contrast, the full BF16 version of Qwen3.8-27B demands 54 GB of space. Conway Research meticulously fine-tuned this compression specifically to safeguard and enhance tool-calling, a critical function that transforms a conversational AI into a sophisticated agent. For development teams, this means access to a powerful 27B-class agent model that operates efficiently within standard llama.cpp frameworks.

Performance Highlights: Unexpected Strengths and Calculated Compromises

Saluki 27B demonstrates a compelling performance profile, retaining an average of 96% of the full Qwen3.8-27B’s capabilities across nine diverse benchmarks. Its most impressive feat is in parallel tool calls, where it achieved 42 successful executions compared to 35 for the uncompressed model, marking a 120% improvement in this specific area. However, the compression strategy led to some anticipated trade-offs. The model shows reduced proficiency in highly demanding cognitive tasks, such as competitive mathematics and multi-step reasoning, with scores dropping approximately 12 to 18 points in these categories. For example, on the AIME 2025 benchmark, Saluki scored 79.2 against the original’s 96.7.

In summary, Saluki 27B remarkably outcompetes its 54 GB predecessor in tool invocation efficiency while residing in a file less than 8 GB. Conversely, tasks requiring intricate mathematical calculations and complex reasoning exhibit a moderate performance dip.

The Multi-Layered Engineering Behind Saluki 27B

The development of Saluki 27B involved a sophisticated, three-tiered optimization process, creating its 2-bit, mixed-precision GGUF architecture designed for local agent deployment:

• The foundation is the Qwen team’s Qwen3.8-27B, a substantial 27B parameter model. It integrates advanced attention mechanisms and natively supports a vast context of 262,144 tokens.
• The second stage involved ISTA-DASLab’s Qwen3.8-27B-GSQ-RCO-GGUF. This technique employs GSQ for precise low-bit scalar grid learning per tensor, and RCO to optimally assign quantization types given a fixed size budget. ISTA’s IQ2_XS, for instance, uses 2.50 bits per weight, resulting in an 8.4 GB file.
• Conway Research implemented its own final optimization pass. This proprietary step further reduced the model's footprint to 7.89 GB, specifically targeting and boosting tool-calling performance. The resulting IQ2-mix file incorporates an imatrix tag, though the full details of this specific recipe remain proprietary.

Comparative Performance Metrics

Conway Research conducted evaluations using both internal comparative assessments and comparisons against publicly available scores:

Key Benchmark Results:

• Underdog Bench: Saluki scored 88, outperforming the full model (84) and PrismML’s Bonsai 2 (70) across 120 BFCL v4 tasks.
• Parallel Tool Calls: Saluki achieved 42 successful parallel tool calls from 100 tasks, exceeding the full model’s 35.
• AIME 2025: Saluki recorded 79.2 (average of 4), while the full Qwen3.8-27B attained 96.7.

Saluki 27B in the Compact LLM Landscape

Positioning Saluki 27B against other compact Qwen3.8-27B builds highlights its unique advantages:

• While PrismML Bonsai 2 27B offers a smaller file size at 5.95 GB, Saluki’s 7.89 GB package runs on standard llama.cpp, enabling full GPU offload without requiring specialized forks. Bonsai 2, conversely, necessitates PrismML’s customized llama.cpp due to its unique data formats.
• Saluki 27B includes an optional vision add-on, available in 629 MB or 928 MB sizes, expanding its multimodal capabilities.
• Both Saluki and its compact competitors, like ISTA GSQ-RCO IQ2_XS (8.4 GB), are released under the Apache 2.0 license, promoting open development.

Conclusion: A New Paradigm for On-Device AI Agents

The release of Underdog Saluki 27B marks a notable achievement in the realm of efficient AI. It successfully condenses the Qwen3.8-27B into a highly performant 7.89 GB model, proving that significant compression does not necessarily compromise crucial capabilities. Its standout performance in tool calling, combined with compatibility with stock llama.cpp, positions Saluki 27B as a compelling option for developers building sophisticated, resource-efficient AI agents.