Fastino Labs has unveiled GLiNER2.5-Decide, an innovative 340-million parameter open-weight decision model. This specialized AI is designed to process natural language alongside structured queries, delivering precise, categorized responses that include probability distributions, confidence levels, and feasibility checks. Its primary application lies in streamlining critical judgment tasks within AI agent workflows, such as intelligent routing, incident triage, sophisticated tool selection, and robust guardrail implementation.
Distributed under the permissive Apache 2.0 license, GLiNER2.5-Decide is readily deployable, with installation facilitated via pip install gliner2. The model exhibits impressive versatility, capable of operating across CPUs, GPUs, and even isolated, air-gapped environments. For organizations seeking managed services, Fastino also provides hosted inference and fine-tuning capabilities through its dedicated GLiNER API.
How GLiNER2.5-Decide Functions
Distinguishing itself from generative AI, GLiNER2.5-Decide operates as a non-generative classifier. It leverages a robust DeBERTa-v3-large encoder, specifically fine-tuned from the gliner2-large-v1 base. Crucially, it avoids producing any free-form text tokens, thereby eliminating the need for complex prompt engineering.
Instead, label sets are dynamically provided during inference calls. Each query within the defined schema explicitly outlines its acceptable responses, indicating whether a single answer, multiple answers, or an ordered value is expected. These schemas can be enriched with detailed instructions, illustrative examples, label descriptions, and sophisticated rules governing inter-question relationships.
The underlying processing involves a two-stage pipeline. Initially, the encoder jointly processes the input text and schema, assigning scores to all permissible answers. Subsequently, a constrained decoder intelligently identifies the highest-scoring combined answer set, strictly adhering to the pre-defined rules and constraints. It is important to note the model's focused scope: it excels as a specialist in operational decision-making, deliberately not performing general reasoning, providing explanations, or tackling open-ended questions.
The Advantage of Joint Decoding
A standout feature is its joint decoding mechanism, vital for ensuring consistent and reliable outputs. Fastino highlights its significance through a guardrail scenario: when processed independently, a prompt might be flagged for injection with a high probability (e.g., 0.82) while simultaneously being deemed safe (e.g., 0.52). This inherent conflict in independent outputs can hinder effective automated responses.
Joint decoding resolves such contradictions by enforcing logical rules, such as "any detected harm necessitates an unsafe verdict." Consequently, the model delivers a unified assessment, for instance, reporting "safety=unsafe" alongside "harm_type=prompt_injection." This cohesive output empowers downstream systems to confidently block, redirect, or escalate actions. The robust schema definitions further allow for the expression of complex implications, exclusions, limits on cardinality, and ordered scales.
Benchmark Performance and CPU Efficiency
Fastino conducted extensive evaluations on its internally developed Fast Decisions suite, comprising 5,100 test examples across 17 diverse datasets covering customer operations, domain routing, and content understanding. GLiNER2.5-Decide demonstrated strong performance, achieving an average exact-match accuracy of 60.1% and outperforming competitors on 9 of the 17 datasets. It particularly excelled in intent routing, scoring 75.3% for support intent and 64.3% for banking intent, significantly surpassing alternative models.
Latency benchmarks underscore the model's practical viability, especially on CPU. For short inputs (64 tokens) with a 2-head, 15-label schema, the median latency (p50) on a 48-vCPU Intel Xeon Platinum 8581C was 167.3 ms. While GPUs naturally offered faster processing (e.g., 38.3 ms on an NVIDIA V100), the CPU performance makes it highly accessible for many deployment scenarios. For longer inputs (1,024 tokens), an NVIDIA A100 achieved 52.6 ms, illustrating scalable efficiency across hardware types.
Versatile Applications and the GLiNER Family
GLiNER2.5-Decide is positioned for a wide array of practical applications, including dynamic model routing, sophisticated tool calling, intelligent browser and computer automation, robust guardrail implementation, contextual pruning, acting as an "LLM-as-a-judge" for evaluation tasks, and powering complex simulations. Fine-tuning capabilities are available locally, supporting full fine-tuning or LoRA-based adaptations via the GLiNER2 trainer.
Expanding its ecosystem, Fastino has also introduced GLiNER2.5-Decide-1B, built upon the Ettin 1B encoder, which achieves a competitive 59.6% on the same benchmark suite. For global deployments requiring multilingual processing, the company offers GLiNER2.5-multi-Decide, a 287M model scoring 56.7%. The foundational architectural principles are comprehensively detailed in the GLiNER2 research paper.
This article is a rewritten summary based on publicly available reporting. For the original story, visit the source.
Source: MarkTechPost
