Liquid AI has recently unveiled LFM2.5-VL-3B-DSpark, an innovative experimental draft model designed to accelerate its existing LFM2.5-VL-3B vision-language model (VLM). Leveraging speculative decoding, this addition significantly enhances inference speed. The company reports impressive performance improvements, reaching up to 3.13 times faster decoding on Apple silicon and 2.66 times faster on an NVIDIA H100 GPU, all while maintaining the integrity of the base model's output.
The DSpark model weights are immediately accessible on Hugging Face in Safetensors and GGUF formats, with day-one integration across SGLang, MLX-VLM, and llama.cpp. While categorized as experimental, the release operates under the LFM Open License v1.0, permitting free commercial use for companies with annual revenues below $10 million.
How Speculative Decoding Boosts VLMs
Traditional language models process one token per forward pass. Speculative decoding introduces a smaller, auxiliary "drafter" model that anticipates and proposes a sequence of several upcoming tokens. The primary target model then efficiently validates this entire proposed block in a single pass, accelerating output generation.
DSpark's implementation in VLMs draws inspiration from Liquid AI's text-focused DSpark drafters. The drafter reads the target model's hidden states and forecasts subsequent tokens. Crucially, the system is modality-agnostic; by the hidden layers, visual and textual inputs are unified as numerical tensors, allowing the same inference algorithm across both.
Drafter Architecture and Performance
The DSpark drafter features a streamlined, attention-only architecture with four layers. It adds approximately 279.5 million parameters to the base model, an 8.9% increase. Training involved supervised fine-tuning over ten epochs on common VLM tasks, conducted on AMD hardware.
Performance evaluations utilized the MMSpec benchmark across six diverse VLM task categories. Key findings for decoding speedup and end-to-end speedup include:
- MLX-VLM (M5 Max MacBook Pro): Up to 3.13x faster decoding, with end-to-end speedups reaching 2.62x.
- llama.cpp (M3 Ultra): Up to 2.14x faster decoding, and end-to-end gains up to 1.77x.
- SGLang (1x H100 80GB): Up to 2.66x faster decoding, with end-to-end speedups reaching 2.27x.
The highest decoding and end-to-end acceleration figures often originate from different tasks. Output quality remains identical under greedy decoding, and the original distribution is preserved during sampling with non-zero temperatures.
Edge Device Performance and Integration
DSpark's acceleration is specific to the decoding phase. Initial image encoding and the "prefill" stage operate at standard speeds. On edge devices, where prefill constitutes a larger proportion of total latency, overall end-to-end speedups can be less dramatic. This phenomenon, explained by Amdahl's law, means a 2.69x faster decoding on TextVQA on an M5 Max might translate to only a 1.56x end-to-end speedup.
For integration, SGLang requires v0.5.19+ with specific algorithm and path arguments. MLX-VLM v0.7.2+ supports the drafter via a dedicated flag, currently limited to greedy sampling. llama.cpp users should pair the GGUF drafter with the LFM2.5-VL-3B-GGUF target. Public integration progress is available in respective pull requests. This release supports 16-bit weights exclusively, with no current support for quantized models.
Key Takeaways
- A 279.5M parameter drafter increases LFM2.5-VL-3B's total parameters by 8.9%.
- Decoding speeds are significantly improved: up to 3.13x faster on M5 Max, 2.66x on H100.
- Output quality is preserved under greedy decoding and matching sampling.
- Image encoding and prefill stages limit overall end-to-end gains, especially on edge hardware.
- The current release supports 16-bit weights; quantized acceleration is not yet available.
This article is a rewritten summary based on publicly available reporting. For the original story, visit the source.
Source: MarkTechPost
