Perplexity AI Unleashes pplx-embed-v2-late: Dual Multimodal Models for Edge and High-Performance Retrieval

Introducing Perplexity AI's New Multimodal Embeddings

Perplexity AI recently announced the release of pplx-embed-v2-late, a novel suite of multimodal embedding models built on a ColBERT-style architecture. This release includes two distinct versions: a compact 0.6 billion parameter model optimized for rapid, cost-effective queries, and a larger 9 billion parameter variant engineered for maximum quality and precision.

Both models are designed to process and retrieve diverse data types, including plain text, images, and rendered PDF pages, all within a unified embedding space. This shared space allows for seamless interaction across different modalities. For deployment, the models are currently available on Hugging Face under an MIT license, permitting self-hosting. Perplexity AI has also indicated plans for a hosted API endpoint in the future.

Key Strengths of pplx-embed-v2-late

The new models introduce several notable advantages:

• Efficiency and Performance: The 0.6B model, utilizing approximately 340 million active parameters for image processing, demonstrates performance comparable to some 8B parameter alternatives.
• Flexible Querying: Users can query a 9B model-generated index using the lighter 0.6B model, effectively recovering a significant portion of the 9B model's text quality at a reduced query cost.
• Compact Vectors: Its 128-dimensional token vectors are substantially narrower (16x to 32x) than those of competing models, which typically range from 2,048 to 4,096 dimensions.
• Open Licensing: The MIT license permits both personal and commercial use, fostering broad adoption.

Considerations and Current Limitations

While powerful, the pplx-embed-v2-late models come with certain caveats:

• Index Size: The ColBERT-style architecture stores one vector per token, meaning index size scales directly with document length, potentially leading to larger storage requirements.
• Image Retrieval Gaps: The models do not currently lead in all image retrieval benchmarks; for instance, Tencent's EVIE model surpasses them on ViDoRe v3 image retrieval.
• Input Modality: A single input cannot simultaneously contain both text and images.
• Reporting: All performance metrics are self-reported by Perplexity AI, and a comprehensive technical report is pending release.

Technical Specifications and Hardware Requirements

The models exhibit distinct technical profiles:

• pplx-embed-v2-late-0.6b: Features 594 million total parameters (approx. 240M text, 340M image active), derived from a pruned Qwen3.5-0.8B base. It outputs 128 dimensions per token and is estimated to require about 1.2 GB of memory. This model is designed for laptops, edge devices, or smaller GPUs, serving as a local query encoder.
• pplx-embed-v2-late-9b: With 9 billion total parameters (7.4B active), this model uses the full Qwen3.5 base. It also outputs 128 dimensions per token, demanding approximately 16-18 GB of memory. This larger variant is intended for datacenter deployment or high-memory GPUs, primarily for building document indices.

Both models require CUDA GPU usage and are compatible with sentence-transformers version 6.0.0 or higher, alongside transformers version 5.4.0 or higher.

Performance Highlights and Benchmarks

Perplexity AI's internal benchmarks reveal impressive results:

• MADQA (Agentic PDF QA): The 9B model achieved an accuracy of 92.4%, making it a top performer and surpassing Mixedbread's retriever (88.9%).
• Domain-Specific Text: The 9B model led all tested models with an nDCG@10 score of 81.3%.
• Q2D-Web: Both models set new highs, with the 9B model reaching 74.8% Recall@1000.

The most significant achievement is the 9B model's 92.4% on MADQA. However, the 0.6B model's 61.2% on ViDoRe v3 Markdown was its lowest score, although it still ranked second among external models tested. A notable strategy involves using the 0.6B model to query an index built by the 9B model, which yielded a 63.5% on ViDoRe v3 image retrieval, outperforming setups where both query and index use the 0.6B model.

Use Cases and Core Functionality

The models encode pages as images, eliminating the need for optical character recognition (OCR) and scoring with MaxSim, where each query token finds its best document token, and their maximum similarities are summed. Both models were distilled from an 18B teacher model using LEAF-style token-level training, which facilitated their shared embedding space.

Optimal applications include visual document search across PDFs, slides, and scanned reports. They are also well-suited for low-latency search scenarios, leveraging the 9B model for cloud-based indexing and the 0.6B model for on-device querying. Furthermore, the models hold promise for agentic RAG (Retrieval Augmented Generation) over extensive PDF or web document collections.

Comparative Landscape

Perplexity AI's offerings position themselves against notable competitors such as NVIDIA nemotron-colembed-vl-8b-v2, TopK topk-embed-v1, and Google Gemini Embedding 2. The pplx-embed-v2-late models distinguish themselves with their compact 128-dimension token vectors and shared embedding space across model sizes, while offering an open MIT license unlike some proprietary alternatives.