Perplexity has officially launched Photon, an internally developed retrieval and ranking engine engineered in Rust. This advanced system now manages all production search traffic for the company's AI-native platform, replacing a previously forked open-source solution. Photon is also the core technology behind a new "Fast Search" mode available through the Perplexity Search API, promising enhanced speed and efficiency. The company reports single-call latencies of 160 milliseconds at p50 and 230 milliseconds at p95 for its standard operations.
Transforming Search Performance
The introduction of Photon addresses critical performance bottlenecks encountered with the prior engine as Perplexity's data index expanded. The original system struggled with significant tail latency, often reaching 800 milliseconds at p99, largely due to data exceeding available RAM and the resulting page faults from cold reads. Furthermore, index merge operations caused latency spikes extending up to 1.2 seconds for durations of 10 to 15 minutes, while cluster recovery times could stretch beyond a week, increasing partial query responses. The development team ultimately determined that constructing a new engine from the ground up offered a more streamlined and cost-effective solution compared to maintaining the existing fork.
Engineered for Speed: How Photon Works
Photon employs a sophisticated architecture designed for minimal latency and high efficiency. Search requests are initially routed by a load balancer to a Photon broker. This broker then distributes requests across a group of data shards, simultaneously monitoring for timeouts. Each shard independently executes retrieval, initial ranking, and a second-stage ranking process. The broker subsequently consolidates the results and retrieves essential document fields.
- Adaptive Posting Lists: Efficiently stores document IDs; short lists are embedded directly, while longer lists are segmented into fixed-range blocks. Sparse blocks use sorted offset arrays and galloping search, contrasting with dense blocks that employ bitmaps for single-bit membership lookups.
- Budgeted Traversal: A WAND-like algorithm separates lists into primary driving lists and secondary probe lists. Preliminary presence checks quickly establish a candidate's maximum potential score, with exact term frequencies only being accessed if a candidate can exceed a set threshold.
- Docblob Records: Each document is assigned a compact record containing frequencies, field masks, and positions. Elias-Fano encoding is utilized for terms, ensuring that the ranking process only decodes the terms that are actually matched, requiring just one lookup per document for ranking.
- Batched Asynchronous Reads: Known record offsets allow disk reads to be processed in batches via io_uring. The system checks the entire batch against its cache simultaneously. Readers operate without locks, and a CLOCK algorithm handles eviction, avoiding a shared LRU list.
- Separate Build and Serve: Indexers construct versioned shard indexes from YTsaurus tables on dedicated nodes. A central controller manages the rotation of serving groups incrementally, pre-warming caches with replayed queries from search logs. This optimized process now builds a complete web index within a few hours.
Significant Production Improvements
The deployment of Photon has yielded impressive results across Perplexity's production environment:
- P99 retrieval and ranking latency plummeted from approximately 800 milliseconds to just 65 milliseconds.
- The new engine operates on roughly 20% fewer serving machines compared to its predecessor.
- It stores approximately 2.5 times more data per document, which Perplexity leverages for enhanced ranking quality.
- Photon achieves substantial memory efficiency, utilizing an estimated 4.6 times less resident memory than an mlock-pinned equivalent dataset.
- Index version switches no longer trigger disruptive latency spikes.
"Fast Search" for Agentic Workflows
Perplexity's "Fast Search" mode combines the Photon engine with a streamlined ranking algorithm specifically tailored for agentic workflows. Performance tests across multiple benchmarks, including WideSearch, BrowseComp, and DSQA, demonstrated that Fast Search delivered comparable task quality (64.3%) to the default preset (64.0%) but at a significantly lower estimated cost per task ($59.73 versus $187.60), representing approximately 68% savings. While offering substantial cost benefits, "Fast Search" does present a trade-off in broader search quality, exhibiting a slight reduction in relevance and answer availability on internal long-tail benchmarks. Therefore, Perplexity suggests using "Fast Search" for routine agent loops and reserving the default preset for more complex or ambiguous queries. Access to this feature is provided via a hosted API, requiring users to set search_type: "fast" in their POST requests at a cost of $1 per 1,000 requests. The underlying Photon engine is not open source, meaning it cannot be self-hosted.
This article is a rewritten summary based on publicly available reporting. For the original story, visit the source.
Source: MarkTechPost
