Fish Audio has rolled out its latest innovation, Drama 3, a text-to-speech (TTS) model designed to transform how developers create spoken audio. This significant release introduces a 35-billion-parameter Mixture-of-Experts (MoE) architecture, notably allowing approximately 3 billion parameters to activate for each token. Drama 3 offers open weights for community access via Hugging Face and is also integrated into the Fish Audio API, presenting a versatile tool for generating rich, acted speech. It provides a robust, self-hostable option suitable for developing narrated dialogue, audiobooks, and dubbing initiatives.
Directable Speech & Multi-speaker Capabilities
A standout feature of Drama 3 is its capacity to interpret plain-language acting directions embedded directly within scripts. Content creators can insert cues such as [barely holding it together] before a line or utilize paired tags like <whisper> to modify specific words or phrases. The system supports directions written in any language, independently of the spoken text. Furthermore, Drama 3 boasts support for 83 languages and facilitates complex multi-character scenes by allowing up to 16 distinct speakers within a single request, identifiable through tags like <|speaker:0|>. The API streamlines production by rendering an entire scene with diverse voices in one unified call.
Under the Hood: Architectural Insights
The architectural foundation of Drama 3 is built upon the Qwen3.5-35B-A3B language model. Its structure includes 40 layers, with a distinct pattern of three Gated DeltaNet layers for every one gated full-attention layer. A key optimization involves retaining a KV cache in only one out of four layers, with DeltaNet layers managing historical context through a fixed-size state. The model incorporates a new Codebook-MTP decoder, a departure from the Fast AR decoder used in its predecessor, S2. This decoder features four shared layers feeding into nine additional layers, each dedicated to an acoustic codebook. Notably, only the initial two codebooks provide feedback to the backbone, a reduction from the ten in S2. The audio tokenizer remains consistent with S2, leveraging the Descript Audio Codec to produce high-fidelity 44.1 kHz audio at roughly 21.5 frames per second. Training involved a multi-stage process, beginning with three pre-training phases, followed by instruction tuning, and concluding with reinforcement learning using GRPO and a single learned human-preference reward model.
Performance Benchmarks & Quality
Fish Audio reports that Drama 3 achieves an impressive 93.2% overall score on the Artificial Analysis Pronunciation Robustness benchmark, securing the top position. This represents a significant improvement over its predecessor, S2.1 Pro, which scored 88.1%. Detailed category performance reveals strengths in brands, places, and technical terms (95.9%), and context-dependent readings (95.3%). Shorthand elements like numbers and dates also performed well at 93.6%. However, the model's weakest area was exact sequences, such as codes or emails, scoring 86.3%. While these benchmarks assess pronunciation accuracy, the model's emotional delivery is currently described qualitatively. On the Artificial Analysis quality leaderboard, Drama 3 Preview holds an Elo rating of 1242, surpassing S2.1 Pro but still trailing some closed API alternatives like ElevenLabs v4 Turbo.
Hardware Requirements for Self-Hosting
For developers opting to self-host Drama 3, specific hardware is required. The Fish Speech installation guide mandates an NVIDIA GPU equipped with at least 48 GB of memory operating in FP8 mode. The model does not support CPU, Apple MPS, or AMD ROCm environments. Mixture-of-Experts components are delivered in FP8, while other weights utilize bf16. Official streaming benchmarks were conducted on a single H200 GPU, demonstrating a median time to first audio of 88.9 ms for one stream and achieving 927.6 frames per second with 16 streams. The serving infrastructure relies on SGLang-Omni, with its optimized kernels validated primarily on Hopper GPUs.
Access, Licensing, and Pricing
Drama 3 is accessible through the Fish Audio API by specifying drama-3 in the model header for POST requests to /v1/tts. Developers are cautioned that incorrect headers will silently default to S2.1 Pro. The standard API pricing is $50 per million characters, though a promotional rate of $10 per million characters is available until November 12, 2026. The open weights are distributed under the Fish Audio Community License v1.0, permitting free use for research and non-commercial endeavors. Commercial applications are also free until annual revenue surpasses USD 500,000. Hosting Drama 3 as a third-party API necessitates a distinct license. All products incorporating the model must display "Built with Fish Audio," and the unauthorized cloning of real voices is strictly forbidden.
This article is a rewritten summary based on publicly available reporting. For the original story, visit the source.
Source: MarkTechPost
