长期以来，语音识别领域有一个经典且棘手的难题——“鸡尾酒会问题”：人类能轻松在嘈杂人群中聚焦某一个人的声音，但传统自动语音识别模型面对多人同时发言时，准确率往往会断崖式下跌。过去数年间，学术界和工业界尝试了波束成形、盲源分离等多种方案，但始终难以在真实复杂场景下做到精准、高效的“听你想听”。这一技术瓶颈，直到近期才被一次开源动作真正撬动。
Humans can effortlessly focus on and understand a specific person’s voice in a noisy crowd, but for traditional automatic speech recognition (ASR) models, the “cocktail party problem” of multiple people speaking at the same time has long been an unresolved industry challenge. To solve this long-standing issue, Xiaomi has recently officially open-sourced the industrial-level target speaker speech recognition large model CocktailASR-1.
The model completely changes the traditional input method of speech recognition tasks, adopting a target speaker identification mechanism based on reference voiceprint. Users only need to provide a reference audio, and the model will use voiceprint embedding to accurately locate the target speaker, transcribing only the speech of that specific person in a multi-person mixed audio, while automatically filtering out others’ speech, reverberation, and background noise, transforming complex mixed audio tasks into efficient target speaker identification.
In terms of low-level architecture design, CocktailASR-1 adopts an end-to-end large language model architecture, consisting of a D2V2 audio encoder, Adapter, and large model decoder connected in series, with all weights integrated in a single checkpoint. In practical use, the input format is the concatenation of the reference audio and the target monaural audio, with one second of silence inserted as a separator. This design gives the model extremely high versatility, allowing it to handle both single-person and complex multi-scenario situations without switching models seamlessly.
Multiple benchmark test data have verified the model’s strong capabilities. In simulated multi-speaker tests, its word error rate (WER) was reduced to 4.11% and 2.90% on the LibriMix2mix and LibriSpeechMix2mix datasets, respectively. In comparison, several well-known models in the same field performed poorly in mixed scenarios, such as in the LibriMix3mix test, where some competitors had WERs ranging from 76% to 121%, while CocktailASR-1’s WER was only 12.29%, demonstrating a revolutionary advantage. Even in more challenging real meeting recording scenarios, such as AMI SDM and AliMeeting Far, the model significantly outperformed existing solutions. And in single-speaker scenarios, it also achieved comprehensive superiority on datasets like LibriSpeech, WenetSpeech, and CommonVoice-zh.
In addition to breakthrough recognition accuracy, the model also has two notable features in terms of engineering implementation and practicality. First, it has strong negative sample rejection capability. When the reference audio corresponds to someone not participating in the current conversation, the model can directly output empty text, effectively preventing smart speakers, in-car voice assistants, and other intelligent devices from being mistakenly triggered by others’ voices. Tests show that its negative sample rejection rate performs excellently across multiple datasets. Second, the model supports chain-of-thought reasoning functionality, which can display its process of identifying the speaker through specific tags before outputting the final answer. Although this feature has minimal impact on accuracy, it greatly enhances the system’s interpretability and debugging convenience.
Currently, the relevant code for CocktailASR-1 has been officially open-sourced on GitHub under the Apache 2.0 license, with an extremely simple dependency environment, requiring only basic libraries such as torch, torchaudio, transformers, and soundfile to easily load and deploy it from HuggingFace. Xiaomi’s new shift in the fundamental logic of speech technology marks the official transition of speech recognition from the traditional “capturing all sounds” to a new development stage focusing on “locking onto one sound.”
从行业视角来看，CocktailASR-1的开源不仅是一次技术成果的公开，更可能重塑智能语音交互的产品逻辑。以往智能音箱或车载助手需要依赖“唤醒词”来避免误触发，而基于声纹锁定目标说话人后，设备可以更自然地“只听主人的话”。这种范式转变让语音交互从“被动监听”走向“主动辨识”，在会议记录、助听设备、安防监控等场景中具有广阔的应用前景。当然，该模型对参考音频的纯净度仍有要求，且长时多轮对话下的稳定性还需要更多实际场景验证，但就目前公开的基准测试结果而言，它已经为业界提供了一个兼顾精度与易用性的高起点。
