嵌入向量模型正在成为AI基础设施中不可忽视的一环——无论是RAG（检索增强生成）的召回质量，还是智能体记忆的长期语义对齐，都高度依赖embedding的精度和效率。就在最近，NVIDIA一口气丢出三枚Nemotron-3-Embed“烟雾弹”：8B版本直接登顶权威检索嵌入基准RTEB，1B版则通过“压榨”3B模型一路修剪到1.14B参数，还顺手掏出了一个专为Blackwell架构优化的4位NVFP4版本。这波操作很值得细品。
NVIDIA recently released the Nemotron3Embed series of embedding vector models, specifically designed for production-level RAG, intelligent question retrieval, code retrieval, and agent memory scenarios. The 8B version ranks first on the retrieval embedding benchmark RTEB, becoming the most powerful open-source model in this field.
The series includes three open checkpoints: Nemotron-3-Embed-8B-BF16 (precision-focused), Nemotron-3-Embed-1B-BF16 (lightweight version), and Nemotron-3-Embed-1B-NVFP4 (4-bit version optimized for Blackwell architecture). All three use a Transformer encoder with bidirectional attention masking training, have a maximum sequence length of 32,768 tokens, support 34 languages, and are open-sourced under the OpenMDW-1.1 license. Notably, all base models are based on the Mistral architecture, with the 8B version derived from Ministral-3-8B-Instruct-2512, and the two 1B variants based on Ministral-3-3B-Instruct-2512.
在RTEB基准的16个公开任务上，8B-BF16版本以平均NDCG@10 78.46分夺得第一，这个分数直接压过此前所有开源embedding模型。1B-BF16版本拿到72.38分，比上一代基线llama-nemotron-embed-vl-1b-v2提升了10.4分——注意后者同样是1B级别。而最特别的1B-NVFP4版本，在Blackwell架构上仅损失0.38分，精度保留99.5%，吞吐量达到了BF16的两倍。这意味着在Blackwell上部署，你几乎可以用一半的算力获得相同质量的召回。
The construction of the 1B model adopted a compression rather than a small-scale training approach. The research team first used NVIDIA ModelOpt’s neural architecture search to prune the 3B base model to 2B, then performed knowledge distillation from the fine-tuned 8B embedding vector teacher model using cosine distance loss and mean square error loss, iterating to 1.14B parameters. The NVFP4 version further conducted quantization-aware distillation, calibrating with 512 samples and training with 20,000 samples, restoring precision in long input scenarios.
部署层面也有明显差异：8B和1B的BF16版本都支持Transformers和Sentence Transformers框架，可以直接用标准流程加载。而1B-NVFP4只支持vLLM 0.25.0/v2/embed接口——这也是NVIDIA加速库的专属通道。在微架构兼容性上，NVFP4版本覆盖Ampere、Hopper、Lovelace和Blackwell，而BF16版本主要面向Ampere、Hopper和Blackwell。值得注意是，NVIDIA还为1B模型专门发布了用Rust构建的NIM微服务，在GB200和RTX PRO6000上，性能达到或超越了vLLM checkpoint。
In terms of application scenarios, the series supports multilingual enterprise search (cross-language retrieval), code retrieval (training data includes code datasets such as SWE-bench), and agent memory (32K token long context support for longer conversation summaries). For cost-sensitive scenarios, a hierarchical RAG strategy of “1B-NVFP4 handling high-capacity recall + 8B handling difficult queries” can be adopted.
最后聊聊场景价值。嵌入模型除了经典的RAG外，这次明确强调了“智能体记忆”和“代码检索”——32K超长上下文意味着agent可以存储更长的对话摘要，而不是被截断。对于成本敏感的企业环境，NVIDIA给出的建议是分层RAG：用1B-NVFP4处理海量召回（吞吐高、成本低），遇到难查询时再交给8B精排。这种“轻量粗筛+重量精排”的架构在工业界已经验证过多次，现在有了原生Blackwell优化的4位模型，性价比曲线会更陡。
NVIDIA also provides complete code examples, covering local inference based on Sentence Transformers and server-side deployment based on vLLM. Queries and documents are distinguished by the `query:` and `passage:` prefixes, and the dot product of the embedded vectors after L2 normalization equals the cosine similarity.
横向对比来看，目前开源embedding领域，阿里GTE系列的1.5B模型在MTEB上表现不错，但Nemotron-3-Embed直接把1B级别的NDCG@10拉到了72.38，并且通过知识蒸馏将8B的知识注入小模型。这种“大模型做老师、小模型做执行”的思路正在成为开源embedding的新范式。而NVFP4这种面向特定硬件架构的极致优化，也给后续AI基础设施的软硬协同提供了样本——毕竟，在Blackwell上跑BF16加两倍吞吐的4位模型，这账怎么算都划算。
