跳到正文
实时
蚂蚁 inclusionAI · Hugging Face 新模型·· 13 小时前

DeepSeek-V4.1-Flash-1017-singprobe模型介绍

inclusionAI/DeepSeek-V4.1-Flash-1017-singprobe

要点速览

SingProbe是DeepSeek-V4.1-Flash-1017的内在流式护栏,其注意力头在生成时读取基础模型的3层隐藏状态,并在每个token上评分查询意图、响应不安全性和幻觉风险,基础模型Probe参数为10.10M,输出8项指标,包括查询意图、响应不安全性、幻觉风险,部署时可用8个H20 GPU,上下文长度16384。

正文

DeepSeek-V4.1-Flash-1017-singprobe

Model Description

SingProbe is an intrinsic streaming guardrail for DeepSeek-V4.1-Flash-1017. Its attention head reads three layers of the base model's hidden states during generation and scores query intent, response unsafety, and hallucination risk at each token.

Base model Probe parameters Tapped layers (zero-based) Outputs
DeepSeek-V4.1-Flash-1017 10.10M [12, 25, 39] 8 query intents + response unsafety + hallucination

The probe uses 8 attention heads with a head dimension of 64 and a 2,048-token window. The release weights use bfloat16. For the method and training code, see the SingProbe technical report and SingProbe repository.

Evaluation

Each benchmark has equal weight in its suite. Query F1 uses generated responses. Response safety F1 uses the benchmark-provided responses. For each F1 row, one threshold is selected across the suite to maximize mean F1.

Task Metric DeepSeek-V4.1-Flash-1017-singprobe Reference baseline
Query intent classification (6 benchmarks) F1 0.8756 YuFeng-XGuard-Reason-8B (0.8714)
Response safety classification (8 benchmarks) F1 0.8668 Qwen3Guard-Gen-8B-strict (0.8604)
Streaming safety (3 benchmarks) Response AUC / token AUC 0.9907 / 0.9522 Qwen3Guard-Stream-8B-strict (0.9640 / 0.8893)
Hallucination detection (6 benchmarks) AUC 0.7951 DRIFT (0.8000)
Deployment characteristic Result
Benign-response false-positive rate 0.009% averaged over 5 datasets at the evaluation's pre-calibration response threshold of 0.987179
Calibrated release thresholds 0.5 for response unsafety and 0.5 for hallucination

Quick Start

Use the token-probe build from SGLang branch 261029. With the wheel installed, both model directories mounted, and eight H20 GPUs available, start the server as follows.

python3 -m sglang.launch_server \
  --model-path /home/admin/DeepSeek-V4.1-Flash-1017 \
  --probe-ckpt /yijian3/536716/SingGuardStream-Final/release/DeepSeek-V4.1-Flash-1017-singprobe \
  --trust-remote-code --tp 8 --ep-size 8 \
  --context-length 16384 --chunked-prefill-size 1024 \
  --max-prefill-tokens 8192 --max-total-tokens 32768 \
  --max-running-requests 16 --mem-fraction-static 0.85 \
  --cuda-graph-backend-prefill disabled --cuda-graph-backend-decode disabled \
  --port 30000

Read the per-token scores from meta_info.token_probe_probs in the /generate response. Keys label_0 through label_7 score query intent, label_8 scores response unsafety, and label_9 scores hallucination risk.

Citation

@article{singteam2026singprobe,
  title = {SingProbe Technical Report},
  author = {Sing Team},
  journal = {arXiv preprint arXiv:2608.30703},
  year = {2026},
}

来源: 蚂蚁 inclusionAI · Hugging Face 新模型 · huggingface.co