inclusionAI/DeepSeek-V4.1-Flash-1017-singprobe
inclusionAI/DeepSeek-V4.1-Flash-1017-singprobe
SingProbe is the intrinsic streaming guardrail of DeepSeek-V4.1-Flash-1017, whose attention heads read 3 layers of hidden states from the base model during generation, and score query intent, response unsafety, and hallucination risk on each token. The Probe parameters of the base model are 10.10M, outputting 8 indicators including query intent, response unsafety, and hallucination risk. It can use 8 H20 GPUs during deployment, with a context length of 16384.
DeepSeek-V4.1-Flash-1017-singprobe
Model Description
SingProbe is an intrinsic streaming guardrail for DeepSeek-V4.1-Flash-1017. Its attention head reads three layers of the base model's hidden states during generation and scores query intent, response unsafety, and hallucination risk at each token.
| Base model | Probe parameters | Tapped layers (zero-based) | Outputs |
|---|---|---|---|
DeepSeek-V4.1-Flash-1017 |
10.10M | [12, 25, 39] |
8 query intents + response unsafety + hallucination |
The probe uses 8 attention heads with a head dimension of 64 and a 2,048-token window. The release weights use bfloat16. For the method and training code, see the SingProbe technical report and SingProbe repository.
Evaluation
Each benchmark has equal weight in its suite. Query F1 uses generated responses. Response safety F1 uses the benchmark-provided responses. For each F1 row, one threshold is selected across the suite to maximize mean F1.
| Task | Metric | DeepSeek-V4.1-Flash-1017-singprobe | Reference baseline |
|---|---|---|---|
| Query intent classification (6 benchmarks) | F1 | 0.8756 | YuFeng-XGuard-Reason-8B (0.8714) |
| Response safety classification (8 benchmarks) | F1 | 0.8668 | Qwen3Guard-Gen-8B-strict (0.8604) |
| Streaming safety (3 benchmarks) | Response AUC / token AUC | 0.9907 / 0.9522 | Qwen3Guard-Stream-8B-strict (0.9640 / 0.8893) |
| Hallucination detection (6 benchmarks) | AUC | 0.7951 | DRIFT (0.8000) |
| Deployment characteristic | Result |
|---|---|
| Benign-response false-positive rate | 0.009% averaged over 5 datasets at the evaluation's pre-calibration response threshold of 0.987179 |
| Calibrated release thresholds | 0.5 for response unsafety and 0.5 for hallucination |
Quick Start
Use the token-probe build from SGLang branch 261029. With the wheel installed, both model directories mounted, and eight H20 GPUs available, start the server as follows.
python3 -m sglang.launch_server \
--model-path /home/admin/DeepSeek-V4.1-Flash-1017 \
--probe-ckpt /yijian3/536716/SingGuardStream-Final/release/DeepSeek-V4.1-Flash-1017-singprobe \
--trust-remote-code --tp 8 --ep-size 8 \
--context-length 16384 --chunked-prefill-size 1024 \
--max-prefill-tokens 8192 --max-total-tokens 32768 \
--max-running-requests 16 --mem-fraction-static 0.85 \
--cuda-graph-backend-prefill disabled --cuda-graph-backend-decode disabled \
--port 30000
Read the per-token scores from meta_info.token_probe_probs in the /generate response. Keys label_0 through label_7 score query intent, label_8 scores response unsafety, and label_9 scores hallucination risk.
Citation
@article{singteam2026singprobe,
title = {SingProbe Technical Report},
author = {Sing Team},
journal = {arXiv preprint arXiv:2608.30703},
year = {2026},
}
Source: Ant Group inclusionAI · Hugging Face new models · huggingface.co