StreamDuo: A Live Commentator and a Research Agent for Unseen Games
StreamDuo: A Live Commentator and a Research Agent for Unseen Games
StreamDuo pairs a trainable commentator with a frozen research agent that searches for game wiki entries to improve cross-game live commentary without training specifically for the target game. StreamDuo was trained on Baldur's Gate 3 and tested on ten unseen new games, increasing entity recall from 0.082 to 0.116, reducing source game leakage from 9.9% to 1.0%, and achieving the best timing, highest evaluation vitality, and overall quality in the live model.
Published on Oct 31
Authors:
,
,
,
,
,
Abstract
StreamDuo pairs a trainable commentator with a frozen research agent that retrieves game wiki entries to improve cross-game streaming commentary without training on target games.
Streaming video LLMs can now commentate live gameplay, but they are trained and tested on the same games. We study cross-game streaming commentary, in which a model trained on one game commentates games it has never seen, and find that current commentators transfer their timing but not their knowledge of the game world. We propose StreamDuo, which pairs a trainable commentator with a frozen research agent that reads the wiki of the game being played. When the commentator sees something it cannot name, it sends the current frames, and the research agent returns a few wiki entries two seconds later while the stream continues. We train only the commentator, on a single game, in two stages. Information-gain labels teach it to ask only where an answer would help it say what the human caster said, and on-policy self-distillation from a copy of itself that sees the ideal answer teaches it to use noisy answers. Trained on Baldur's Gate 3 and tested on ten unseen games, StreamDuo raises entity recall from 0.082 to 0.116 over Proact-VL trained on the same data, cuts source-game leakage from 9.9% to 1.0%, and achieves the best timing and the highest judged liveliness and overall quality among streaming models.
View arXiv page View PDF Project page GitHub 1 Add to collection
Get this paper in your agent:
hf papers read 2611.01120
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2611.01120 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2611.01120 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2611.01120 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
Source: Hugging Face Daily Papers · huggingface.co