从碎片化到融合：一项关于多模态嵌入的综述与路线图
From Fragmentation to Fusion: A Survey and Roadmap for Omnimodal Embeddings
嵌入研究通过对比学习和 Transformer 架构正走向多模态统一，提出以转换、对齐和联合嵌入为核心的框架整合不同模态。嵌入模型支撑着现代 AI 系统，但研究因模态而碎片化，各社区发展孤立。本文提出覆盖度、可比性和组合性框架，通过转换扩展覆盖、对齐实现可比、联合嵌入达成组合，首次对多模态嵌入研究提供统一视角。
Published on Oct 30
Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
Embedding research across modalities is converging toward shared principles via contrastive learning and Transformers, and a framework of coverage, comparability, and compositionality is proposed to unify it through transformation, alignment, and joint embedding.
Embedding models underpin virtually every modern AI system, from search engines to recommendation systems. Despite shared mathematical foundations, research on embedding models remains fragmented by modality: the communities develop largely in isolation, seldom exchanging insights. We argue that this fragmentation obscures a deeper convergence. Driven by contrastive learning, Transformer architectures, and a granularity shift, embedding research across modalities is converging toward a common set of principles, even as modality-specific challenges remain underexplored. To reason across this convergence without conflating it, we propose a framework of three progressively harder capabilities: coverage, comparability, and compositionality. Three paths toward unification satisfy these in turn: transformation broadens coverage, alignment enables comparability, and joint embedding, the terminus we advocate, aims at compositionality. This position paper traces the resulting evolution, providing the first unified, cross-cutting treatment of omnimodal embedding research.
View arXiv page View PDF GitHub 19 Add to collection
Get this paper in your agent:
hf papers read 2611.00172
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2611.00172 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2611.00172 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2611.00172 in a Space README.md to link it from this page.
Collections including this paper 2
来源: Hugging Face Daily Papers · huggingface.co