From Fragmentation to Fusion: A Survey and Roadmap for Omnimodal Embeddings
From Fragmentation to Fusion: A Survey and Roadmap for Omnimodal Embeddings
Embedding research is moving towards multimodal unification through contrastive learning and Transformer architectures, proposing a framework centered on transformation, alignment, and joint embedding to integrate different modalities. Embedding models support modern AI systems, but research is fragmented by modality, with each community developing in isolation. This paper proposes a coverage, comparability, and combinability framework, extending coverage through transformation, achieving comparability through alignment, and reaching combinability through joint embedding, providing a unified perspective on multimodal embedding research for the first time.
Published on Oct 30
Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
Embedding research across modalities is converging toward shared principles via contrastive learning and Transformers, and a framework of coverage, comparability, and compositionality is proposed to unify it through transformation, alignment, and joint embedding.
Embedding models underpin virtually every modern AI system, from search engines to recommendation systems. Despite shared mathematical foundations, research on embedding models remains fragmented by modality: the communities develop largely in isolation, seldom exchanging insights. We argue that this fragmentation obscures a deeper convergence. Driven by contrastive learning, Transformer architectures, and a granularity shift, embedding research across modalities is converging toward a common set of principles, even as modality-specific challenges remain underexplored. To reason across this convergence without conflating it, we propose a framework of three progressively harder capabilities: coverage, comparability, and compositionality. Three paths toward unification satisfy these in turn: transformation broadens coverage, alignment enables comparability, and joint embedding, the terminus we advocate, aims at compositionality. This position paper traces the resulting evolution, providing the first unified, cross-cutting treatment of omnimodal embedding research.
View arXiv page View PDF GitHub 19 Add to collection
Get this paper in your agent:
hf papers read 2611.00172
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2611.00172 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2611.00172 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2611.00172 in a Space README.md to link it from this page.
Collections including this paper 2
Source: Hugging Face Daily Papers · huggingface.co