Cohere discusses the choice of shared or dedicated inference for Embed & Rerank workloads.
When analyzing Embed & Rerank workloads, shared inference is suitable for batch processing (such as e-commerce indexing) to improve cost-effectiveness, while dedicated inference is better for re-ranking with a high number of candidate documents (such as fine-tuning search results); the choice should be based on request characteristics (batch processing/single query/candidate count), traffic patterns (continuous/bursty), and utilization, rather than purely price comparison.












