Skip to content
Live
Trending storyDeveloping

Buoyant Open Source Kinference Inference Control Plane

3 reports3 outlets37 minutes agoUpdate

The story so far

No overview yet — start with the coverage timeline below.

Developments

2 developments

  1. 18 Nov, 00:11 · 1 reports
    Inference engineering for software engineers
    Pragmatic Engineer:Inference engineering for software engineers
  2. 4 Nov, 08:00 · 2 reports
    Introducing Kinference
    vLLM Blog:Introducing Kinference: An Open Source Inference Control Plane for vLLM and Kubernetes

Coverage timeline

Follow the coverage to see every side of the story.

18 Nov
  1. Pragmatic Engineer
    Inference engineering for software engineers

    Uber、Pinterest and AT&T reduced their per-token costs by 50%、90% and 56% respectively by adopting open-source models. Inference providers like Modal、Fireworks AI and Baseten run open-source weight models, unlike AI labs that train their own cutting-edge models, they provide GPU infrastructure and competitive pricing, allowing customers to deploy LLMs through multi-tenant APIs or dedicated GPUs.

7 Nov
  1. MarkTechPost
    Buoyant Releases Kinference: An Open Source Inference Control Plane for Kubernetes and vLLM

    Buoyant has released Kinference, an open-source Kubernetes and vLLM inference control plane, with the current version being v0.1.0-alpha.7. It distributes requests in front of vLLM worker nodes through routing policies, token rate limiting, KV cache-aware load balancing, and tenant-specific metrics. In benchmark tests, for shared prefix workloads, the median TTFT decreased from 6.88 seconds to 0.31 seconds (95%).

4 Nov
  1. vLLM BlogFeatured
    Introducing Kinference: An Open Source Inference Control Plane for vLLM and Kubernetes

    Buoyant releases open-source project Kinference, as an inference control plane for vLLM and Kubernetes, providing token rate limiting, tenant usage and cost tracking, KV cache-aware load balancing, and inference strategies.

Heat over time

Not enough data yet to draw a trend.