Buoyant has released Kinference, an Apache 2.0 inference control plane for vLLM on Kubernetes. It adds declarative routing policies, KV-cache-aware load balancing and per-tenant metrics, and cut median TTFT 95% in Buoya…
开放权重的发布会对闭源定价形成压力、扩大可及性,改变整个生态「自建 vs 采购」的权衡。
摘要仅供参考,请点击来源链接查看全文。演示条目为示意。