Maximize cache hit rates in multi-agent LLM serving systems using KV-cache reuse, agent-aware scheduling, and workflow-aware caching.
This research focuses on maximizing cache hit rates in multi-agent LLM serving systems. It investigates KV-cache reuse techniques, including prefix caching, segment-level caching, and collective sharing of stable context like system instructions and agent roles. The scope also covers agent-aware scheduling for cache locality and workflow-aware caching that considers agent execution graphs, alongside semantic intent caching and validated tool output caching. The objective is to aggressively cache stable, shared, high-reuse context while carefully managing volatile intermediate states.