Weekly Research Intelligence — Optimizing AI Systems

Exact OR and Advanced RL for LLM Serving and Agentic Control

This research front unifies advanced Operations Research (OR) techniques and sophisticated Reinforcement Learning (RL) algorithms to tackle critical optimization challenges in Large Language Model (LLM) systems. The core theme involves applying methods like Mixed-Integer Linear Programming (MILP), Max-Flow, Dynamic Programming, Queueing Theory and Robust Optimization alongside PPO variants (GRPO, SAMPO, SRPO, LCO), multi-objective RL (PAMA, RMOD) and RL-driven orchestration (AgentConductor, DCcluster-Opt). These are primarily focused on optimizing LLM serving (inference scheduling, resource allocation, KV cache management, speculative decoding) and enhancing multi-agent LLM systems (coordination, code generation, alignment, robust control).

Key contributions include significant advancements in LLM serving and scheduling. Helix ([12]) and Cascadia ([29]) demonstrate MILP/Max-Flow for heterogeneous GPU scheduling, achieving up to 3.3x throughput gains. SageServe ([4]) and AReaL-Hex ([18]) use ILP for forecast-aware auto-scaling and asynchronous RL training, yielding 25% GPU hour savings and 1.5x throughput increases, respectively. Inference acceleration is boosted by DREX ([8]), CacheFlow ([13]), SBS ([21]), PROBE ([22]), ECHO ([23]) and DiP-SD ([19]), showing 10-62% TTFT reduction (CacheFlow) and up to 5.35x speedup (ECHO). For multi-turn agents, SAGA ([24]) and Continuum ([27]) introduce workflow-atomic scheduling and KV cache TTL, resulting in 1.64x speedup and 3.66x delay reduction. In multi-agent systems, AgentConductor ([7]) uses GRPO for dynamic topology generation, achieving 14% gains on APPS. SAMPO ([9]) and LCO ([30]) stabilize agentic RL training, with SAMPO reaching 92% success on ALFWorld and LCO outperforming PPO/GRPO by 7.8% on MATH500. SRPO ([14]) and UMAD ([15]) improve multi-agent coordination, boosting joint accuracy by up to 19%.

This front is in an emerging phase, driven by the escalating complexity and operational costs of LLM inference and the growing demand for robust, intelligent multi-agent systems. The trajectory indicates a strong convergence of formal OR methods for optimal resource allocation and scheduling with advanced RL for dynamic control and sophisticated agentic behavior. Future work will likely focus on seamlessly integrating offline OR optimization with online adaptive RL policies, developing unified frameworks that bridge system-level optimizations (e.g., KV cache, heterogeneous hardware) with agent-level intelligence (e.g., multi-turn reasoning, tool use) and scaling these solutions to larger, geo-distributed LLM deployments and more complex multi-modal agentic tasks, while also improving the theoretical guarantees and robustness of RL algorithms for LLM control.

Operations Research for LLM Inference, Training and Agentic System Optimization

This research front is dedicated to applying Operations Research (OR) and formal optimization techniques, including Mixed-Integer Linear Programming (MILP), Linear Programming (LP), Dynamic Programming (DP) and combinatorial optimization, to enhance the efficiency and performance of Large Language Model (LLM) systems. Key application areas span LLM inference serving (e.g., KV cache management, MoE load balancing, multi-modal scheduling), distributed LLM training and resource allocation within LLM-powered agentic workflows. Frameworks like DORA, BAMAS, Helium, ALTO, Memo, TORTA, VectorLiteRAG, ODAR, FineEP, QR-Adaptor, Glia, FORGE, FlexSP, MuxTune and cuGenOpt are central to these efforts, demonstrating a clear shift from heuristic-based approaches to theoretically grounded solutions.

Key contributions include DORA achieving state-of-the-art accuracy on MATH500 with 3.5x fewer FLOPs by optimizing reasoning path allocation and BAMAS reducing multi-agent system costs by ~80% on GSM8K/MBPP using ILP for LLM selection. Helium provides up to 100x speedup for agentic workflows via proactive KV cache management, while ALTO accelerates LoRA tuning by 13.8x with MILP-driven scheduling. FineEP improves MoE throughput by up to 47.6% using LP for token scheduling and TORTA reduces distributed inference latency by ~15% through RL supervised by Optimal Transport. ODAR matches PRM performance on MATH without training, cutting compute by 82% and MuxTune achieves 5x throughput gains for multi-task PEFT via DP-based multiplexing. These results consistently demonstrate that formal OR methods significantly outperform traditional heuristics in various LLM system contexts.

This front is rapidly emerging, driven by the increasing computational demands and complexity of LLM systems. The trajectory indicates a strong and necessary shift from heuristic-based approaches to theoretically grounded, provably optimal or near-optimal OR solutions for critical bottlenecks. Future work will likely focus on integrating these formal methods into dynamic, real-time systems, addressing multi-objective optimization and extending to more complex, stochastic and multimodal workloads. The next papers will likely combine advanced OR techniques with learned components (e.g., RL, GNNs) for adaptive, robust and scalable resource management across heterogeneous hardware, further solidifying the role of OR in generative AI.

Integer Programming and Queueing for LLM Infrastructure and Agentic Workflows

This research front is defined by the rigorous application of Operations Research (OR) techniques, predominantly Integer Linear Programming (ILP)/Mixed-Integer Programming (MIP) and queueing theory, to address critical optimization challenges across Large Language Model (LLM) systems. The focus spans efficient resource allocation and scheduling for LLM inference and training, as well as the formal design, orchestration and robustness of multi-agent LLM systems. Key areas include optimizing frameworks like vLLM and Sarathi-Serve, MoE expert placement, GPU multi-tenancy, agentic workflow serving and LLM prompt engineering.

Papers in this front demonstrate significant quantitative improvements. For instance, HAP [2] achieves up to 1.77x speedup in MoE inference using ILP for dynamic hybrid parallelism. SkyNomad [9] delivers 1.25-3.96x cost savings for AI batch jobs by optimizing multi-region spot instance usage. AdaptOrch [3] improves multi-agent task completion by 9.8% on SWE-bench through task-adaptive topology routing. TRAIL [20] reduces mean latency by 2x and time-to-first-token by 24x in vLLM via embedding-based output length prediction. JIGSAWSERVE [52] shows an 11.3x capacity improvement for compound inference DAGs using MILP. Furthermore, BandPO [5] and DynaMO [57] enhance LLM policy optimization, with BandPO gaining ~10 points on math tasks and DynaMO achieving +11.8% Pass@1. OCCAM [10] provides 40% cost reduction with<1% accuracy drop for model selection using ILP, while DFLOP [21] accelerates MLLM training throughput by 1.2x-3.6x with hybrid ILP/LPT scheduling.

This front is rapidly maturing, characterized by a decisive shift from heuristic-driven system design to mathematically rigorous, provably optimal or near-optimal solutions. The immediate trajectory points towards integrating these OR solutions into real-time, dynamic and highly heterogeneous environments, tackling challenges such as non-stationary workloads, multi-cloud deployments and complex multi-objective optimization (e.g., balancing cost, carbon emissions, latency and quality). A parallel trend involves leveraging OR to directly guide LLM behavior, as seen in prompt engineering [36] and optimization-steered agentic planning [58], indicating a deeper fusion of OR principles within AI systems themselves.