Weekly Research Intelligence — Optimizing AI Systems

MILP-Driven Resource Optimization for Heterogeneous LLM Serving and Agentic Systems

The unifying theme of this research front is the application of formal Operations Research (OR) methodologies, predominantly Mixed-Integer Linear Programming (MILP) and Integer Linear Programming (ILP), to optimize complex Large Language Model (LLM) systems. This includes fine-grained resource allocation on heterogeneous GPU clusters for LLM serving, efficient inference-time scheduling and principled design and orchestration of multi-agent AI systems. These OR approaches replace traditional heuristics, providing mathematically optimal or near-optimal solutions for critical performance and cost objectives.

Key contributions span several areas. In LLM serving, papers like FineEP [2] and Helix [7] demonstrate MILP/Max-Flow for MoE load balancing and dynamic request routing on heterogeneous GPUs, achieving up to 47.6% throughput improvement and 3.3x decode throughput, respectively. AREAL-HEX [4] uses MILP for asynchronous RL training on heterogeneous GPUs, yielding 1.5x higher throughput. For agentic systems, OSCAR [3] formulates agentic planning as a two-stage MIP, improving mAP@5 by 23.13% on CIRCO. MaMa [9] employs Stackelberg Security Games for automated safe agent design, reducing attack success rates from 50% to 15-25%. Other notable contributions include ILP for KV cache-aware tree search pruning (ETS [14]), diffusion-augmented Distributionally Robust Optimization for LLM resource provisioning (3D-Learning [12]) and hybrid RL-MILP schedulers (RLTune [13]).

This research front is rapidly emerging and maturing, driven by the increasing complexity and scale of LLM deployments and agentic systems. The trajectory indicates a strong shift towards integrating rigorous OR methods into the core design and runtime of AI infrastructure. Future work will likely focus on scaling these exact solvers to larger, more dynamic environments, developing hybrid OR-learning architectures for adaptive control and extending these principles to increasingly complex multimodal and multi-agent workflows. The next papers will probably explore real-time, adaptive MILP/ILP solutions for geographically distributed LLM inference and novel OR formulations for robust, explainable agentic planning under uncertainty.

Operations Research for LLM Inference and Multi-Agent System Optimization

This research front focuses on applying rigorous Operations Research (OR) methodologies, including Integer Linear Programming (ILP), Mixed-Integer Programming (MIP), Reinforcement Learning (RL), Queueing Theory and Stochastic Control, to optimize complex challenges in Large Language Model (LLM) inference systems and multi-agent AI architectures. Key themes include efficient resource allocation, scheduling and dynamic coordination for LLM serving, as exemplified by frameworks like BAMAS for budget-aware multi-agent systems, MoETuner for Mixture-of-Expert (MoE) serving and SAGA for workflow-atomic agent inference.

Significant contributions include BAMAS achieving ~80% cost reduction on benchmarks like GSM8K and MBPP by optimizing LLM selection and agent topology. MoETuner demonstrated a 17.5% speedup on multi-node H200 clusters for MoE expert placement using ILP. For inference scheduling, Staggered Batch Scheduling (SBS) reduced Time-to-First-Token (TTFT) by 30-40% and improved throughput by ~20% for DeepSeek-V3, while RPS-Serve achieved a 78.5% TTFT reduction for text in multimodal LLM workloads. Feather introduced a prefix-aware scheduler yielding 2-10x higher throughput for prefix-shared workloads. In multi-agent systems, AgentConductor used GRPO to dynamically evolve interaction topologies, showing ~14% gains on APPS. Tang et al. applied combinatorial optimization to KV cache eviction, achieving 80% cache reduction with minimal degradation.

This front is rapidly emerging, driven by the increasing complexity and scale of LLM deployments and multi-agent systems. The trajectory indicates a shift from static, offline optimization to dynamic, adaptive and real-time OR solutions that can respond to fluctuating workloads, heterogeneous hardware and evolving agent behaviors. Future work will likely focus on integrating these diverse OR techniques into holistic, end-to-end optimization frameworks that can provide provable guarantees while maintaining practical scalability and interpretability.

Integer Programming and Queueing Theory for LLM Serving and Agentic Orchestration

This research front focuses on applying rigorous Operations Research (OR) methodologies, including Integer Linear Programming (ILP), Linear Programming (LP), Mixed-Integer Linear Programming (MILP), Satisfiability Modulo Theories (SMT), convex optimization and queueing theory, to fundamentally optimize Large Language Model (LLM) inference, serving and agentic AI systems. The unifying theme is a shift from heuristic-driven system design to mathematically grounded solutions for complex resource allocation, scheduling and control problems. Key areas of application include dynamic KV cache management, multi-agent orchestration, speculative decoding and carbon-aware resource allocation within frameworks like vLLM, SGLang and Sarathi-Serve.

Key contributions include MILP-based GPU scheduling for LoRA tuning (ALTO, 13.8x speedup) and MoE inference (HAP, 1.6x speedup), as well as compound inference serving (JIGSAWSERVE, 11.3x capacity increase). Queueing-theoretic frameworks prove throughput optimality for work-conserving LLM schedulers (Li et al., Bari et al.), with practical implementations like SLAI achieving 53% TTFT reduction and 26% capacity increase. Dynamic KV cache management is optimized through cost-benefit models (Continuum, 3.66x delay reduction) and Logits Cache (Hive, 1.76x speedup). Agentic workflows benefit from multi-agent reasoning (Glia, 1.3-1.7x better RT) and task-adaptive orchestration (AdaptOrch, 9.8% SWE-bench improvement). Speculative decoding is enhanced by budget scheduling (ECHO, 5.35x speedup) and convex minimization (Global Resolution, 10,000x faster than LP). Furthermore, carbon-aware scheduling (FREESH, 45% emission reduction) and Bayesian optimal learning for prompt engineering (Wang et al.) demonstrate the breadth of OR applications.

This front is rapidly maturing, characterized by a strong push to replace ad-hoc heuristics with provably optimal or near-optimal OR solutions. The trajectory indicates a continued focus on integrating these mathematical models into production-grade LLM infrastructure, particularly for dynamic, multi-objective and multi-agent scenarios. Future work will likely emphasize robustness under uncertainty, real-time adaptability and the joint optimization of hardware, software and algorithmic components across distributed, heterogeneous environments.