Weekly Research Intelligence — Optimizing AI Systems

MILP-Driven Resource Allocation for LLM Serving and Multi-Agent Orchestration

This research front unifies two critical areas: the application of Operations Research (OR), particularly Mixed-Integer Linear Programming (MILP) and Integer Linear Programming (ILP), for optimizing Large Language Model (LLM) serving and resource allocation and the development of multi-agent LLM systems for complex reasoning and design tasks. Key frameworks include PROBE for MoE inference, AREAL-Hex and Helix for heterogeneous GPU scheduling, SageServe for cloud autoscaling and Staggered Batch Scheduling for efficient inference. In multi-agent systems, Glia and MaMa explore automated system design, while AdaptOrch and MARINE focus on adaptive orchestration and reasoning refinement.

Several papers demonstrate significant performance gains through OR-driven approaches. PROBE achieved 1.32x speedup in prefill latency for MoE inference. AREAL-Hex and HetRL showed up to 1.5x and 10.76x throughput increases, respectively, for RL training on heterogeneous GPUs using MILP and multi-level search. SageServe reduced GPU hours by 25% for LLM serving on production traces. Helix achieved 3.3x decode throughput gains by formulating serving as a max-flow problem. Staggered Batch Scheduling reduced Time-To-First-Token (TTFT) by 30-40% and improved throughput by 15-20% for DeepSeek-V3. In multi-agent systems, Glia outperformed AlphaEvolve-like systems by 1.3-1.7x in mean response time for LLM serving optimization and MaMa reduced attack success rates from ~50% to ~15-25% using adversarial co-evolution. DynaMO improved Pass@1 by 11.8% over GRPO for mathematical reasoning.

This front is emerging rapidly, driven by the increasing complexity of LLM architectures (e.g., MoE, long-context) and the need for efficient, robust deployment. The trajectory indicates a strong shift towards integrating formal OR methods (MILP, DP, queueing theory) directly into LLM serving and training pipelines, moving beyond heuristics. For multi-agent systems, the focus is on robust, adaptive orchestration and automated design, often leveraging adversarial training or meta-learning. The likely next papers will involve more sophisticated hybrid OR-ML approaches, real-time adaptive scheduling for highly dynamic workloads and frameworks that can automatically generate and validate agentic system designs under uncertainty, potentially integrating generative ambiguity sets from 3D-Learning or the 'Offline MIP -> Online ICL' paradigm from OSCAR.

Mathematical Optimization for LLM Inference Serving and GPU Resource Allocation

This research front focuses on applying rigorous mathematical optimization and algorithmic foundations, including linear programming (LP), mixed-integer linear programming (MILP), constraint programming (CP-SAT), queueing theory and stochastic control, to address the complex resource allocation and scheduling challenges inherent in large language model (LLM) inference serving. The unifying theme is a shift from ad-hoc heuristics to principled operations research (OR) methods for managing dynamically growing KV caches, prefill-decode asymmetry, multi-agent workflows and heterogeneous GPU infrastructures.

Key contributions include the Global Resolution algorithm for optimal multi-draft speculative sampling, achieving 10,000x speedup over general LP solvers with acceptance gains up to +1.71%. HAP utilizes ILP for dynamic MoE parallelization, yielding up to 1.77x speedups. Dynamo and AMPD leverage ILP for disaggregated serving, improving SLO attainment by 67-340%. FREESH combines MILP with MIAD for carbon-aware serving, reducing emissions by 45% and energy by 28%. SAGA introduces workflow-atomic scheduling, resulting in 1.64x speedup and 1.22x better memory utilization for AI agent inference. Other notable advances include Entropic-Time Inference for 30-45% throughput increase, IEMAS using Min-Cost Max-Flow for 80.2% KV-cache hit rates and DiP-SD's fractional MILP for up to 1.93x throughput in distributed speculative decoding.

This front is rapidly emerging, driven by the increasing complexity and scale of LLM deployments. The trajectory indicates a strong move towards integrating OR techniques with real-time system dynamics. Future work will likely focus on developing adaptive, provably optimal online control policies that account for stochastic arrivals, multi-objective trade-offs (cost, carbon, latency, fairness) and the intricate interactions within multi-agent and disaggregated LLM architectures.

ILP/RL for Dynamic LLM Serving and Robust Agentic Coordination

This research front unifies advanced Operations Research (OR) techniques, notably Integer Linear Programming (ILP) and Mixed-Integer Programming (MIP) (e.g., MoETuner, JIGSAWSERVE, FineMoE, Puzzle), with Reinforcement Learning (RL) (e.g., AgentConductor, SAMPO, SRPO, LCO) to address critical optimization challenges in large language model (LLM) serving and multi-agent AI systems. The core theme revolves around developing dynamic, hardware-aware resource allocation and scheduling strategies for LLM inference, alongside robust policy optimization and coordination mechanisms for complex agentic workflows.

Key contributions demonstrate significant quantitative improvements. For LLM serving, ILP/MIP formulations achieve up to 17.5% speedup in MoE serving (MoETuner), 11.3x capacity increase for compound inference (JIGSAWSERVE), 5.3x higher goodput with dynamic Tensor Parallelism (Nitsum), 78.5% TTFT reduction for multimodal LLMs (RPS-Serve) and 2-10x throughput gains for prefix-aware scheduling (Feather). Queueing theory provides throughput-optimal scheduling policies (Li et al., Bari et al.). For multi-agent systems, ILP-based LLM selection reduces costs by ~80% (BAMAS), RL-optimized orchestrators improve code generation by ~14% (AgentConductor) and robust policy optimization (SRPO, SAMPO) prevents free-riding and stabilizes agentic RL, achieving +25.2% average score on ALFWorld. Bayesian methods (B4, Best-of-Infinity) enhance code assessment and reduce evaluation costs by 2-5x. System-level optimizations include GPU-accelerated metaheuristics (cuGenOpt), MILP for NAS (Puzzle) and convex optimization for cloud resource allocation (BOA Constrictor, SkyNomad).

This front is rapidly maturing, moving from foundational OR formulations to highly specialized, hardware-aware and dynamic optimization strategies. The trend is towards integrating OR solvers directly into runtime systems (e.g., real-time ILP for scheduling, dynamic programming for pipelining) and leveraging LLMs not just as targets but as components within optimization loops (e.g., LLM orchestrators, reward models, feature extractors). The next wave of papers will likely focus on holistic, multi-objective optimization across heterogeneous hardware (CPU/GPU/edge), incorporating uncertainty and developing adaptive, self-optimizing systems that learn and reconfigure in real-time.