Weekly Research Intelligence — OR for Generative AI

Optimizing LLM Inference and Automated OR Problem Generation

This research front explores the application of advanced Operations Research (OR) principles and AI techniques to two critical challenges within the LLM ecosystem: optimizing LLM inference serving and automating the generation of OR models. For LLM inference, papers focus on sophisticated scheduling algorithms to manage GPU resources, KV cache memory, and request batching under various constraints. Concurrently, other works leverage large language models themselves to automatically formulate and solve complex OR problems, bridging the gap between natural language problem descriptions and executable optimization models.

Key contributions in LLM inference scheduling include Nested WAIT (Paper 1) for multi-stage online scheduling, achieving superior throughput and reduced latency on vLLM/Sarathi. Staggered Batch Scheduling (SBS) (Paper 2) for DP+EP architectures reduced Time-to-First-Token by 30-40% and increased throughput by 15-20% on DeepSeek-V3. Memory Constrained Shortest First (MC-SF) (Paper 4) achieved near-optimal latency (within 5% of hindsight optimal) for KV cache-aware online scheduling. Justitia (Paper 6) introduced a virtual-time based fair scheduler, reducing average job completion time by ~60%. In automated OR modeling, OR-R1 (Paper 3) integrated SFT and Test-Time Group Relative Policy Optimization (TGRPO) to improve modeling accuracy by +4.2% over ORLM. ORLM (Paper 7) and the OR-Instruct framework fine-tuned LLMs with synthetic data, outperforming GPT-4 by up to 38.4% on benchmarks like NL4OPT and IndustryOR.

This front is emerging, with significant activity in both LLM inference optimization and automated OR modeling. The trajectory suggests continued innovation in developing more robust and adaptive scheduling policies for increasingly complex LLM architectures and deployment scenarios. For automated OR, the next papers will likely focus on enhancing LLM capabilities for solution ranking, handling more diverse and complex problem types, and integrating multi-agent collaboration for sophisticated problem-solving.

Optimal LLM Inference Scheduling with Queueing Theory and MIP

This research front unifies advanced Operations Research techniques, specifically many-server queueing network models, Mixed-Integer Programming (MIP), and stochastic control, to achieve optimal or near-optimal LLM inference scheduling and resource allocation. Key themes include managing prefill-decode contention, optimizing disaggregated expert parallelism in Mixture-of-Experts (MoE) models, and automatic operator-level parallelism planning for distributed deep learning.

Key contributions include Lin et al.'s (2026) 'Gate-and-Route' policy derived from a fluid LP, demonstrating ~30% lower revenue loss than OPT on Dolly-15k. She et al. (2025) used MIP for operator-level parallelism, reducing pipeline bubbles by 50% for DeepSeek V3. Bari et al. (2025) introduced RAD and SLAI schedulers, achieving a 53% reduction in median TTFT and 26% capacity increase over Sarathi-Serve on Mistral-7B. Pang et al. (2025) combined offline bin-packing with an online Lagrangian heuristic, improving utilization by 8.86% over vLLM FCFS on LLaMA-65B. FinDEP (2025) optimized MoE inference with fine-grained scheduling, yielding up to 1.61x throughput improvement on Qwen3-235B, while DREX (2025) showed 2-12% throughput gains on Llama-EE-70B using dynamic rebatching for early-exit LLMs.

This front is emerging, characterized by a strong trend towards rigorous mathematical modeling to solve complex LLM serving challenges, moving beyond heuristic-driven approaches. The next papers will likely focus on relaxing simplifying assumptions (e.g., exponential service times), integrating stochastic programming for uncertainty, and extending these optimal control strategies to more heterogeneous and dynamic LLM architectures, such as those with speculative decoding or Mixture-of-Depths.

Linear Programming for MoE LLM Mixed-Precision Quantization and Pruning

This research front centers on the Mixture Compressor (MC and MC#) frameworks, which leverage Operations Research techniques, specifically Linear Programming, for the extreme compression of Mixture-of-Experts (MoE) Large Language Models. The core theme involves optimizing mixed-precision quantization and dynamic expert pruning to significantly reduce model size while preserving performance, enabling deployment on resource-constrained hardware.

The initial paper, "Mixture Compressor for Mixture-of-Experts LLMs Gains More" (Huang et al., 2025-02), introduces a hybrid post-training quantization and dynamic pruning approach. It uses Linear Programming to optimally allocate mixed bit-widths (1-3 bits) to experts based on activation frequency, achieving strong empirical results such as compressing Mixtral 8x7b to ~16GB with only a ~4% drop in zero-shot accuracy, outperforming uniform GPTQ. Building on this, "MC#: Mixture Compressor for Mixture-of-Experts Large Models" (Huang et al., 2025-10) refines the framework by combining Pre-Loading Mixed-Precision Quantization (PMQ) via Linear Programming with Online Top-any Pruning (OTP) using Gumbel-Softmax sampling. This unified approach achieves a 6.2x weight reduction on DeepSeek-VL2 with less than 2% accuracy loss, further demonstrating the efficacy of OR-driven compression.

This front is currently emerging, with two closely related papers published in 2025, the second building directly on the first. The trajectory suggests a continued focus on refining these OR-driven compression techniques. The likely next paper would explore the adaptation of these methods to new model architectures or more challenging, complex reasoning tasks, while also addressing hardware-specific optimizations.

Integer Linear Programming for Heterogeneous LLM Serving Resource Allocation

This research front is unified by the application of Integer Linear Programming (ILP) and Mixed-Integer Linear Programming (MILP) to optimize various aspects of Large Language Model (LLM) serving. The core theme revolves around efficient resource allocation, scheduling, and deployment strategies for LLM inference, particularly addressing challenges posed by heterogeneous GPU clusters, disaggregated serving architectures, and geographically distributed systems. Specific problem domains include multi-round inference, Mixture-of-Expert (MoE) model placement, cascade serving, and carbon-aware caching.

Key contributions include Dynamo's ILP-based offline deployment for multi-round inference, achieving up to 340% SLO attainment improvement (Paper 1). Jiang et al. (Paper 3) demonstrated ~25% throughput gains over SOTA systems like Helix by co-optimizing GPU composition and parallelism using MILP for heterogeneous clouds. CASCADIA (Paper 5) introduced a bi-level optimization (MILP for deployment, Chebyshev for routing) for cascade serving, yielding 2.3x average throughput gains. SageServe (Paper 6) achieved 25% GPU-hours savings and $2.5M/month savings by coupling ILP with ARIMA forecasting for auto-scaling. Helix (Paper 10) formulated distributed LLM serving as a max-flow MILP, achieving up to 3.3x throughput gains on mixed GPU clusters by dynamic per-request routing. Other notable works include ILP for MoE expert placement (Paper 4), carbon-aware KV cache management (Paper 9), and hybrid RL-Optimal Transport for temporal-aware GPU allocation (Paper 7).

This front is rapidly maturing, driven by the increasing complexity and scale of LLM deployments. The consistent success of ILP/MILP in achieving significant performance, cost, and energy efficiency gains across diverse LLM serving scenarios indicates a strong and active research trajectory. Future work will likely focus on developing more scalable and dynamic OR solutions, integrating these with advanced LLM-specific optimizations, and expanding to multi-objective and real-time adaptive systems.

OR-Driven Adaptive Resource Allocation for LLM Inference and Test-Time Search

This research front focuses on applying advanced Operations Research techniques to optimize various aspects of Large Language Model (LLM) inference and serving. Key approaches include Bayesian adaptive stopping for efficient ensemble evaluation, gradient-based scheduling in GoodSpeed for distributed speculative decoding, PROBE's predictive Lookahead Pipelining for MoE inference, and ETS's Integer Linear Programming for KV cache optimization in tree search. Additionally, DORA employs embedding-based resource allocation for test-time search, and VectorLiteRAG uses analytical modeling for RAG serving.

Contributions include Bayesian adaptive sampling (2-5x compute reduction on AIME/GPQA) and MILP for LLM ensembles. GoodSpeed demonstrates gradient-based scheduling for distributed speculative decoding, achieving fair goodput on H100/L4 clusters. PROBE's Lookahead Pipelining yields a 1.3x speedup for Qwen3-MoE-235B inference, while ETS leverages ILP to achieve 1.8x KV cache reduction and 1.4x throughput increase on MATH500 compared to REBASE. VectorLiteRAG provides 1.5x throughput gains for RAG serving on H100/L40S, and DORA achieves state-of-the-art accuracy on MATH500 with 3.5x fewer FLOPs by optimizing test-time search.

This front is rapidly maturing, characterized by the increasing sophistication of OR methods integrated directly into LLM serving and reasoning pipelines. The trajectory points towards more tightly coupled, real-time optimization, where OR solvers dynamically adapt resource allocation and search strategies. Future work will likely focus on developing unified frameworks that combine predictive modeling, adaptive sampling, and advanced combinatorial optimization to handle the stochastic and dynamic nature of LLM workloads across diverse hardware architectures and reasoning tasks.

Convex Optimization and Game Theory for Robust Multi-Objective LLM Alignment

This research front unifies recent advancements in applying Operations Research techniques, specifically convex optimization and game theory, to the challenging problem of multi-objective alignment for Large Language Models (LLMs). Papers introduce frameworks like PAMA, Safety Game, and Robust Multi-Objective Decoding (RMOD) to manage conflicting objectives such as harmlessness, helpfulness, sentiment, and length control, often at inference time.

Key contributions include the PAMA algorithm, which transforms multi-objective RLHF into an O(n) convex optimization problem with a closed-form solution, outperforming MORLHF and MGDA-UB on LLaMA-2 7B for harmlessness. The Safety Game formulates black-box LLM agent alignment as a zero-sum game solvable by an LP solver at inference, achieving up to two-fold accuracy improvement on SafetyBench. RMOD introduces a maximin two-player game for robust multi-objective decoding, solving a convex optimization problem at each step to maximize worst-case value, outperforming MO-DPO and scalarized baselines by +1.2% WCWR on Anthropic HH.

This front is rapidly growing, demonstrating the power of OR principles to bring robustness and efficiency to LLM alignment. The trajectory indicates a strong focus on mathematically grounded, inference-time control mechanisms. Future work will likely focus on extending these frameworks to more complex, dynamic, and multi-agent scenarios, improving their scalability to a greater number of objectives, and integrating these control mechanisms into broader agentic architectures.