35 recent papers · updated 2026-04-26 · sources deduplicated from arXiv, OpenAlex and Semantic Scholar
2026-04-19 | Peking University Method: Hive multi-agent inference infrastructure with Logits Cache and Agent-Aware Scheduling | LLM role: inference_engine_optimization
Hive is an LLM inference infrastructure that optimizes multi-agent and test-time scaling workloads by introducing Logits Cache for redundant sampling paths and Agent-Aware Scheduling for KV cache eviction. The results are backed by solid empirical evidence, demonstrating a 1.11x-1.76x speedup for re-sampling and a 33%-51% reduction in KV cache miss rates on Qwen3-8B. The key insight is that caching intermediate logits (not just KV states) allows the engine to skip expensive forward passes during stochastic resampling of shared prefixes, while evicting KV cache based on an agent's structural contribution outperforms standard LRU. This is highly relevant for scaling LLM evolutionary search and multi-agent optimization, as it provides concrete systems-level techniques to drastically reduce the inference costs associated with branching generation and complex agent coordination.
2026-04-22 | Tsinghua University Method: Joint optimization of batching, user-to-batch assignment and integer draft lengths formulated as a fractional mixed-integer program, solved by scanning batch counts and iteratively alternating between MILP-based subproblems using Dinkelbach's method. | LLM role: none
This paper formulates the scheduling of distributed speculative decoding (local drafting, centralized verification) as a fractional mixed-integer program to maximize multi-user token throughput. The authors demonstrate up to 1.93x throughput improvements over greedy batching in simulated edge deployments using Qwen3 models. The key insight is that the complex fractional objective of throughput (expected accepted tokens per unit time) can be efficiently decoupled and solved using the Dinkelbach method combined with alternating optimization for batch assignment and draft lengths.
2026-04-16 | Imperial College London, Independent Researcher Method: Aggregate LLM Pipeline for performance prediction combined with a hierarchical heuristic search for joint throughput/latency optimization of GPU allocations and topology-aware fractional placement | LLM role: none
Scepsy is a serving system that schedules multi-LLM agentic workflows onto GPU clusters by profiling relative LLM execution times to create an Aggregate LLM Pipeline and using a hierarchical heuristic for fractional GPU allocation. The results are strongly backed by empirical numbers on a 16-GPU cluster, showing up to 2.4x higher throughput and 27x lower latency compared to baselines like Kubernetes HPA and Ayo. The key insight is that instead of modeling the highly variable end-to-end latency of dynamic agentic workflows, systems can achieve stable steady-state performance predictions by modeling the aggregate fractional demand each LLM places on the system.
2026-04-06 | University of Washington, NVIDIA, Rice University, University of Waterloo, Cisco Research, Independent Researcher Method: SLO-aware dynamic programming scheduler with intelligent video preemption, elastic sequence parallelism and dynamic batching | LLM role: none
GENSERVE optimizes the co-serving of text-to-image and text-to-video diffusion models on shared GPUs using a dynamic programming scheduler that jointly manages step-level preemption, dynamic batching and elastic sequence parallelism. The results are empirically backed, demonstrating up to a 44% improvement in SLO attainment over baselines like SRTF on an 8-GPU cluster. The most useful takeaway is their two-stage DP formulation: they first generate a small set of anchored candidate actions (hold, resume, scale SP) per request, then run a knapsack DP to maximize global SLOs in under 2ms.
2026-03-27 | IMDEA Software Institute, Universidad Politécnica de Madrid Method: Modality-aware dynamic priority scheduling with aging mechanism | LLM role: target_of_optimization
RPS-Serve introduces a modality-aware scheduler for multimodal LLMs that classifies requests into 'rocks' (video), 'pebbles' (image) and 'sand' (text) based on predicted prefill latency and memory, using dynamic priorities and aging to prevent head-of-line blocking. The results are real and backed by solid systems experiments on vLLM, showing a 78.5% reduction in time-to-first-token for latency-critical text requests compared to FCFS and EDF baselines. The core takeaway is that multimodal workloads completely break standard text-only LLM serving assumptions because video/image prefill times and KV-cache footprints are orders of magnitude larger.
2026-03-26 | Microsoft Gray Systems Lab, SK Telecom, Yonsei University Method: Data-driven co-optimization of 3D parallelism configuration and runtime microbatch scheduling using empirical profiling, an expected makespan minimization algorithm and a hybrid ILP/LPT scheduler. | LLM role: none
DFLOP optimizes distributed 3D parallelism for multimodal LLM training by combining offline profiling with an online ILP-based microbatch scheduler to minimize pipeline bubbles caused by heterogeneous data inputs. The results are real and backed by extensive hardware experiments, showing up to 3.6x throughput improvements over Megatron-LM.
2026-03-19 | Independent Researcher, Shenzhen, China Method: GPU-accelerated metaheuristic framework with 'one block evolves one solution' CUDA architecture, two-level adaptive operator selection (AOS) and hardware-aware resource management. | LLM role: modeling_assistant
cuGenOpt is a GPU-accelerated metaheuristic framework that uses a 'one block evolves one solution' CUDA architecture and JIT compilation to solve combinatorial optimization problems. The results are rigorously backed by hardware benchmarks, showing it matches OR-Tools on small instances and vastly outperforms MIP solvers, though it struggles with large-scale VRP (>200 nodes) due to memory limits.
2026-03-17 | Carnegie Mellon University, UC Berkeley, UNC-Chapel Hill, UC Santa Cruz Method: Continual meta-learning with LLM-based gradient-free skill evolution and RL-based LoRA fine-tuning using a process reward model | LLM role: policy_executor, skill_generator, reward_model
MetaClaw is a continual learning framework for LLM agents that combines gradient-free skill evolution (distilling failures into reusable prompt instructions) with asynchronous RL fine-tuning guided by a process reward model. The results are backed by solid empirical gains, showing an 8.25x improvement in end-to-end task completion on a 30-day simulated CLI benchmark.
2026-03-17 | National University of Singapore Method: Workflow-aware LLM serving framework integrating proactive KV cache management, global prompt caching and cost-based cache-aware scheduling based on a templated radix tree | LLM role: none
Helium optimizes LLM serving for batch agentic workflows by modeling them as query plans and using a Templated Radix Tree (TRT) to enable proactive KV caching and cache-aware scheduling. The results are rigorously backed by numbers, demonstrating up to 1.56x speedups over state-of-the-art systems (vLLM, Parrot, KVFlow) on complex multi-agent workflows and the authors even validate their greedy scheduler's optimality gap against an MILP solver. The most valuable takeaway is the TRT abstraction, which captures global prefix hierarchies across a DAG of LLM calls to maximize KV cache reuse, rather than relying on reactive, per-call caching.
2026-03-18 | Shanghai Jiao Tong University Method: VCG-based Min-Cost Max-Flow (MCMF) for bipartite matching, guided by Hoeffding Tree predictive QoS models | LLM role: none
IEMAS routes client requests to distributed LLM agents by formulating the assignment as a Min-Cost Max-Flow bipartite matching problem, using VCG auctions to align economic incentives with KV-cache reuse. The results are backed by solid vLLM simulations, demonstrating an 80.2% KV-cache hit rate and a 35% cost reduction over baselines like GraphRouter.
2026-03-17 | MBZUAI, McGill University, University of Chicago, Tensormesh Inc Method: Two-phase optimization combining M/G/c Kimura approximation for analytical sweep and discrete-event simulation (DES) for verification | LLM role: none
This paper introduces a two-phase capacity planner (M/G/c analytical sweep followed by discrete-event simulation) to find minimum-cost GPU fleet configurations for LLM inference under strict latency SLOs. The results are rigorously backed by simulation across multiple GPU profiles and workload traces, demonstrating that intuitive sizing rules often fail (e.g., slower GPUs can be cheaper due to KV-slot multipliers and analytical models approve broken fleets for high-variance agent traffic).
2026-03-17 | Shanghai Jiao Tong University, Chinese Academy of Sciences, University of Chinese Academy of Sciences Method: GPU coroutines abstraction decoupling logical execution contexts (vCtx) from physical GPU resources (pCtx) via dynamic context binding and cooperative preemption | LLM role: none
DETSHARE introduces 'GPU coroutines' to decouple logical execution contexts from physical GPU resources, enabling fine-grained spatial sharing without modifying kernels to preserve semantic determinism. The results are highly credible and backed by strong empirical numbers on A800/Hopper GPUs, demonstrating up to 79% higher training throughput and 69% lower inference latency compared to temporal sharing baselines.
2026-03-13 | University of Illinois Urbana-Champaign Method: Phase-aware runtime (HeteroServe) with modality-level partitioning, embedding-only transfer and cross-type work stealing | LLM role: none
Yu et al. demonstrate that partitioning multimodal LLM inference at the modality boundary (vision encoder vs. language decoder) reduces cross-device transfer costs by O(L), dropping requirements from GB-scale NVLink to MB-scale PCIe. This enables heterogeneous serving architectures where cheap compute-dense GPUs (RTX 4090) handle vision and expensive bandwidth-dense GPUs (A100) handle language. Results are strongly backed by real hardware deployments, showing a 37% improvement in cost-efficiency over homogeneous vLLM baselines.
2026-03-09 | University of Illinois Urbana-Champaign Method: Mixed Integer Linear Programming (MILP) for joint optimization of model variants, GPU spatial partitions and task-graph-informed budgeting | LLM role: none
JIGSAWSERVE uses a Mixed Integer Linear Programming (MILP) formulation to jointly optimize model variant selection (accuracy scaling) and fine-grained GPU spatial partitioning (MIG/MPS) for serving compound inference DAGs. The results are strongly backed by empirical numbers on real hardware (H100s), demonstrating an 11.3x capacity improvement over the closest prior work (Loki) while maintaining under 0.6% SLO violations.
2026-03-05 | Kingsoft Cloud Method: Hybrid theoretical modeling (M/M/1 queuing theory) and empirical benchmarking for P/D resource calculation | LLM role: none
This paper proposes a hybrid resource allocation method for disaggregated Prefill/Decode (P/D) inference, using M/M/1 queuing theory to model prefill throughput under TTFT constraints and empirical profiling for decode. The results are real and validated on NVIDIA H200 clusters running DeepSeek-V3.1.
2026-03-05 | Nanyang Technological University, Unaffiliated Method: SLO-aware elastic system combining a two-layer Prompt Bank for initial prompt selection and a Workload Scheduler for dynamic multi-GPU allocation | LLM role: feature_extractor
PromptTuner is a cluster management system for LLM prompt tuning that combines a 'Prompt Bank' (retrieving similar past prompts to speed up convergence) with a hierarchical scheduler (warm/cold GPU pools) to meet latency SLOs. The authors demonstrate real-world efficacy on 32-96 GPU clusters, showing 4-8x reductions in SLO violations compared to INFless and ElasticFlow.
2026-03-05 | Fudan University, Shanghai Innovation Institute Method: Band-constrained Policy Optimization (BandPO) using a Band operator to project f-divergence trust regions into dynamic, probability-aware clipping intervals | LLM role: policy_agent
BandPO replaces the standard static clipping in PPO/GRPO with dynamic bounds derived from projecting f-divergence trust regions, specifically addressing a bottleneck where allowable updates vanish for low-probability tokens. Empirical results are rigorous, showing consistent gains (2-10%) on math benchmarks and crucially, maintaining policy entropy where baselines collapse. The key takeaway is that standard clipping scales update margins linearly with probability, effectively freezing rare tokens; BandPO decouples this, allowing the model to actually reinforce novel, high-advantage tail strategies.
2026-03-03 | Fudan University, Shanghai Innovation Institute, Shanghai Academy of AI for Science Method: Bayesian Adversarial Multi-agent Framework for AI4S (BAMF-AI4S) with recursive co-optimization of generated code, test cases and prompts, guided by a non-LLM-based Bayesian updating rule and Bayesian Optimization for code performance estimation. | LLM role: code_writer, decomposition_guide, prompt_optimizer, test_case_generator, solution_generator
The authors propose a multi-agent framework for scientific code generation that couples an adversarial 'Challenger' (generating difficult test cases) with a 'Solver', governed by a Bayesian update rule. Crucially, they employ Bayesian Optimization with a kernel based on code embeddings (AST + text) to estimate solution quality before running expensive tests, effectively acting as a learned surrogate model. Results on SciCode and ScienceAgentBench are strong, showing small models (Qwen-32B) outperforming GPT-4o when using this loop.
2026-03-03 | Tencent Hunyuan, Hong Kong University of Science and Technology Method: Native visual-generation agentic model (VisionCreator) unifying Understanding, Thinking, Planning and Creation (UTPC) capabilities, optimized via Progressive Specialization Training (PST) and Virtual Reinforcement Learning (VRL) with LtrReward in VisGenEnv. | LLM role: agentic_model
This paper introduces VisionCreator, an agent trained via 'Virtual Reinforcement Learning' (VRL) where tool outputs and logic are simulated to train long-horizon planning policies without incurring expensive real-world execution costs. They employ a 'Plan-Driven Reward' model (combining LLM-based plan verification with rule-based execution checks) and prove theoretical bounds for the sim-to-real transfer, achieving performance superior to GPT-5 on visual tasks.
2026-03-03 | University of Minnesota-Twin Cities Method: Multi-agent framework with rubric-based agentic reinforcement learning (GRPO) | LLM role: decomposition_guide, code_writer, evaluator
StitchCUDA automates end-to-end GPU program generation using a multi-agent framework, but its core contribution is a training recipe that solves reward hacking in code optimization. They decompose expensive multi-turn agentic RL into single-turn 'atomic skills' (generation vs. refinement) and use GRPO with an LLM-evaluated 'Rubric Reward' (e.g., 'Did you use tiling?') rather than just sparse outcome metrics. This prevents the model from gaming the system (e.g., wrapping PyTorch code) and forces actual optimization behavior.
2026-03-04 | Nanjing University, Suzhou Laboratory, Shanghai Artificial Intelligence Laboratory Method: Neuro-symbolic framework integrating Sparse Monte Carlo Tree Search (MCTS) with Sibling-Aware Expansion, Memory-Driven Global Planning and a Differentiable Physics Engine for continuous ratio optimization. | LLM role: semantic_generator
Chen et al. introduce a neuro-symbolic MCTS framework for mixed discrete-continuous optimization, applying it to solvent design. They solve the LLM context bottleneck via 'Sparse State Storage' (storing only state abstractions and reconstructing paths on-demand) and fix mode collapse using 'Sibling-Aware Expansion' (conditioning the generator on sibling nodes to force orthogonality).
2026-03-03 | DoorDash, WithMetis.ai Method: Prompt-level optimization using GEPA and MAMUT GEPA | LLM role: evaluator, evolutionary_search, decomposition_guide, user_simulator
This paper presents a production-grade framework for optimizing multi-agent systems by jointly evolving prompt bundles (MAMUT) rather than optimizing agents in isolation. They validate this on a grocery assistant, showing that system-level optimization outperforms local sub-agent optimization by ~7% because it captures coordination dynamics (e.g., context passing) that local metrics miss. The most stealable insight is their 'Judge Calibration' loop: they use evolutionary search (GEPA) to optimize the evaluator's prompt to match human labels (91.4% agreement) before using that judge to optimize the agents.
2026-03-03 | Shanghai Jiao Tong University, National University of Singapore Method: Hierarchical spatial-temporal backbone multiplexing with unified PEFT representations, dynamic programming for task fusion, priority-based subgraph scheduling and chunk-based data alignment | LLM role: subject_of_optimization
MuxTune introduces a hierarchical scheduler for multi-tenant PEFT that uses Dynamic Programming to optimally fuse tasks (spatial batching) or interleave them (temporal multiplexing) based on a pipeline cost model. Empirical results on H100s show up to 5x throughput gains over NeMo and S-LoRA, validated by ablation studies.
2026-02-26 | City University of Hong Kong, Southern University of Science and Technology Method: Adaptive Iterated Local Search (AILS) with LLM-driven Evolutionary Computation for Automatic Heuristic Design (AHD) of ruin heuristics | LLM role: heuristic_generator
This paper integrates LLM-driven evolutionary search into the AILS framework to evolve 'ruin' heuristics for CVRP, employing a Chain-of-Thought 'voting' mechanism to filter out poor heuristics before expensive evaluation. The results are empirically strong: they claim 8 new Best-Known Solutions on the CVRPLib large-scale benchmark, outperforming HGS and AILS-II. This is a direct proof-of-concept that LLM-evolved components can beat hand-crafted SOTA on hard OR instances.
2026-02-26 | Alibaba Group, Harbin Institute of Technology, Shenzhen Method: Test-time rectify-or-reject pruning framework with retrieval-augmented rectifier, failure-driven indicator pool and dual-stage deduplication | LLM role: rectifier, teacher, deduplicator, reasoning_engine
Wang et al. propose a test-time 'firewall' for multi-agent systems that intercepts messages and validates them against a retrieved set of error patterns (mined from offline failure trajectories). They achieve ~6% accuracy gains on math benchmarks by iteratively rectifying or pruning erroneous outputs before they propagate. This effectively turns the 'graveyard' of failed runs into a persistent memory that improves sample efficiency.
2026-02-25 | DeepSeek-AI, Peking University, Tsinghua University Method: Dual-path KV-Cache loading architecture with CNIC-centric traffic management and adaptive request scheduling | LLM role: none
Wu et al. introduce DualPath, a system that breaks the storage I/O bottleneck in agentic inference by utilizing idle decode-node bandwidth to load KV-cache and transferring it to prefill nodes via RDMA.
2026-02-25 | Caltech Method: Strategically Risk-Averse Policy Optimization (SRPO) based on Risk-Averse Quantal Response Equilibria (RQE) | LLM role: collaborative_agent
Qu et al. introduce Strategically Risk-Averse Policy Optimization (SRPO), which trains agents against a 'constrained adversary' that minimizes their reward within a KL-divergence bound of the partner's current policy. Theoretical results prove this objective eliminates free-riding equilibria and experiments on GSM8K multi-agent debate show it prevents 'lazy' agreement, improving joint accuracy by up to 19% when pairing heterogeneous LLMs (e.g., 0.6B with 4B). The key takeaway is that robustness to partner deviation—enforced via this specific adversarial objective—is a more principled way to fix lazy agent behavior than prompt engineering or simple dropout.
2026-02-22 | Meituan, Tsinghua University, Fudan University, Peking University Method: Dual-pronged optimization framework (DynaMO) combining variance-minimizing dynamic rollout allocation and gradient-aware advantage modulation for GRPO-based policy optimization | LLM role: policy_model
DynaMO introduces a dual-pronged optimization for RLVR: a dynamic rollout allocation strategy that prioritizes problems with high gradient variance (proxied by Bernoulli variance of success/failure) and a gradient modulation technique to stabilize updates. The results are strong (+11.8% Pass@1 over GRPO on Qwen-7B) and backed by clear ablations.
2026-02-25 | University of California, Los Angeles, University of Wisconsin–Madison Method: Stable Agentic Multi-turn Policy Optimization (SAMPO) integrating sequence-level clipping, fine-grained advantage estimation and dynamic filtering | LLM role: policy
The authors dissect why standard RL (GRPO/PPO) fails in multi-turn agentic tasks, identifying that token-level importance sampling (IS) clipping allows negative-advantage outliers to destabilize training. They propose SAMPO, which enforces sequence-level clipping and integrates fine-grained step-level advantages (similar to process rewards) to stabilize learning. The results are rigorous, showing a jump from ~50% to 92% success on ALFWorld by fixing the gradient update mechanics rather than just prompt engineering.
2026-02-25 | Georgia Institute of Technology, Korea University, Sogang University, Ewha Womans University Method: Multi-Agent-based Electrocatalyst Search Through Reasoning and Optimization (MAESTRO) framework using LLM agents and a Machine Learning Force Field (MLFF) surrogate model | LLM role: evolutionary_search
Mok et al. propose MAESTRO, a multi-agent LLM framework for optimizing single-atom catalysts that explicitly separates search into exploration and exploitation phases, bridged by a textual 'Exploration Report.' Results are validated against high-fidelity DFT calculations, showing the system learns to break theoretical scaling relations via in-context learning, outperforming memory-less baselines.
2026-02-23 | Cornell University, University of Maryl and College Park Method: Differentiable Scheduling Optimization via Gaussian Reparameterization with Augmented Lagrangian Method | LLM role: none
GauS replaces the standard categorical (Gumbel-Softmax) relaxation in differentiable scheduling with Gaussian variables defined by mean and variance, reducing parameter space from O(N*D) to O(N). Results are strong: it scales to 57k nodes where previous differentiable methods OOM and exact solvers timeout, while maintaining near-100% GPU utilization. The key takeaway is a specific modeling technique: using Gaussian distributions to represent discrete ordinal values (like time steps) naturally captures temporal proximity and provides smoother gradients than categorical buckets.
2026-02-19 | Shanghai Jiao Tong University, Meituan Method: Reinforcement learning (GRPO) optimized multi-agent system with LLM-based orchestrator agent for dynamic layered DAG topology generation | LLM role: orchestrator
AgentConductor trains an LLM orchestrator via GRPO to dynamically generate and refine layered DAG interaction topologies (output as YAML) for code generation, optimizing for both correctness and token efficiency. The key innovation is a multi-objective reward that combines execution correctness with a 'difficulty-aware density penalty,' forcing the model to learn a policy that scales graph complexity with task hardness. Results are strong, showing ~14% gains on APPS while reducing token costs.
2026-02-20 | Lehigh University Method: APEMO (Affect-aware Peak-End Modulation for Orchestration), a runtime scheduling layer that reallocates reasoning effort and repair across a trajectory under fixed computational budgets by operationalizing temporal-affective signals. | LLM role: agents_being_orchestrated
Shi et al. introduce APEMO, a runtime orchestration layer that monitors agent trajectories for behavioral instability (e.g., repetition, drift) and dynamically reallocates a fixed compute budget to 'repair' these segments rather than spreading compute uniformly. The results are statistically rigorous, using bootstrap CIs to demonstrate significant improvements in trajectory robustness and completion rates without model retraining.
2026-02-18 | Korea National Open University Method: Task-adaptive topology routing algorithm based on DAG structural properties (parallelism width, critical path depth, coupling density) combined with an adaptive synthesis protocol | LLM role: decomposition_guide, executor, arbiter, synthesizer
AdaptOrch introduces a control layer that dynamically routes tasks to one of four agent topologies (Parallel, Sequential, Hierarchical, Hybrid) by analyzing the task's dependency graph properties (parallelism width, coupling density). The results are strong and credible, showing a 9.8% improvement on SWE-bench over single-model baselines and significantly outperforming static multi-agent architectures like standard MoA. The most valuable takeaway is the Topology Routing Algorithm: a linear-time heuristic that maps DAG structure to optimal agent coordination patterns.
2026-02-16 | University of Cambridge, Peking University, Shanghai Jiao Tong University, Ant Group, Southeast University Method: Adaptive routing and prefill reordering for online scheduling, combined with an Integer Linear Programming (ILP) based offline deployment planner | LLM role: none
AMPD introduces a disaggregated serving framework tailored for multi-round LLM agents, utilizing an offline ILP solver to optimize resource allocation (TP/DP configurations) and an online adaptive routing mechanism to handle incremental prefill tasks. The results are strong, showing 67-340% improvements in SLO attainment over vLLM and NVIDIA Dynamo by dynamically routing incremental prefill to decode workers when slack exists. 5) for partitioning prefill/decode resources under global GPU constraints and the insight that multi-agent workflows create a unique 'incremental prefill' bottleneck that standard disaggregation handles poorly.
Same pipeline, pointed at whatever you're working on — with relevance scored to how you read.
Request access →