RI

#1From Soliloquy to Agora: Memory-Enhanced LLM Agents with Decentralized Debate for Optimization Modeling

2026-04-28 | Tsinghua University, University of Chicago Booth School of Business, Shanghai Jiao Tong University, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen) |2604.25847

This paper introduces Agora-Opt, a multi-agent framework for optimization modeling that combines decentralized debate across heterogeneous LLMs with a read-write memory bank. The results are backed by strong empirical evidence, achieving state-of-the-art Pass@1 accuracy (84.6%) across 7 OR benchmarks and outperforming both frontier zero-shot models and fine-tuned OR models. The key insight is that decentralized debate, where consensus is driven by solver-verified endpoints rather than a centralized LLM judge, can synthesize correct formulations even when all initial agent proposals are flawed. Furthermore, storing the trajectories of how these disagreements are resolved in a dedicated 'debate memory' allows the system to continuously improve its collaborative reasoning without parameter updates.

#1A2DEPT: Large Language Model-Driven Automated Algorithm Design via Evolutionary Program Trees

2026-04-27 | National University of Defense Technology, University of Electronic Science and Technology of China, Academy of Military Medical Sciences |2604.24043

A2DEPT advances LLM-driven algorithm design by evolving complete, open-ended solver programs via a tree-structured search with hierarchical operators and an automated dependency repair loop. The results are rigorously backed by empirical data, showing a 9.8% reduction in the mean normalized optimality gap relative to state-of-the-art baselines (FunSearch, EoH, ReEvo) across standard combinatorial optimization benchmarks like CVRP and FJSP. The key insight is the decoupling of mutations into hierarchical operators (macro-mutation for workflow restructuring vs. micro-tuning for local refinement) paired with a closed-loop dependency repair mechanism, which allows the LLM to safely rewrite system-level logic without permanently breaking executability. This is highly relevant for LLM evolutionary search research, as it provides a concrete, implementable mechanism to escape the performance ceiling of fixed algorithmic templates and synthesize entirely novel solver architectures.

#2SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolution

2026-04-27 | The University of Hong Kong, City University of Hong Kong, JIUTIAN Research, China Mobile |2604.24372

SeaEvo augments LLM-driven evolutionary search by maintaining a persistent, population-level memory of natural-language strategy descriptions, clustering them to ensure diverse parent selection and tracking strategy saturation. Backed by strong empirical results, it achieves up to a 66% improvement on systems optimization tasks (like GPU model placement) over state-of-the-art backbones like ShinkaEvolve, while reducing cumulative API costs. The key insight is the Stratified Experience Retrieval (SER) module, which clusters candidate strategies using text embeddings and selects parents based on behavioral complementarity rather than just fitness, preventing premature convergence on syntactic variants of the same flawed idea.

#3Beyond the Training Distribution: Mapping Generalization Boundaries in Neural Program Synthesis

2026-04-30 | Friedrich Schiller University Jena |2604.27551

Voigt et al. evaluate the out-of-distribution generalization limits of transformers in program synthesis by mapping arithmetic programs into continuous syntactic and semantic manifolds. Backed by rigorous evaluation on millions of programs, they demonstrate that while transformers can extrapolate semantically using known syntax, they suffer a massive >30% performance drop when forced to extrapolate to novel syntactic structures. The key insight is that pure autoregressive scaling yields strictly log-linear returns for structural extrapolation, exposing a fundamental bottleneck in LLMs that cannot be overcome by compute alone. This is highly relevant for LLM evolutionary search, as it provides empirical proof that evolutionary algorithms are strictly necessary to navigate beyond the learned data manifold. Furthermore, the community could directly adopt their dual-manifold projection technique to measure and enforce population diversity during evolutionary search.

#1CacheFlow: Efficient LLM Serving with 3D-Parallel KV Cache Restoration

2026-04-28 | University of Illinois Urbana-Champaign, National University of Singapore |2604.25080

CacheFlow optimizes KV cache restoration in long-context LLM serving by formulating it as a 3D-parallel scheduling problem (token, layer, GPU) that overlaps recomputation and I/O transfer. The results are backed by strong empirical numbers, showing a 10%-62% reduction in Time-To-First-Token (TTFT) compared to state-of-the-art frameworks like vLLM and SGLang across diverse hardware and bandwidth conditions. The key insight is that the optimal balance between recomputation (which scales quadratically) and I/O loading (which scales linearly but is bandwidth-bound) can be achieved using a batch-aware two-pointer scheduler that prioritizes I/O for requests with the longest remaining lengths to restore.