RI

#1Harnessing Agentic Evolution

2026-05-13 | Tsinghua University, Université de Montréal & Mila, The Hong Kong University of Science and Technology (Guangzhou), Nanyang Technological University, Shanghai Jiao Tong University, Singapore University of Technology and Design, DeepWisdom |2605.13821

Zhang et al. introduce AEVO, a framework that formulates LLM evolutionary search as an interactive environment where a meta-agent edits the underlying search procedure or agent context rather than directly generating candidate solutions. The results are backed by strong empirical evidence, achieving state-of-the-art performance on Anthropic's Kernel optimization task (1138 cycles) and outperforming baselines like OpenEvolve and HyperAgents across multiple open-ended discovery and reasoning benchmarks. The key insight is separating the candidate generation loop from a meta-editing phase, allowing the system to dynamically revise its own selection rules, feedback summaries and memory structures when search plateaus.

#2Budget-Efficient Automatic Algorithm Design via Code Graph

2026-05-11 | Massachusetts Institute of Technology, University of California, Berkeley |2605.10598

This paper proposes a graph-based search framework for LLM-driven automatic algorithm design where algorithms are represented as paths in a directed acyclic graph (DAG) of code blocks, allowing the LLM to generate composable corrections rather than full rewrites. The approach is backed by strong empirical results on TSP, LRP and BRP, demonstrating up to 22% smaller optimality gaps than full-algorithm baselines at equal token budgets. The key insight is the use of a Random Forest surrogate model combined with Shapley values to perform correction-level credit assignment, effectively isolating the impact of specific code edits from the overall algorithm fitness. This is highly relevant for LLM evolutionary search; the DAG representation and Shapley-based credit assignment provide a concrete architectural blueprint for drastically improving sample efficiency, search signal quality and observability in automated heuristic discovery.

#3Teacher-Aware Evolution of Heuristic Programs from Learned Optimization Policies

2026-05-11 | Shanghai Jiao Tong University, Shenzhen Technology University, Shanghai Polytechnic University |2605.10634

This paper introduces a teacher-aware evolutionary framework that uses pre-trained neural optimization policies to provide step-by-step behavioral feedback to guide the LLM-based evolution of symbolic heuristics. The results are backed by strong empirical numbers, showing consistent improvements over standard performance-driven baselines (EoH, ReEvo) on scheduling, routing and graph optimization benchmarks, while maintaining the fast inference of static code. The key insight is that instead of relying solely on delayed endpoint performance, one can query a black-box neural policy on the specific states visited by a candidate heuristic and use the behavioral discrepancies (top-1 action agreement) to prompt targeted LLM revisions. This is highly relevant for research in LLM evolutionary search, as it provides a concrete, actionable implementation of a process reward model that directly improves search guidance and sample efficiency.

#4RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards

2026-05-11 | Google Cloud AI Research, University of Illinois Urbana-Champaign |2605.10899

Li et al. introduce RubricEM, an RL framework for long-horizon LLM agents that uses dynamically generated rubrics to structure trajectories, assign stage-wise process rewards via Stage-Structured GRPO and train a reflection memory bank for cross-episode transfer. The results are rigorously backed by numbers, showing their 8B model outperforming 30B+ open models and approaching proprietary systems on four long-form research benchmarks. The key insight is that rubrics can serve as a shared interface across the entire RL loop, acting simultaneously as structural priors for planning, dynamic process reward models for intermediate credit assignment and structured memory for experience reuse.

#5LEVI: Stronger Search Architectures Can Substitute for Larger LLMs in Evolutionary Search

2026-05-10 | Independent Researcher |2605.09764

LEVI introduces a cost-efficient framework for LLM-guided evolutionary search that replaces expensive frontier-model reliance with a stronger search architecture. The authors demonstrate that by combining a CVT-MAP-Elites archive, role-aware LLM routing (using small models for local edits and large models only for structural paradigm shifts) and rank-preserving proxy benchmarks, they can outperform existing frameworks like ShinkaEvolve and GEPA on systems-research and prompt-optimization tasks at 3.3 to 6.7x lower cost. The results are rigorously backed by empirical cost and performance metrics across 11 benchmarks.

#6Evolutionary Ensemble of Agents

2026-05-09 | National University of Singapore |2605.09018

Yu and Yang introduce Evolutionary Ensemble (EvE), a framework that co-evolves functional code solvers alongside the guidance and skills of the coding agents that produce them. The results are backed by rigorous ablations on a complex positional-encoding design task, demonstrating that continuous agent evolution prevents the phase mismatch and stagnation seen with static or frozen agents. The key insight is the dual-population credit assignment mechanism, which evaluates and evolves agent guidance based strictly on the downstream performance gains of the code they generate in a synchronous race.

#7FlashEvolve: Accelerating Agent Self-Evolution with Asynchronous Stage Orchestration

2026-05-08 | University of California, San Diego, Georgia Institute of Technology |2605.08520

FlashEvolve accelerates LLM-based agent evolution by replacing synchronized execution stages with asynchronous workers and queues, allowing proposal, rollout and evaluation to overlap. Backed by strong empirical numbers, it achieves up to 4.9x higher proposal throughput and reaches higher validation scores significantly faster on benchmarks like IFBench and HotpotQA compared to synchronous baselines. The key insight is 'Reflective Async' for handling data staleness: unlike continuous weights in RL, stale language artifacts (prompts or code) are inspectable and can be semantically patched by the LLM to incorporate recent pool updates, preventing wasted compute. This is highly critical for scaling LLM evolutionary search and optimizing LLM serving scheduling, providing a concrete architectural blueprint to drastically reduce the wall-clock time of evolutionary discovery pipelines.

#8CVEvolve: Autonomous Algorithm Discovery for Unstructured Scientific Data Processing

2026-05-12 | Argonne National Laboratory |2605.11359

CVEvolve is an LLM-based evolutionary search framework that autonomously discovers data-processing algorithms using a generate-tune-evolve loop, lineage-aware sampling and SQL-backed persistent memory. The results are backed by strong empirical evidence, showing significant improvements over baselines, such as reducing X-ray image registration error from 0.98 to 0.12. The key insight is the use of a relational SQL database to manage search history and lineage (avoiding context bloat) combined with a dedicated holdout-test agent to prevent over-optimization on small development sets.

#9TacoMAS: Test-Time Co-Evolution of Topology and Capability in LLM-based Multi-Agent Systems

2026-05-10 | Carnegie Mellon University, University of Science and Technology of China, National University of Singapore, Shanghai AI Lab |2605.09539

TacoMAS introduces a test-time co-evolution framework for LLM multi-agent systems that updates agent capabilities rapidly (via memory refinement) and communication topology slowly (via birth-death operations and edge edits). The results are backed by strong empirical evidence, achieving an average 13.3% accuracy improvement over 20 baselines across four diverse benchmarks. The key insight is the separation of evolution into two time scales—fast capability updates to handle immediate subtasks and slow topology updates to preserve coordination stability—which prevents the system from diverging during test-time adaptation. Furthermore, using a meta-judge to assign contribution scores to individual agents effectively acts as a process reward model to guide the replicator dynamics.

#10HMACE: Heterogeneous Multi-Agent Collaborative Evolution for Combinatorial Optimization

2026-05-08 | Westlake University |2605.07214

HMACE decomposes LLM-based heuristic generation into a multi-agent workflow (Proposer, Generator, Evaluator, Reflector) to automate heuristic design for combinatorial optimization problems. Backed by strong empirical numbers, it achieves lower optimality gaps on TSP and Online BPP than SOTA baselines like EoH and CORAL, while using significantly fewer tokens (e.g., 0.13M vs 0.24M for TSP). The key insight is the integration of a CVT-MAP-Elites archive as a 'Reflector' memory to enable behavior-aware retrieval of past heuristics, which, combined with a lightweight deterministic pre-filter, drastically improves the sample efficiency of the evolutionary search.

#11CoupleEvo: Evolving Heuristics for Coupled Optimization Problems Using Large Language Models

2026-05-07 | Karlsruhe Institute of Technology |2605.06341

CoupleEvo extends LLM-driven heuristic evolution to coupled optimization problems by generating interacting Large Neighborhood Search (LNS) destroy operators using sequential, iterative, or integrated prompting strategies. Results are backed by empirical evaluations on the Inventory Routing Problem and a multi-robot pre-marshalling problem, showing the approach outperforms classical matheuristics but slightly trails highly specialized state-of-the-art solvers. The key insight is that decomposing the evolution of interacting heuristics into sequential or iterative phases yields significantly better and more stable convergence than integrated generation, which overwhelms the LLM's search complexity.

#12SkillOS: Learning Skill Curation for Self-Evolving Agents

2026-05-07 | Google Cloud AI Research, Massachusetts Institute of Technology, University of Illinois Urbana-Champaign |2605.06614

SkillOS trains a dedicated skill curator model using Grouped Reward Policy Optimization (GRPO) to insert, update and delete reusable skills in a persistent repository for a frozen agent executor. The results are backed by strong empirical evidence, showing up to a 9.8% relative performance improvement and a 6.0% reduction in interaction steps on ALFWorld, WebShop and math reasoning benchmarks compared to strong memory baselines. The key insight is the training recipe for delayed memory feedback: grouping related tasks sequentially so that skills extracted from early tasks are explicitly rewarded based on their utility in solving later tasks, augmented by intermediate process rewards for skill conciseness and quality. This is highly relevant for LLM evolutionary search and multi-agent optimization, as it provides a scalable, RL-driven mechanism to maintain a persistent, cross-run memory of successful heuristics or strategies without relying on static rules.

#13Intentmaking and Sensemaking: Human Interaction with AI-Guided Mathematical Discovery

2026-05-07 | Google DeepMind |2605.05921

Bäuerle et al. present a qualitative user study and interactive dashboard for AlphaEvolve, formalizing the iterative human-AI workflow into 'intentmaking' (problem definition) and 'sensemaking' (result interpretation). While results are qualitative (based on observations of 11 mathematicians running 2,300+ experiments) rather than algorithmic benchmarks, the operational insights are highly valuable. The key actionable insight is the implementation of a 'critique agent' and a low-cost test stage during experiment setup, which helps users catch underspecified constraints and prevent evolutionary reward hacking before launching expensive, multi-day compute jobs.

#14Graph-Grounded Optimization: Rao-Family Metaheuristics, Classical OR and SLM-Driven Formulation over Knowledge Graphs

2026-05-13 | Samyama.ai |2605.12204

This paper introduces a graph-grounded optimization paradigm where operations research problem components are sourced directly from property knowledge graphs via Cypher queries, rather than from lossy natural language descriptions. The results are backed by strong empirical numbers across 7 real-world KGs, demonstrating that while classical solvers and metaheuristics succeed, frontier SLMs (including the MILP-tuned OptiMind) fail almost completely at zero-shot formulation. The key insight is the identification of a 'data-binding pathology': LLMs generate syntactically valid optimization code that silently fails or returns trivial solutions because it binds to hallucinated data structures instead of the actual operational database.

#1Attention Once Is All You Need: Efficient Streaming Inference with Stateful Transformers

2026-05-13 | LayerScale, Inc. |2605.13784

Norgren introduces a stateful transformer inference architecture that decouples data ingestion from query processing, maintaining a persistent KV cache to achieve constant-time query latency for streaming workloads. The approach is backed by strong empirical results, demonstrating a 2.4x to 5.9x speedup over state-of-the-art engines like vLLM and SGLang on streaming benchmarks while maintaining approximately 43ms latency regardless of context size. The key insight is the use of Flash Queries, which utilize idle GPU cycles between data arrivals to pre-compute answers to registered queries against the evolving context, effectively pushing user-visible latency toward zero.

#2FATE: Future-State-Aware Scheduling for Heterogeneous LLM Workflows

2026-05-08 | University of Science and Technology of China |2605.07238

FATE introduces a future-state-aware scheduler for heterogeneous LLM workflows (e.g., multi-agent DAGs) that uses a CP-SAT solver to optimize both immediate execution costs and downstream state preservation. The results are backed by solid empirical evidence, showing an 8.9% reduction in normalized makespan and an 8.8% reduction in P95 latency over the strongest baseline on a WfCommons-derived benchmark. The key insight is that LLM workflow scheduling cannot be myopic; it must explicitly model how current placement decisions alter future execution states, specifically regarding model residency, cross-device transfer and KV-cache/prefix reuse. This is highly relevant to the team's work on OR formulations for LLM serving scheduling, as the rolling-horizon constraint programming formulation and state-conditional cost estimators offer a directly implementable approach for optimizing multi-agent execution on GPU clusters.

#3Requests of a Feather Must Flock Together: Batch Size vs. Prefix Homogeneity in LLM Inference

2026-05-07 | Indian Institute of Technology Bombay |2605.06046

Feather introduces a prefix-aware LLM inference scheduler that uses a novel Chunked Hash Tree for fast prefix detection and a reinforcement learning policy to dynamically balance batch size against prefix homogeneity. The results are strongly backed by empirical evidence, demonstrating 2-10x higher end-to-end throughput compared to vLLM and SGLang baselines while reducing CPU scheduling overhead by up to 1000x. The key insight is a paradigm shift for inference batching: maximizing batch size is sub-optimal for prefix-shared workloads, as moderately small, prefix-homogeneous batches achieve higher throughput by maximizing spatial and temporal locality in KV cache accesses.

#4PipeSD: An Efficient Cloud-Edge Collaborative Pipeline Inference Framework with Speculative Decoding

2026-05-13 | Zhejiang University, Northeastern University, University of Surrey, Zhongguancun Institute of Artificial Intelligence |2605.13319

PipeSD accelerates cloud-edge collaborative LLM inference by using dynamic programming to optimally batch and pipeline draft tokens, alongside a Bayesian optimization-tuned dual-threshold mechanism for triggering speculative verification. The results are backed by empirical hardware measurements, demonstrating 1.16x–2.16x speedups and up to 25% energy reduction over baselines like EdgeLLM on standard benchmarks. The key insight is formulating the token-batching decision—balancing communication startup overhead against immediate transmission—as a dynamic programming problem to perfectly overlap edge-side autoregressive generation with network transmission.

#5DisagMoE: Computation-Communication overlapped MoE Training via Disaggregated AF-Pipe Parallelism

2026-05-10 | ByteDance Seed, University of Washington, Cornell University |2605.11005

DisagMoE optimizes large-scale Mixture-of-Experts (MoE) model training by disaggregating attention and feed-forward network (FFN) layers onto separate GPU groups and using a multi-stage pipeline to overlap communication and computation. The results are backed by strong empirical evidence, demonstrating up to 1.81x throughput speedups over Megatron-LM and 1.34x over state-of-the-art overlap methods on a 128-GPU H800 cluster. The key insight is the use of a Compute-Communication roofline model, solved via MILP, to asymmetrically allocate GPU and NIC resources based on the distinct arithmetic intensities of different model components (compute-bound attention vs. communication-bound FFN).