RI

#1Execution-Verified Reinforcement Learning for Optimization Modeling

MUST-READ

2026-04-01 | Chinese Academy of Sciences, Nanjing University, Nanjing University of Science and Technology | 2604.00442

M=7 P=8 I=7

EVOM trains LLMs for operations research modeling using execution-verified reinforcement learning (GRPO/DAPO) based solely on solver outcomes, bypassing expensive process-level supervision. The results are backed by solid empirical evaluations on OptiBench, NL4OPT, and IndustryOR, demonstrating that it matches or beats process-supervised SFT (ORLM) and enables zero-shot transfer to new solvers (e.g., Gurobi to OR-Tools). The key takeaway is that outcome-only RL prevents the model from overfitting to solver-specific syntax (a major flaw in SFT), forcing it to learn invariant mathematical structures; additionally, their two-stage cold-start trick (LLM-translate 100 samples -> SFT -> RL) is a highly stealable technique for adapting to new environments. This is highly relevant for our OR-Bench project, and we should consider implementing execution-verified RL baselines and leveraging their cold-start adaptation trick when targeting new solvers in our evolutionary search pipelines.

#1CliffSearch: Structured Agentic Co-Evolution over Theory and Code for Scientific Algorithm Discovery

MUST-READ

2026-04-01 | IBM Research | 2604.01210

M=8 P=8 I=9

CliffSearch is an LLM-based evolutionary framework that co-evolves algorithm theory and code, using specialized agents for crossover, two-path mutation (exploration vs. repair), and explicit reviewer gating. The results are backed by concrete empirical runs on nanoGPT optimizer discovery and transformer hyper-connection search, demonstrating the discovery of genuinely novel geometric routing and optimizer variants rather than trivial hyperparameter tweaks. The single most useful takeaway is the 'reviewer-gated selection' where an LLM explicitly scores candidates on originality and correctness as a hard survival gate before benchmark scores are considered. This is highly relevant for our AlgoEvo project; we should immediately steal the two-path mutation (novelty vs repair) and the originality hard-gate to prevent our populations from converging on unoriginal benchmark-hacking.

#2COvolve: Adversarial Co-Evolution of Large-Language-Model-Generated Policies and Environments via Two-Player Zero-Sum Game

MUST-READ

2026-03-30 | Örebro University | 2603.28386

M=8 P=6 I=9

COvolve uses LLMs to adversarially co-evolve Python code for both environments (tasks) and policies (agents), using Policy Space Response Oracles (PSRO) to compute a mixed-strategy Nash equilibrium (MSNE) that prevents catastrophic forgetting. The results are backed by solid empirical data across MiniGrid, PyGame, and CARLA, demonstrating that the MSNE approach maintains robust performance across a growing historical archive of environments much better than greedy retention. WHAT WE LEARNED: The use of an empirical payoff matrix and MSNE to evaluate new code against an archive of past opponents/environments is a powerful memory and evaluation mechanism for LLM evolution. This matters immensely for us: we can steal this exact architecture for AlgoEvo or EvoCut by co-evolving hard OR instances (environments) alongside our solver heuristics (policies), using MSNE to ensure our heuristics do not overfit to specific instance types.

#3Evolutionary Discovery of Reinforcement Learning Algorithms via Large Language Models

MUST-READ

2026-03-30 | Machine Perception and Interaction Lab, Örebro University, Sweden | 2603.28416

M=8 P=7 I=8

This paper evolves executable reinforcement learning update rules using LLMs as macro-mutation and crossover operators, explicitly forbidding standard RL mechanisms to force the discovery of novel algorithms. The results are backed by solid empirical evaluations on Gymnasium benchmarks, showing the evolved algorithms match or beat standard baselines like PPO and SAC on several tasks, though they struggle on a few complex continuous control environments. The single most useful takeaway for us is their diversity-aware crossover, which uses normalized Levenshtein distance to penalize recombining near-duplicate parents, alongside a post-evolution step where the LLM proposes bounds for internal scalar parameters before a final sweep. We should immediately test the Levenshtein-penalized crossover in AlgoEvo to prevent diversity collapse, and adopt the LLM-bounded HPO step to ensure we aren't discarding good heuristics simply because of bad default scalar parameters.

#1Rocks, Pebbles and Sand: Modality-aware Scheduling for Multimodal Large Language Model Inference

MUST-READ

2026-03-27 | IMDEA Software Institute, Universidad Politécnica de Madrid | 2603.26498

M=5 P=8 I=7

RPS-Serve introduces a modality-aware scheduler for multimodal LLMs that classifies requests into 'rocks' (video), 'pebbles' (image), and 'sand' (text) based on predicted prefill latency and memory, using dynamic priorities and aging to prevent head-of-line blocking. The results are real and backed by solid systems experiments on vLLM, showing a 78.5% reduction in time-to-first-token for latency-critical text requests compared to FCFS and EDF baselines. The core takeaway is that multimodal workloads completely break standard text-only LLM serving assumptions because video/image prefill times and KV-cache footprints are orders of magnitude larger. This matters directly for our GPUSched project: any OR formulation we develop for LLM inference scheduling must now explicitly model this extreme multimodal variance rather than assuming homogeneous text workloads.

#2DFLOP: A Data-driven Framework for Multimodal LLM Training Pipeline Optimization

DISCUSS

2026-03-26 | Microsoft Gray Systems Lab, SK Telecom, Yonsei University | 2603.25120

M=6 P=5 I=7

DFLOP optimizes distributed 3D parallelism for multimodal LLM training by combining offline profiling with an online ILP-based microbatch scheduler to minimize pipeline bubbles caused by heterogeneous data inputs. The results are real and backed by extensive hardware experiments, showing up to 3.6x throughput improvements over Megatron-LM. The single most useful takeaway for us is their hybrid online scheduling architecture: running an ILP solver asynchronously on the CPU to schedule the next batch while the GPU processes the current one, with a strict time limit that falls back to a fast Longest Processing Time (LPT) heuristic. Although this targets training, this exact asynchronous OR-based scheduling architecture and their adaptive throughput correction mechanism are directly stealable for our GPUSched project on LLM serving optimization.