The Best ICML Papers of 2026, Simply Explained

The Best ICML Papers of 2026, Simply Explained

AI, But Simple Issue #113

2025 was the year diffusion models moved out of image generation and into language and the year RL from verifiable rewards became the default way to finish training an LLM.

We see this trend seep into 2026, especially at ICML. ICML 2026 recently ran from July 6-11, receiving 23,918 reviewed submissions and accepting 6,352 papers.

Today, we’ll walk through 4 key papers: the two Outstanding Papers, plus 2 honorable mentions that have attracted large research attention.

Papers Covered

1. The Flexibility Trap (Random Generation Order Makes DLMs Reason Worse)

2. High-accuracy sampling for diffusion models

3. How much can language models memorize?

4. And more…

What's Helpful to Know

• Diffusion language model (DLM/dLLM)

  • An LLM that uses diffusion for generation rather than an autoregressive approach.

  • They can either iteratively refine tokens or embeddings, generating portions of the full response at each denoising step.

• Pass@k

  • Sample k answers; a problem is passed if any answer is correct. As k grows, Pass@k measures how much of the solution space a model can reach.

• RLVR

  • Reinforcement learning from verifiable rewards, where the reward comes from an automatic check, such as running unit tests.

• Density (p(x))

  • The density p(x) is the probability density function (PDF) over the input data distribution. Higher values are assigned to samples more likely under the data.

Random Generation Order Makes DLMs Reason Worse

A normal LLM generates tokens autoregressively, meaning left to right, one token per forward pass.

A diffusion language model can take one of two approaches: diffuse on tokens (unmasking over time) or diffuse on embeddings (a continuous diffusion, then round to the nearest token).

A diffusive approach has 2 main benefits.