- AI, But Simple
- Posts
- The Best ICML Papers of 2026, Simply Explained
The Best ICML Papers of 2026, Simply Explained
AI, But Simple Issue #113
This week’s PRO issue includes a Colab notebook where we’ll run experiments and validate results from the 5 papers mentioned in the issue.
Please do read the full article before attempting the code tutorial.
As always, the code in this issue can be executed on a normal home PC and does not require a large amount of VRAM.
The monthly interview question bank can be found here. We’re going to be making many improvements in August, so keep an eye out!
These are industry standard questions that are tested across major tech companies, AI research firms, and more. The questions are sorted by difficulty and topic.
Check back frequently for more questions, and let us know what type of questions you would like to see next!
2025 was the year diffusion models moved out of image generation and into language and the year RL from verifiable rewards became the default way to finish training an LLM.
We see this trend seep into 2026, especially at ICML. ICML 2026 recently ran from July 6-11, receiving 23,918 reviewed submissions and accepting 6,352 papers.
Today, we’ll walk through 4 key papers: the two Outstanding Papers, plus 2 honorable mentions that have attracted large research attention.
Papers Covered
The Flexibility Trap (Random Generation Order Makes DLMs Reason Worse)
High-accuracy sampling for diffusion models
How much can language models memorize?
And more…
What's Helpful to Know
Diffusion language model (DLM/dLLM)
An LLM that uses diffusion for generation rather than an autoregressive approach.
They can either iteratively refine tokens or embeddings, generating portions of the full response at each denoising step.
Pass@k
Sample k answers; a problem is passed if any answer is correct. As k grows, Pass@k measures how much of the solution space a model can reach.
RLVR
Reinforcement learning from verifiable rewards, where the reward comes from an automatic check, such as running unit tests.
Density (p(x))
The density p(x) is the probability density function (PDF) over the input data distribution. Higher values are assigned to samples more likely under the data.
Random Generation Order Makes DLMs Reason Worse
Paper: The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models
Ni, Wang, Yue, et al., Tsinghua University and Alibaba, Outstanding ICML 2026 Paper
A normal LLM generates tokens autoregressively, meaning left to right, one token per forward pass.
A diffusion language model can take one of two approaches: diffuse on tokens (unmasking over time) or diffuse on embeddings (a continuous diffusion, then round to the nearest token).
A diffusive approach has 2 main benefits.