The Quiet Arrival of Infinite Context

Sep 21, 2026

You probably have not heard that compaction, one of the oldest, most optimized parts of the agent stack is changing. You might have expected a post-training innovation or a technological leap. Then we tried it on Z.ai’s GLM 5.3 Flash.

It worked in our tests. That suggests special post-training may not be needed, but it does not prove it.

The compaction bargain

Compaction has been part of building long-running agents since the early chat APIs. In a nutshell, when the LLM context window fills, the harness generates a summary of the conversation and continues. As models became more capable, windows grew, and agents took on longer tasks. The summary remained the bridge between one window and the next.

A lot of engineering hours went into compaction performance. Better prompts, better handoffs, more rules about what to preserve, and domain specific adaptation. Boris Cherny, Claude Code creator, said “The compaction prompt is probably one of the most optimized prompts in all of Claude Code.” Several startups offer optimized compaction as a service (higher recall, lower costs). Tell-tale signs a technological discontinuity loomed.

A different approach

Now a different approach has appeared to little fanfare. Codex has recently released an experimental context-management mode. Our anecdotal conversations indicate almost no one knows about this. The open-source pi-posthorse project implemented the same broad pattern: save notes, keep the full history searchable and append-only, and start a fresh window. There are rumors of a similar direction for Claude Code.

The extremely ordinariness of the ingredients are the big surprise. Files. Search. Reliable tool calls. A way to reset the window. We had suspected that making this work required post-training: perhaps the model had learned a special way to manage its state across windows – OpenAI might have a competitive advantage in the frontier model tug-of-war. But, we tried this summary-free compaction with GLM 5.3 Flash and Opus 5. The GLM result suggests that this probably does not depend on special post-training for context management. We cannot be sure: maybe the necessary behaviors are not spring-summer 2026 children. Our working hypothesis is that competent tool use may be enough. Models around the later Opus 4 generations may already have had this ability. A capability may have been available for months while AI engineers kept grinding summary prompts. The promise of infinite, lossless agent context has finally arrived.

Alex Zhang likely inspired the change to Codex with Recursive Language Models (RLMs). The full paper, submitted on New Year’s Eve, describes how a model can examine context stored outside its window. The model chooses how to divide that context and makes recursive model calls over selected parts. RLMs and the rollover process described here share the use of external context, but RLMs also use recursive calls to process it.

Zhang’s much more recent post, Language model harnesses are compositional generalizers, helps explain why dynamic workflows like summary-free compaction are so powerful. A harness can divide a large, unfamiliar task into smaller tasks that the model already knows how to handle. His experiments show that RLM training on shorter tasks can transfer to longer tasks and to other domains with similar structure. This is a fundamentally more trainable behavior and thus better than tuning prompts for compaction.

Results for GLM5.3 Flash

What changed in UFO

We have now replaced compaction with posthorse-style context management in UFO, and it is working well in production. Our evaluations also show some large gains: 5× improvements on cost and latency, and equal or better recall. The compaction works equally well across domains – code and work, which have very different semantics. Artifact-heavy work (documents, file reading) are more impacted.

But the larger story is already interesting: the decisions in harness matter to cost, latency, performance – it’s not all frontier model performance. Change to how you organize the agent’s work can unlock capabilities we’ve been waiting for since the very beginning.

Eternal Sunshine of the Spotless Mind