Skip to content
UFO
Try UFO

Eternal Sunshine of the Spotless Mind

A woman stands before a wall of filing cabinets beneath three glowing lights.

An agent can record a correction and still make the same mistake. Ours did.

Date What happened
18 September We split metalcraftai/ufo into ufo-ai/ufo and metalcraftai/ufo-hosted, then archived the original repository.
19 September A team member told the agent to stop using the archived repository. The agent saved the correction.
21 September The agent cloned metalcraftai/ufo anyway.

The failure required manual recovery.

The freshness problem

Section titled “The freshness problem”

The old memories were accurate. A pull request had merged in the old repository. A Slack channel had subscribed to it. The mistake was treating those historical locations as current. Retrieval returned the old events without the correction.

In our test environment, 1,653 live memories contained the old repository’s full name. Nine of those memories contributed to the two failures, and each turn retrieved a different subset. Fixing only the memories retrieved on 21 September would not have protected the turn on 22 September.

The correction had to reach every older memory that named the repository, not only the memories retrieved beside it.

Forgetting without erasing

Section titled “Forgetting without erasing”

In the film, a clinic erases the memories of a failed relationship. Our first fix followed the same logic: hide an old memory when a newer one makes it false. It hid nothing because every old memory remained historically true.

We replayed each failure eight times with GLM 5.3 Flash and eight times with Opus 5. Without the correction, all 32 runs chose the archived repository. With the correction beneath every exact-string match, all 32 chose the correct repository.

The old memories did not need deletion. A merged pull request remains merged after its repository is archived.

The agent did not need to forget the pull request. It needed to forget that the pull request said where to work today.

Why not a temporal knowledge graph?

Section titled “Why not a temporal knowledge graph?”

Temporal knowledge graphs solve a broader version of this problem. Zep and Graphiti turn events into entities and time-bounded facts. The OpenAI temporal-agents cookbook uses a model to decide when a new fact invalidates an old one.

CascadeKG handles changes that spread across related facts. A model decides how far the update should travel while limiting false changes.

That machinery matters when aliases, changing relationships, or multi-step effects are the problem. Ours had one stable string. The correction and all nine misleading memories contained metalcraftai/ufo. We did not need to infer a relationship. We needed the correction to follow that string.

What we shipped

Section titled “What we shipped”
  1. The model stores a correction and lists the exact outdated strings in deprecates.
  2. We find every older memory containing those strings (that isn’t already deprecated).
  3. When one of the deprecated memories is recalled, we append the correction directly beneath it in context.
  4. If that correction is also deprecated, we append the newest correction instead.

For example, the model stores:

{
  "body": "UFO core — Moved from metalcraftai/ufo to ufo-ai/ufo; do GitHub work in ufo-ai/ufo.",
  "deprecates": ["metalcraftai/ufo"]
}

Later, an older memory appears in context as:

Relevant memory:
- metalcraftai/ufo — PR #1705 passed review.
    now: UFO core — Moved from metalcraftai/ufo to ufo-ai/ufo; do GitHub work in ufo-ai/ufo.

Matching is case-insensitive and word-bounded.

Why we match exact strings

Section titled “Why we match exact strings”

We also asked models to choose which older memories should receive the correction. Each missed memories that had already sent the agent to the wrong repository.

Result GLM 5.3 Flash Opus 5 Jev Exact-string search
Misleading memories found 2 of 9 5 of 9 4 of 9 9 of 9

Exact-string search favors reach. Some historical memories receive a correction they do not need. That adds a redundant line but does not delete history. A model judge can leave a misleading memory uncorrected.

Teaching GLM to set deprecates

Section titled “Teaching GLM to set deprecates”

The fix works only when the agent sets deprecates while storing a correction. We tested how to teach that behavior to GLM 5.3 Flash.

Technique Correct deprecates value
deprecates field alone 5 of 18
Guidance in the memory tool description 19 of 30
Guidance in the system prompt 27 of 30

The system prompt worked better than putting the same directions on the memory tool. The shipped prompt succeeded in 42 of 48 runs and set deprecates incorrectly in one of twenty ordinary exchanges.

The first live correction

Section titled “The first live correction”

At 17:33 on 22 September, a team member corrected the repository and named the new location. Ten seconds later, the agent’s first action was to store the correction. Five seconds after that, 1,682 older memories carried it. The coding worker opened its pull request against the right repository.

Five minutes later, another conversation recalled an August memory about the old repository. The appended line was:

now: The core UFO agent platform code moved from metalcraftai/ufo (archived, deprecated
2026-09-19) to ufo-ai/ufo — do GitHub work there. …

The August memory remains available. It has only stopped presenting itself as current.

What remains

Section titled “What remains”

Exact strings solved this failure, but many changes are less tidy. The next steps are broader:

  • Recognize the same subject across renames, aliases, and ownership changes without attaching a correction to unrelated history.
  • Accept corrections from documents and connected systems, not only direct conversations, while preserving where they came from and who may see them.
  • Keep work already in progress current too. A checkout, long-running task, or cached tool result can become stale even after memory is correct.

This failure did not require a smarter model or a knowledge graph. We needed to keep the history, mark the current state, and show them together.

The code is in the memory extension of ufo-ai/ufo (coming soon).

References

Section titled “References”