Rulebooks That Compile into World Models

When an agent must learn hidden environment rules from interaction, Memento 3 keeps those hypotheses in a revisable rulebook and compiles them into executable code for prediction, replay, and planning.

Read the paper ↗

The short version

What it is

Rulebook plus executable The agent keeps a written account of environment rules alongside code that turns those rules into predictions it can run.
Replay and fidelity checks Before using a proposed model, the agent checks that its code reproduces past observations and that the code matches the written rules.

Why it matters

A written explanation can be plausible but operationally wrong, and code can contain bugs even when the intended rule is right. Keeping both artifacts makes those failure modes easier to separate and lets the agent test repairs against its complete interaction history.

Before getting into the machinery, here is the short version: the agent keeps a written rulebook about how its environment works, then compiles that rulebook into code it can run. When a prediction misses, it checks whether the rule itself needs revision or whether the code failed to implement the rule, and tests the proposed repair against what it has already observed.

Operational predictions: A language model can narrate what it thinks an environment does, but planning needs a concrete answer to “if I take this action, what happens next?” The canonical example is a small interactive world where an action changes a position and the environment reports a new observation. A description can suggest the rule; executable code can be queried for the predicted next observation. The general point is that a planner needs a model it can use, not just a story. The code is still only as good as the rules it implements—this is not a guarantee that compilation makes a hypothesis true.
Persistent semantic memory: The agent keeps its current hypotheses in a natural-language rulebook rather than relying only on the latest prompt or prediction. In that same interactive world, the rulebook might say what an object is, what an action does, and what counts as completing the goal. The rulebook is like a notebook of reusable rules; the executable is the working simulator built from it. The analogy breaks because the simulator can contain implementation mistakes even when the notebook is clear. This separation lets the agent ask whether it misunderstood the environment or merely wrote the code wrong.

So the real question was never just how to make a prediction—it was how to tell those two errors apart. After an action, the agent records the observation and reflects on the mismatch, if there is one. It can keep the pair, revise the rulebook and rebuild the code, or repair the code while keeping the rulebook. That is the intuition behind the observation–revision–compilation–verification loop: the new experience is evidence, and the proposed model has to account for it. What prevents a repair from fixing only the latest miss while breaking an earlier prediction?

The agent replays its recorded actions through the proposed simulator and checks each predicted observation against the actual one. In the interactive-world example, a candidate that predicts the newest room correctly but gets an earlier move wrong is not a clean explanation of the history. Replay is like rerunning a test suite after a code change: it checks the whole known trace, not just the failing test. The important limit is that passing replay means consistency with recorded observations, not truth about situations the agent has not encountered.

There is another check because a program can replay successfully while contradicting the rulebook it is supposed to implement. The agent therefore also assesses whether the executable is faithful to the written rules. In mechanism terms, replay checks behavior against experience; fidelity checks whether code matches its stated specification. Neither check subsumes the other, and fidelity is not the same kind of exact execution test as replay. So when multiple pairs pass both checks, how should the agent choose?

Simplicity is a preference among admissible models, not a replacement for verification: first require that the code match the observed trace and the rulebook, then prefer a less elaborate account when appropriate. I like this separation because it makes clear what the tests establish and what remains a modeling judgment. The broader lesson is not that the shortest rulebook must be right, but that simpler explanations can be preferred only after they meet the behavioral and semantic constraints. What can the system do when one model is not enough to guide exploration? That belongs to the separate population-based exploration treatment, not this module.

Read the deep dive

Why People Wanted It

A language model can describe what it thinks an environment does, but a description alone cannot reliably answer the next operational question: if I take this action from the state I believe I am in, what observation should follow? The agent also receives completion feedback that offers evidence about whether a goal was reached without necessarily disclosing the full goal specification .

So the real question was never just how to write down a plausible explanation. It was how to keep a revisable semantic account of the environment, make that account executable for prediction and planning, and reject versions that fail to explain what has actually happened.

The Core Move

Keep two linked artifacts: a natural-language rulebook that says what the environment means, and executable code that makes those rules operational. Write the persistent hypothesis as $h_t=(\psi_t,\theta_t)$ : the pair is the rulebook and its executable realization.

The interaction history supplies evidence; the rulebook expresses rules meant to generalize beyond that history; and the executable implements those rules for prediction, replay verification, and planning . The language model is frozen: it revises or repairs this external learning state rather than learning by changing its model weights .

Quiz

Why does Memento 3 keep both a natural-language rulebook and an executable program?

They are duplicate copies used only to increase confidence in a prediction.

The rulebook changes the frozen language model's weights, while the program records those changes.

The rulebook expresses hypotheses; the program operationalizes them for prediction and planning.

The program stores the interaction history, while the rulebook is used only to choose actions.

Why the Rulebook Is More Than a Prompt

Now the version for readers who want more of the machinery: the rulebook is persistent semantic memory, not merely an instruction attached to one prediction. It records hypotheses about entities, action semantics, dynamics, goals, and termination conditions, while the program supplies a transition predictor and an inferred goal condition in its own internal state space .

This separation matters because a prediction mismatch has at least two possible causes: the stated environment rule may be wrong, or the code may fail to implement a rule that the rulebook already states correctly . A rulebook therefore constrains which programs count as faithful realizations; denote the set of faithful implementations by $[[\psi]]$ . In words, $[[\psi]]$ is the collection of executable programs that implement the meaning of rulebook $\psi$ .

The distinction gives reflection somewhere precise to act: revise the semantic hypothesis when the rule itself is suspect, or repair code while retaining the rulebook when implementation is the problem . But how can the agent distinguish a plausible account from one that actually predicts its experience?

Why Exact Replay Filters Models

Here is the part that gets technical: verification re-executes the recorded actions and checks whether the candidate program predicts every recorded next observation. Let $\operatorname{verify}(\theta;H_t)$ be true exactly when executable $\theta$ passes that check over history $H_t$ . In words, the program must reproduce the observations already seen, not merely fit the latest one.

For deterministic executables that terminate, replay gives a finite pass-or-fail check against the recorded trajectory . Each new proposal is tested against the whole history, so a repair for the latest discrepancy must still explain earlier observations . This eliminates programs that assign zero likelihood to any recorded observation, but it does not by itself establish that the code faithfully expresses the rulebook . What keeps an exact replay pass from endorsing code that contradicts its own stated rules?

Why Acceptance Has Two Conditions

Now the deeper constraint: an admissible pair must satisfy both observation consistency and semantic fidelity. The first is tested by exact execution against the record; the second asks whether $\theta\in[[\psi]]$ , meaning the executable implements the rulebook . Fidelity is assessed by the frozen language model, while replay is an execution check; acceptance is therefore not based on language-model confidence alone .

The loop is observation, reflection, possible rule revision, compilation or code repair, and verification. On failure, it returns to reflection and proposes again; on acceptance, replay also reconstructs the executable’s current internal state for the next planning round . This is a useful division of labor, but it leaves a final choice: among several pairs that pass, which one should the agent prefer?

Why Simplicity Is a Preference, Not a Proof

Here is one more layer for readers who want the model-selection distinction: admissibility and preference are different jobs. Replay and rulebook fidelity define which pairs are acceptable; a simplicity preference can then favor a less elaborate explanation among those that remain. Simplicity does not make a model correct, and it cannot replace either check. The mechanism described here grounds acceptance in execution where consequences are observable, while semantic fidelity remains an assessment of whether code matches its rulebook .

Minimal Causal Code

This sketch isolates the visible state update: append the new interaction, propose a rulebook–program pair, replay the complete history, and accept only if both checks pass. The reflection and compilation functions stand in for the frozen model’s proposals; they are intentionally not implemented here.

def update(history, rulebook, program, action, observation):
    history = history + [(action, observation)]

    proposed_rulebook, proposed_program = reflect_and_compile(
        history, rulebook, program
    )

    if not exact_replay(proposed_program, history):
        return update_after_failure(history, rulebook, program)

    if not faithful(proposed_program, proposed_rulebook):
        return update_after_failure(history, rulebook, program)

    return history, proposed_rulebook, proposed_program

Quiz

What must be true for a proposed rulebook–program pair to be accepted?

The rulebook must be short; replay and implementation fidelity are optional.

The executable must reproduce recorded observations on replay and be assessed as faithful to its rulebook.

The program must match only the latest observation, even if earlier observations disagree.

The language model must assign high confidence to the proposed program, regardless of replay.

What You Gain, and What You Lose

You gain an inspectable semantic memory, executable predictions that can drive planning, and a hard replay test that forces repairs to account for the full observed trajectory . Separating rule revision from code repair also makes the source of a mismatch an explicit diagnostic question .

You do not get a guarantee that the rulebook is true: exact replay only checks the observations in the record, and semantic fidelity is separately assessed by the language model . The method also depends on producing executable proposals that terminate for replay and on having a useful internal state representation; it is not a claim that natural-language explanations automatically become correct programs. The tradeoff is deliberate: more explicit structure and verification in exchange for a proposal-and-check loop that may need repeated revisions before a pair is admissible .

Core Takeaway

A rulebook says what the world should do; executable replay tests whether that account survives contact with the full history.