ASSAY: No Action Without a Prediction

Abstract

ASSAY is a general agent harness built so that an LLM agent reasons its way through an unfamiliar world, learns that world from interaction at test time, and carries what it learns into later runs. A world is attached through one small adapter and a registry of permitted actions, and the agent is never told what its actions do: it discovers each by predicting its effect and paying for the answer. The single governing rule is that there is no action without a prediction, and every prediction is graded in code against the world's own response. That discipline is what forces the agent to build a world model under scarcity, which is where the reasoning and the test-time learning come from. The agent internalizes what it learns through self-declared instruments over its observations and through knowledge that carries across runs, demoted to hypothesis until re-earned, a tested capability whose measurement is the next phase. A by-product of the same discipline, not its aim, is a fully auditable record: every event lands on a hash-chained journal that a standalone public tool re-verifies from the artifacts alone, and a single ungated action invalidates a run. We evaluate the same harness on three worlds. On the ARC-AGI-3 public set it scores RHAE 96.54, confirmed by the benchmark server on a public scorecard, winning 24 of 25 games under hard action caps at a measured 8.0% exploration overhead and approximately zero capability cost over completed games. On the Factorio Learning Environment the same design fail-closes the sanctioned-interface bypass a published agent fell to. On the OOLONG long-context benchmark an early sweep places it in the same band as mature general harnesses across a 128K to 4M token ladder. The ARC-AGI-3 campaign also produced an epistemic finding: on six occasions the agent proved a level impossible, each proof consistent with every recorded transition and wrong exactly where none had gone, and a three-step audit of each proof's unexercised rules converted five into wins within existing budgets.