Terminal-RSI: Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine

The short version

The problem

Terminal agents must actually repair or build software so that a separate checker accepts the result; plausible answers and partial fixes are not enough. Learning from these tasks is difficult because long logs can distort which actions get credit, while changing the agent can make its value estimates unreliable.

The idea

The paper alternates between collecting verified successful solutions and improving the acting policy with reinforcement learning, while carrying the value model forward and recalibrating it after each policy change. It also assigns learning credit across meaningful interaction turns rather than every generated word, and adjusts rewards so passing some checks helps but never counts as much as verified completion.

What it got

The resulting 27-billion-parameter agent achieved a 50.58% average success rate across four terminal-task test suites, versus 20.68% for its starting model. On a long-horizon benchmark it reached 0.37 process reward and strictly completed 5 of 46 tasks.