AutoEnvScaling: Automating the Data Flywheel with Terminal Agents

Simon Yu, Baolin Peng, Bo Liu, Zhengyan Shi, Zhaoyang Wang, Hao Cheng et al.

arXiv 2610.14163 44 pages Read the paper

The short version

The problem: Agents that work through a computer terminal need many realistic practice tasks, and once an agent improves, old tasks can become too easy. People have to design and check new tasks by hand, which limits how quickly training can grow.
The idea: AutoEnvScaling gives an agent a workspace and tools to build complete practice tasks, run them, and fix problems—like a researcher preparing an exercise and checking its answer key. It also uses the learner’s attempts to choose what to build next, and checks whether each task works and is useful for that learner.
What it got: It produced valid tasks at up to 25 times the rate of prompt-only generation, while cutting the cost per valid task by up to 75%. When one model both built tasks and learned from them, its Terminal-Bench 2.1 score rose by 12.2 points.

Modules from this paper

AutoEnvScaling: Building Training Environments in a Terminal

Useful training environments are costly to design by hand; AutoEnvScaling lets a proposer inspect solver failures, build and test complete tasks in a sandbox, then admit only tasks validated and calibrated for the current solver.

agentsreinforcement-learningdata-generationterminal-agents

Co-Evolving Proposer and Solver

When environment generation and task solving need to improve together, AutoEnvScaling lets one policy play both roles and trains it with role-specific rewards and advantages across rounds.

AutoEnvScalingself-playreinforcement-learningterminal-agents

Generated Clarification Tasks

Agents often attempt underspecified tasks without asking what is missing; AutoEnvScaling generates tasks with withheld requirements so agents can practice asking useful questions before acting.

agentsclarificationreinforcement-learningtask-generation