Elastic GPU Scheduling for Long-Horizon Agents
Long-running agent executions can outlast the GPU allocation that started them; QwenGyre shifts cells between rollout and training at safe boundaries while rerouting live requests and admitting satellites into an ongoing training stream.
Read the paper ↗What it is
- Waterlevel scheduling The scheduler counts dispatched executions that have not finished, including queued and running work, and moves a GPU cell to training only when the rollout capacity left behind can still cover them.
- Harness-preserving rerouting When a GPU changes jobs, the agent's harness and workspace stay in place. Requests are redirected from the old model engine to another available engine.
- Streaming training Training data is packed into small pieces as it becomes ready. More GPU cells can join the same batch by pulling the core cell's current weights and claiming work.
Why it matters
Long-running agents need their execution environment to survive GPU role changes. This design reclaims rollout capacity at safe boundaries and overlaps training with data preparation, at the cost of explicit coordination for routing, weight snapshots, and synchronization.
Before I get into the mechanics, here is the short version: QwenGyre lets GPU cells change jobs as rollout work winds down, instead of keeping every cell fixed in one role for a whole cycle. It keeps the agent's harness and workspace alive while model requests move to another engine, and it feeds training in small pieces so additional cells can join an active batch.
- Capacity: Long-running executions don't finish together, so a fixed rollout/training split can leave GPUs waiting on whichever side is slower. The scheduler needs a signal for how much rollout work is still committed, not just a snapshot of GPU utilization.
- Continuity: A model engine is only one part of an agent execution. The harness and workspace hold the ongoing interaction, so switching GPU roles should move the requests, not make the agent start over.
- Training supply: Waiting until every training example is fully packed would delay useful work. A stream of small pieces lets the core begin and lets other cells join as data and capacity are available.
So what does it mean to count committed rollout work? The canonical example is a queued execution: it has been dispatched but has not launched yet, so a running-only count would miss work that still needs a rollout slot. Add the executions already running, and you have the outstanding demand the scheduler must protect. QwenGyre calls this count the waterlevel: dispatched executions minus finished executions. In plain terms, it falls only as executions close, and a cell can move to training when the rollout capacity left behind still covers that count. This is deliberately conservative; it is not a forecast of how long each agent will run.
But capacity alone doesn't make training useful. The scheduler also checks whether enough ready groups are available for the batch, and requires that supply to hold for a configured interval before it starts a transition. Cells move in a fixed order: the core first, then satellites. The core keeps the authoritative weights and optimizer state; satellites can pull its current weights. Why not transition every GPU as soon as the waterlevel drops? Because rollout still has queued and active executions to serve, and training needs a dependable supply of ready groups.
Here is the concrete continuity case: an agent is in the middle of using a repository through its harness when its GPU cell is needed for training. The harness and workspace remain outside that GPU worker. The source engine stops taking new requests; the scheduler reserves another engine and updates the route, then the proxy retries the outstanding request there. In effect, the engine changes but the agent's working world stays put. Optional KV-cache transfer can avoid recomputing a prefix, though transferring cache itself needs coordination. What must move with the request, and what can stay where it is?
For training, picture the core consuming a batch while a satellite becomes available. Rather than waiting for a whole new batch, the satellite pulls the core's current weights and claims micro-steps from a shared stream. The packing buffer prepares those pieces as ready groups arrive; cells claim disjoint work as they have capacity. The weights stay fixed during gradient accumulation, and the joining window closes before the optimizer update. Then the core combines gradients and performs the update. That gives the system a way to add workers without handing optimizer ownership to each one. What keeps a late-arriving cell from training on a different version of the model?
Overall, I like the separation of concerns here: execution state survives outside the GPU role, while a single core stays responsible for optimizer state. The cost is that “elastic” does not mean free-form or instantaneous. Safe movement depends on outstanding rollout demand, ready training supply, request routing, consistent weight snapshots, and coordinated pipeline state.
Read the deep dive
Why People Wanted It
Long-horizon agents make GPU allocation awkward. Rollouts can remain active while a training batch is becoming ready, but reserving every GPU for rollout wastes capacity and stopping an execution just to reclaim its GPU breaks the agent's ongoing interaction. The real question is not simply how to divide GPUs between rollout and training: how can the system reclaim capacity without interrupting the harnesses that own live work, and start training before every example has been materialized?
The Core Move
Treat GPU cells as elastic roles, and move them from rollout to training only when outstanding rollout work is covered and enough training data is ready; keep the execution harnesses alive while requests are rerouted.
Let count dispatched executions and count finished executions. The rollout waterlevel is : queued plus running work that has not yet finished. A cell can leave rollout when remaining rollout capacity still covers that waterlevel.
Why Waterlevel Makes Reallocation Safe
Now the version for readers who want more of the machinery: the waterlevel is a conservative count of committed execution demand, not a utilization estimate. Let count launched executions. Then is queued work and is running work, so . Dispatch, launch, and completion preserve ; retries of a model request stay inside one execution and do not change these counts. An execution increments once when it completes, fails, times out, or is cancelled.
Within a burst, dispatch quota stays fixed while available rollout slots are filled from the queue. At a transition boundary, the scheduler tests whether rollout capacity after moving the next cell still covers . It also requires a sustained supply of ready, unassigned groups: the count is capped by how many groups the current batch still needs, must reach the configured fraction of batch size, and must remain there for a configured interval. Cells move serially, core first and then satellites by index, so the training allocation grows as a prefix rather than an arbitrary subset.
At burst end, satellites return to rollout and the core synchronizes updated weights to rollout engines. The core can remain in training for the next batch only if both the ready-group supply and rollout-capacity checks still pass, including room for pending executions. Why make the core special? Because it owns the authoritative weights and optimizer state; satellite cells pull its parameters, but do not own optimizer state.
Why Live Harnesses Survive a Role Change
Here is the part that gets technical: the GPU role changes, but the execution's harness and workspace remain outside the GPU worker, so changing an engine does not mean rebuilding the agent's world. Before a source engine gives up a live execution, it stops accepting new requests. The scheduler reserves destination capacity and changes routes, then cancels outstanding source requests; the proxy retries them at their destinations. Standalone rollout engines are preferred, followed by rollout cells in descending cell order.
When enabled, KV-cache handoff reduces prefix recomputation: the source pins cache blocks before cancellation, the destination reads them after writes stop, and the source releases them after acknowledgement and takeover. Only after rerouting and cache handoff does the cell release rollout caches and temporary buffers and restore training resources. Backend processes and communication groups stay alive across the role change. What makes the transition compatible with a live harness is the separation between the durable execution state and the replaceable model-serving engine.
Why Streaming Training Can Grow Its Membership
Now the deeper training detail: the core exposes its authoritative weight buffers before each step, and a satellite joining in the predefined order pulls the current snapshot while the core continues processing the batch. Weights remain fixed during gradient accumulation, and the controller closes the joining window before the optimizer update, so a late participant does not mix parameter versions within that accumulation window.
The trajectory buffer holds execution records and rewards; a separate packing buffer selects ready groups and emits small micro-steps on demand. Cells atomically claim disjoint micro-steps as capacity becomes available, so faster or earlier cells can consume more of the stream. Each participating cell runs a continuous one-forward-one-backward pipeline; when data pauses, stages finish outstanding communication and wait at consistent points, then drain remaining work when the stream closes. After the batch, the core aggregates and normalizes gradients from participating cells, clips them, and performs one optimizer update. For nonfinal updates it broadcasts new parameters to current satellites; after the final update, the rollout policy is published. How can variable participation still produce one update? The core remains the synchronization and optimizer authority while the stream and communication groups accommodate the active cell prefix.
Minimal Causal Code
This toy loop makes the boundary decision visible. It omits the real scheduler's asynchronous launches, engine transitions, and training-supply timer.
# Cumulative execution counts for the current burst.
dispatched = 0
launched = 0
finished = 0
# Before a burst, publish a fixed quota of new executions.
dispatched += new_quota
# Fill the currently available rollout slots; the quota does not change here.
while rollout_slots > 0 and launched < dispatched:
launch_execution()
launched += 1
rollout_slots -= 1
# On completion, failure, timeout, or cancellation, count the execution once.
def on_execution_closed():
global finished
finished += 1
waterlevel = dispatched - finished # queued plus running executions
# Reassign the next cell only if remaining rollout capacity covers that work.
if remaining_rollout_capacity_after_move >= waterlevel:
move_next_cell_to_training()
What You Gain, and What You Lose
You gain a way to reclaim GPUs as outstanding rollout demand falls, rather than waiting for every execution in a batch to finish. Keeping harnesses and workspaces outside GPU workers lets requests move between engines without discarding the execution's environment; streaming micro-steps let training begin and grow its participating cell prefix while data is being packed.
The tradeoff is control-plane and synchronization complexity. Capacity is governed by a conservative outstanding-work count and a ready-data condition, so a safe transition can leave GPUs idle when either condition is not met. Live rerouting adds request and optional cache-handoff coordination. Dynamic membership also requires consistent weight snapshots, prefix-specific communication groups, pipeline pause/drain behavior, and a single core to aggregate gradients and own optimizer updates. This is not arbitrary, instantaneous GPU sharing: it is ordered, boundary-aware role change with explicit coordination costs.
Move the GPU role, not the agent's world: count outstanding work, reroute its requests, and let training cells join the core's stream.