GroundingPI: Staged Spatial Training and Reward-Based Refinement

Why People Wanted It

A model can understand an image and still fail at the more exact job of naming an object and placing it in the right region. Spatial outputs have to be both meaningful and structured: a plausible phrase is not enough if its coordinates drift, its boxes are missing, or its OCR text is attached to the wrong region. So the real question was not just how to teach a model to see, but how to turn broad visual-language ability into coordinate-aligned outputs and then refine the errors that supervised examples leave behind.

The Core Move

GroundingPI first builds general multimodal understanding, then teaches structured spatial responses with supervised fine-tuning, and finally optimizes generated responses using group-relative task rewards. The source specifies causal next-token training for the supervised stages, but the supplied pages do not give a single formula for the complete staged pipeline.

Why General Understanding Comes Before Spatial Specialization

Now the version for readers who want more of the training machinery: the model is prepared in successive stages, moving from visual-language alignment to joint multimodal learning, then general visual and video understanding, and finally spatial specialization. In the initial alignment stage, the visual encoder and language model are frozen while the projector is trained on image-caption supervision. Joint pretraining then updates the visual encoder, projector, and language model on text-only and general image–text data. A further stage trains on visual question answering, instruction following, captions, and videos; spatial-specialization data is not separately mixed into that stage.

The point of this ordering is that spatial fine-tuning starts from a model already trained for general multimodal understanding, rather than asking the coordinate tasks to carry that whole burden. The supplied pages describe these stages as causal next-token prediction. What changes when the task becomes spatial rather than general?

Why Coordinate Alignment Uses the Same Token Prediction Objective

Here is the part that gets technical: coordinate alignment is a supervised fine-tuning stage over spatial task groups, including detection, GUI grounding, referring-expression tasks, OCR, document layout, dense pointing and counting, and visual prompting. Under teacher forcing, the model predicts the assistant response token by token; supervision includes semantic labels, coordinates, and protocol markers while excluding prompt and visual positions. Quantized coordinate tokens use the same cross-entropy objective as other valid response tokens, with no additional IoU regression loss.

This makes the training target a structured response sequence, not a separate box-regression head. The stated schedule first updates all modules and then freezes the vision encoder and projector for language-side refinement. If the model is trained to imitate reference outputs, how do we directly reward coverage and precision at generation time?

Why Group-Relative Rewards Separate Coverage from Precision

Now the deeper reinforcement-learning layer: GRPO samples eight autoregressive responses for each image–query pair and compares task-specific rewards within the prompt’s response group. Grounding uses a set-completeness reward and a strict localization-and-output-quality reward. The set reward asks whether each reference has a class-valid prediction with overlap; because each reference is matched independently, one prediction can serve more than one reference. The strict reward adds multi-threshold detection F1, matched-box overlap, format, count, ordering, and penalties for duplicate or oversized boxes.

The two grounding rewards are standardized separately and combined with weights 0.7 and 0.3. OCR uses a different target: Hungarian one-to-one matching scores text together with geometry, with complete word- and line-level views considered. Its composite reward includes exact text-and-region agreement, softer partial-match feedback, count agreement, and format gating. The resulting advantages guide a clipped GRPO update against a frozen SFT reference, updating language parameters. Why keep OCR’s text–geometry matching distinct from grounding’s coverage-and-localization rewards?

Minimal Causal Code

This is a schematic illustration of the visible state update in a group-relative training loop, not a reproduction of the paper’s full optimization details or a claim about its implementation. The paper pages specify the stages and reward design, but do not provide this PyTorch code.

# responses: one group of sampled outputs for a prompt
# reward_fn evaluates task-specific grounding or OCR quality
rewards = [reward_fn(response) for response in responses]
mean_reward = sum(rewards) / len(rewards)
variance = sum((r - mean_reward) ** 2 for r in rewards) / len(rewards)
scale = (variance + 1e-8) ** 0.5
advantages = [(r - mean_reward) / scale for r in rewards]

optimizer.zero_grad()
for response, advantage in zip(responses, advantages):
    log_prob = policy.log_prob(response)
    loss = -advantage * log_prob
    loss.backward()
optimizer.step()

What You Gain, and What You Lose

The staged design separates broad visual-language learning from spatial specialization, and supervised coordinate alignment gives structured outputs a direct token-level target. Reward-based refinement can distinguish missing coverage from imprecise localization, while OCR can reward agreement between recognized text and its region rather than text alone.

The tradeoff is that the reward definitions and matching rules become part of the training target: coverage matching can reuse a prediction, while strict localization penalizes duplicates and oversized boxes; OCR instead uses one-to-one assignment across text and geometry. The method also adds a reinforcement post-training stage after supervised training, and the supplied description confines its policy updates to language parameters with a frozen SFT reference. This module is about the training progression and reward logic; the shared coordinate-token interface is covered separately.

Quiz

What is the role of coordinate-alignment supervised fine-tuning in GroundingPI?

To replace token prediction with a separate IoU regression objective for boxes.

To teach structured spatial responses through causal token supervision, including coordinates and protocol markers.

To optimize action trajectories directly during spatial pretraining.

To train only the visual encoder on image-caption pairs.

$ check

Quiz

Which distinction best captures the post-training rewards for grounding and OCR?

Grounding uses only box regression, while OCR uses only output length.

Both tasks use only exact text matching, without considering geometry.

Grounding balances reference coverage and strict localization; OCR matches text and geometry together, including word- and line-level views.

OCR rewards coverage by allowing each prediction to match every reference independently.

$ check