Schema-Conditioned Typed Decisions for Structured Extraction

Why People Wanted It

A conventional extractor has to learn its labels as part of its model. That makes a new schema awkward: you want to ask for different entity or relation types without rebuilding the whole prediction setup. The real question is whether the schema can participate in reading the text, rather than being bolted on after a generic representation has already been made.

Structured extraction also has different-shaped decisions. An entity or event trigger is a span; a relation connects two spans in a direction. Can one model treat these as related choices while still allowing the answer to be “none of the queried types”?

The Core Move

Encode the text and the queried type names (and optional descriptions) together, let type representations and candidate representations interact, then score each span or ordered span pair over the queried types plus a null class.

For a candidate $c$, the decision is a distribution over $\{0\}\cup K$, where $K$ is the set of queried types and $0$ means null. No additional formula is needed to state the core move: the source describes the decision this way, and gives the scoring details below.

Quiz

What does the null class mean in a typed span or span-pair decision?

The candidate belongs to none of the queried types.

The schema contains no type descriptions.

The candidate is removed before encoding.

$ check

Why Types and Candidates Attend to Each Other

Now the version for readers who want more of the machinery: the interaction layer is not just a shared encoder; it explicitly lets the queried types and candidate spans exchange information before scoring. Each type or candidate starts with a representation, and a learned embedding indicates whether it is a null, type, or span item. The sequence passes through pre-norm Transformer blocks with full self-attention and no position information. The resulting type vector can therefore reflect the other queried types and overlapping candidates, while a candidate vector can reflect the schema around it.

This interaction matters when the same text span could plausibly fit related labels: the score is conditioned on the other choices present in this instance, not only on the span in isolation. The source states the attention setup, but does not give a standalone equation for this interaction. What should the final decision do with those contextualized representations?

Why a Span Decision Has a Null Class

Here is the part that gets technical: a candidate span makes exactly one class choice from the null class and the queried types whose annotation status is known. The entity score uses a bilinear form between a projected span representation and a projected type representation, plus a learned bias; the null score depends on the span through a learned null vector. A softmax turns those logits into the class distribution.

A span representation combines its first and last token representations, the mean of its tokens, and a width embedding. This is the scoring target’s geometry: boundary evidence, span-wide evidence, and length information meet before the type decision. Some spans can carry more than one gold type, such as a coarse and a fine type; training maximizes their total probability. Unknown types are omitted from the decision rather than treated as absent. Why does the same idea need a different candidate shape for relations?

Why Relations and Events Reuse the Same Decision Pattern

For readers who want the deeper connection: a relation candidate is an ordered pair of distinct entity spans. Its representation combines the two span vectors, their element-wise product, and the mean of the tokens between them (zero when there are none). It then makes one choice among “no relation” and the queried relation types, using its own interaction layer and bilinear scoring setup. Because the pair is ordered, reversing the spans creates a different candidate and can express the opposite direction.

Events are represented through the same span and pair decisions rather than a separate output form. First, event types are queried as entity types and trigger spans are predicted. For arguments, event types and roles are queried: argument spans are entity decisions, and each trigger-to-argument pair is a relation decision labelled with a role. At inference, triggers are predicted first, then a second pass proposes arguments and assigns roles to trigger-span pairs. This reuses the decision shapes, but leaves a practical dependency: what happens to argument proposals when a trigger is missed?

Minimal Causal Code

This loop shows the visible decision shape, not the model’s attention or bilinear scoring. The logits_for function stands in for those learned components.

import torch

# Each candidate gets logits for null plus the queried types.
for candidate in candidates:
    logits = logits_for(candidate, queried_types)
    probabilities = torch.softmax(logits, dim=-1)
    choice = probabilities.argmax(dim=-1)

    if choice == 0:
        prediction = None
    else:
        prediction = queried_types[choice - 1]

What You Gain, and What You Lose

The schema is part of the encoding and candidate interaction, so the model can make decisions relative to the types requested for that instance. A shared span/pair decision pattern also gives entities, relations, triggers, and arguments a common structure, with null as an explicit outcome.

The setup still depends on candidate construction: spans are bounded by a maximum width, and relation decisions operate on ordered pairs of entity spans. Event arguments are proposed after trigger prediction, so the second stage depends on the first. And adding a null class does not by itself solve how to select candidates at inference; thresholding and calibration are a separate concern.

Quiz

How are event triggers and arguments represented in this approach?

Arguments are generated as text and do not use span-pair decisions.

Triggers are spans, and arguments are spans connected to triggers by role-labelled relation decisions.

Both triggers and arguments are predicted only as whole-sentence labels.

$ check