FDE-Gym’s Online Practice Platform
People and agents need a comparable way to practice customer delivery; FDE-Gym puts them in shared simulated workspaces with common stakeholders and graders.
Read the paper ↗What it is
- Shared practice environment FDE-Gym lets people and AI agents work in the same simulated customer workspaces, talk with the same stakeholders, and receive scores from the same graders.
- Human baseline Industry experts are using the platform for recorded practice on the least-specified tasks; the results have not yet been reported.
Why it matters
A common environment gives people and agents a shared way to practice and be evaluated. It also makes clear what the platform can offer now—and what remains unfinished: a reported human comparison.
Before getting into the setup, here’s the short version: FDE-Gym gives people and AI agents a common simulated place to do customer-delivery work. They use the same workspaces, interact with the same stakeholders, and are judged by the same graders. That makes the platform useful for practice and comparison, though it doesn’t mean the human-versus-agent results are already in.
-
Comparable practice: A benchmark score is useful, but it doesn’t give a person a place to rehearse the work. The platform puts human participants into the simulated customer workspace used for agent evaluation, so the activity itself is shared rather than merely similar.
-
Reproducible interactions: The canonical example is asking a stakeholder for a missing constraint. In this environment, the stakeholder’s knowledge and conditions for disclosing information are set in advance; responses are scripted partly because changing the user-simulation model can shift measured agent success rates. The simulation also includes things a competent delivery engineer has to manage, like patience, evidence requirements, prior conversations, and availability. It is a controlled stand-in for customer interaction, not a full reproduction of a real person.
So is a common workspace enough to make a human baseline? It gives people and agents a common setting, but the score still has to come from the same grader. That common evaluation is part of the platform’s design: both kinds of participant are judged by the same graders. Other parts of FDE-Gym define how tasks and scores work; the point here is that the online practice path uses the same evaluation machinery.
- A baseline still being gathered: The human collection is under recorded conditions, with industry experts working on L3 tasks—the level based on the sponsor’s original request. The results are planned for a future version, so this setup is a way to collect a comparison, not a published conclusion about whether people or agents do better.
I like the restraint in that distinction: a platform can make a comparison possible without pretending the comparison is already settled. What remains open is how human performance will look once that baseline is reported.
Read the deep dive
Why People Wanted It
A benchmark can tell you how an agent performs, but it does not automatically give people a place to practice the same kind of work or a fair way to compare their attempts. Customer delivery is not just answering a prompt: you work inside a customer workspace, gather context, and decide when and how to communicate. The practical question is whether people and agents can face a common environment and be judged on a common basis.
The Core Move
FDE-Gym extends its simulated customer-workspace setup to human participants: people and agents use the same workspaces, interact with the same stakeholders, and receive evaluations from the same graders.
That shared setup is the key move. It makes a human practice path part of the same environment used to evaluate agents, rather than creating a separate exercise with different rules.
Why a Shared Workspace Makes Comparison Possible
Now the version for readers who want more of the machinery: the platform’s comparison depends on holding the interaction environment and evaluation procedure in common. The platform description specifies three shared elements: workspaces, stakeholder interactions, and graders. This does not make human and agent behavior identical; it makes their attempts legible under the same setup.
The boundary matters. Stakeholders are scripted for reproducibility, with fixed knowledge, misconceptions, and disclosure conditions; the simulation also includes communication constraints such as limited patience and temporary unavailability. Those are features of the surrounding benchmark environment, not a claim that the platform reproduces every quality of a real customer relationship.
Why the Human Baseline Is Still in Progress
Here is the part that gets more technical: a common grader is necessary for a comparison, but it does not itself supply human performance results. Human participants use the online platform under recorded conditions, and the reported collection is focused on industry experts working on L3 tasks. The baseline is being collected; its results are reserved for a future version.
That distinction keeps the platform’s present role clear: it provides a shared practice and evaluation setting, while the human baseline remains an ongoing measurement rather than a published score.
Minimal Causal Code
This sketch makes the shared-environment idea visible; it is not a claim about the platform’s implementation.
workspace = make_customer_workspace(task)
stakeholders = make_scripted_stakeholders(task)
grader = make_task_grader(task)
for participant in participants: # a person or an agent
attempt = participant.work(workspace, stakeholders)
result = grader.evaluate(attempt)
record(participant, result)
What You Gain, and What You Lose
A shared workspace, stakeholder set, and grader give people and agents a common place to practice and a more comparable evaluation basis. Scripted stakeholder responses support reproducibility, but they are not a substitute for the variability of real customers. And while a human baseline is being collected from industry experts on L3 tasks, those results are not yet available in this account. The platform offers a route to comparison, not a completed verdict about how humans and agents perform relative to each other.
A shared workspace and grader make human practice comparable with agent evaluation—but the human baseline is still being collected.