Skip to content
Gym · agent training

Agents that get better at your work. Every change earns its release.

Tell Gym what an agent needs to improve. It builds practice from real cases, proposes a change to the agent configuration or trains the model, proves the result on work it never saw, and waits for an owner to approve.

Self-improvement loopSupplier review · Finance OpsIteration03/ 07
  1. 01 Run24 cases on v3
  2. 02 AnnotateC-14, C-15: credit notes ignored
  3. 03 ReflectNet credit notes everywhere
  4. 04 Proposep-0423 · instructions: net credit notes
  5. 05 Eval18/24 (+1) · 2 regressions
  6. 06 ReviewGate G2 failed · rejected
  7. 07 ShipReverted: v3 stays live
  8. Next iterationv3 → v4reverted
Judge pass ratelive shipped held ✗reverted
40%60%80%100%base1234567Iteration 1: v2 67% (shipped)Iteration 2: v3 71% (shipped)Iteration 3: v4 75% (reverted)Iteration 4: v5 79% (shipped)Iteration 5: v6 83% (held)Iteration 6: v7 83% (held)Iteration 7: v8 88% (shipped)
Eval · v4 on 24 cases75%
Judge pass
18/24 (+1 vs v3)
Fixed
C-14, C-15, C-20
Regressed
C-03, C-08
  • G1Pass rate ≥ live version
  • G2No regressions
The Gym loop, one iteration per version: run, annotate, reflect, propose, evaluate, review, ship. Illustrative data.
Two ways to improve

Change the configuration, or change the weights. Gym tries the cheaper one first.

Most gaps close with a better prompt, tool or retrieval rule. Some need the model itself to learn. Gym measures both against the same tests and keeps the one that wins.

  • 01

    Edit the agent configuration

    Instructions, tools, retrieval and routing change. No retraining, so the change is fast to test and easy to read in a diff.

  • 02

    Train the model

    When a skill has to live in the weights, Gym trains an adapter or a small model on practice built for the goal.

Fig. 1 · Weights or configurationIllustrative data
The same failure fixed two ways: an edit to the configuration, or an update to the weights. Both go through the same gate.
Fig. 2 · Configuration editIllustrative data
An improvement made by editing the configuration: the failing cases pass, nothing is retrained.
Practice from real work

Environments mined from your own cases. Common work first, then the hard exceptions.

Gym turns past runs, tickets and reviewed answers into practice environments with checks it can score. Held-out cases are set aside before any training starts.

Fig. 3 · Environment miningIllustrative data
Real cases clustered, de-duplicated and turned into environments with verifiable checks.
Fig. 4 · Gym pipelineIllustrative data
Gym: mined environmentsEnvironments are mined from real workflow history: a closed case is cut at a decision point, the part before becomes the task, the part after grades it. An agent is scored per environment, improved, proven on held-out cases and released only when a person approves. Point at an environment or a stage.WORKFLOW HISTORYCRMREAD-ONLYERPREAD-ONLYMESREAD-ONLYEvent logobject-centricevent ↔ order, items, invoiceCut at a decision pointSTART STATEHELD BACKONE ENVIRONMENTmfg · rework orderWorld state + toolsseeded db · same toolsTaska real case, cutVerifierend state · many pathsPolicy checksno release w/o approvalENVIRONMENT SUITE9 ENVS · SCORE NOWo2ccredit block0.68o2cpartial delivery0.71p2pthree-way match0.82p2pprice variance0.66p2pduplicate invoice0.88mfgscrap reporting0.55mfgrework order0.44scmlate supplier0.73scmstock transfer0.79| before ━ after (demo scores)Agent under testmodel + instructions + toolsactscoreImprovetune instructions or trainHeld-out proofbasetrainedRelease · a person approvesevidence stays attachedFAILURES DECIDE THE NEXT TASKS MINEDAGENTS PROPOSE · CODE CHECKS · A PERSON APPROVESverifier per task · held-out split never trained on

Point at a part 9 environments · 317 tasks · weakest: mfg · rework order

From a goal to a release: mined environments, training or configuration search, held-out evaluation and review.
Weight training

When the model has to learn it. Rewards from checks, not opinions.

Gym trains with reinforcement learning from verifiers, distils from a stronger teacher, and orders practice so skill builds. Each run is a recipe you can read and rerun.

Reinforcement and distillation. GRPO and on-policy distillation.

GRPO samples several answers per task, scores them with the environment's checks and pushes the model toward the better ones. On-policy distillation lets a small model learn from a larger teacher on its own attempts.

Fig. 5 · GRPOIllustrative data
K samples per task, verifier rewards, group-normalised advantages.
Fig. 6 · On-policy distillationIllustrative data
A teacher scores the student's own attempts token by token, so the student learns where it actually goes wrong.

Curriculum and recipes. Practice ordered as skill rises.

Tasks get harder as the pass rate climbs. The whole run, from data to evaluation, is a staged recipe kept with the version it produced.

Fig. 7 · CurriculumIllustrative data
Tasks ordered by difficulty; the frontier moves as the model's skill rises.
Fig. 8 · Recipe pipelineIllustrative data
A training recipe as stages: data, filter, train, evaluate, with a gate between each.
Adapters and routing

Many specialists on one base model. Each request goes to the one that fits.

Trained skills ship as LoRA adapters beside frozen weights. Many adapters share one served model, mix per request, and routers send each task to the model or agent that handles it best for its cost.

Fig. 9 · LoRAIllustrative data
A low-rank adapter beside frozen weights: a small fraction of the parameters.
Fig. 10 · Multi-LoRAIllustrative data
Many adapters served on one base model.
Fig. 11 · Dynamic LoRAIllustrative data
Adapters mixed per request. Move the pointer to change the mix.
Fig. 12 · Model routerIllustrative data
Requests routed to models by cost and quality, including your own tuned models.
Fig. 13 · Task routerIllustrative data
Tasks routed to the specialist agent that handles them.
Recursive improvement

Gym keeps a population of candidate versions that differ in how they work, not only in how well. Improvements compound: the loop that improves the agent is itself measured and improved.

Fig. 14 · Quality-diversity searchIllustrative data
Candidate proposals on a grid of behaviours; the best in each cell is kept if it passes the eval gate.
Fig. 15 · Recursive improvementIllustrative data
Nested loops: runs improve the agent, and evaluations improve the loop that proposes changes.
Proof and release

Agents propose. People approve. Every version keeps its evidence.

Each candidate arrives as a proposal: the diff, the eval results against the version in use, and a recommendation. A named owner approves or rejects it, and the lineage records who, when and why.

Fig. 16 · ProposalIllustrative data
instructionsagent:supplier-reviewpending reviewappliedrejectedsource · reflectionp-0427 · Sep 24

Say when the data can't answer

Rationale On C-17 and C-18 the agent gave a confident figure and a reason that are not in any table. When the data cannot answer, it should say so first, name what is missing and offer the closest figure the data does support.

agents/supplier-review/instructions.md+4−1
@@ −12,4 +12,7 @@Answering
1212Answer with the figure first, then the table or page it came from.
1313Use the recipes in the wiki for totals and matches.
14−If a question needs data you do not have, give your best estimate.
14+If the data cannot answer the question, say so in the first line.
15+Name what is missing (a forecast, a reason, a risk score) and
16+offer the closest figure the data does support.
17+Never present an estimate as a recorded value.
1518Keep answers under 120 words unless asked for detail.
Eval evidence · same 24 cases
v3 · live17/24
v5 · proposed19/24
+2 cases (71% → 79%)
Fixed
C-17, C-18
Regressed
none
  • G1Pass rate ≥ live version19/24 vs 17/24 on v3
  • G2No regressionsevery case v3 passed still passes
lineage: v3 → v5evidence: eval run ev-0193 · annotations a-240, a-241gates pass
A proposed change with its diff, its scores against the live version and the owner's decision.
Fig. 17 · Version lineageIllustrative data
Lineageagent:supplier-review · 8 versionsshipped candidate ✗blocked revert
revertv113/24✓ R. Patelv216/24✓ R. Patelv317/24✓ A. Lindqvistv418/24✗ A. Lindqvistv519/24✓ R. Patelv620/24✓ A. Lindqvistv720/24✓ R. Patelv821/24✓ R. PatelLIVE
The version tree: each version with its evidence, its approver and the version it replaced.

Bring us one capability to improve. We will show the path to a proven release.

One goal, practice from your cases, held-out proof and a release your team approves.