Skip to content

Research note

Environment mining for agent gyms

Reviewed runs become checkable environments.

Research note · Training → Ongoing

Every gym needs environments: a task, the tools an agent may use, a frozen slice of the world to act on, and a way to tell whether it did well. Writing them by hand does not scale, and the ones people write tend to test what the author already knew the agent could do. The runs the platform already keeps are a better source. Each one is a real question someone asked, the tools that were used, the evidence that was found, and, when a reviewer looked at it, a judgement of whether it was right.

This note describes how we turn those runs into environments, and the guards that stop the result from being a flattering mirror.

From a reviewed run to an environment

A run becomes a candidate when a reviewer has either approved its answer or corrected it. Unreviewed runs are not mined: without a human judgement there is nothing reliable to grade against. From each candidate we extract four parts.

  1. The task, rewritten to remove anything the agent should have discovered itself (dates of the answer, names of the culprit table).
  2. The tools, restricted to the ones the original run was allowed, with write actions removed.
  3. A snapshot of the data the run touched, pinned to the time of the question, so later records cannot leak the answer.
  4. A checker, built from the reviewer’s judgement: which sources must be cited, which root causes count as correct, which actions are forbidden.
Fig. 1Mining environments from real cases: runs are filtered by review, de-duplicated, given checkers and difficulty, and only then admitted to the gym. Illustrative data.
env/line-b-fpy-w38.yamlyaml
task: "Explain the drop in first-pass yield on Line B, week 38"
tools: [sql.read, docs.search, lots.genealogy]
snapshot: workspace@2026-09-21   # frozen data, read-only
checker:
  must_cite: [table:inspection_results, doc:SOP-curing-rev4]
  root_cause_in: ["press 7 cure temperature", "cure spec revision"]
  forbid: [write_actions]
difficulty: 0.62                 # from historical solve rate
provenance: run 41e9, reviewed by quality owner

The checker is deliberately narrow. It does not grade prose. It checks things a program can check: that the answer cites the inspection table and the curing procedure, that the stated cause is one of the accepted ones, that no write was attempted. That makes the reward verifiable, which is the property that matters most if these environments are used for reinforcement learning rather than only for evaluation.1Group-relative policy optimisation (Shao et al., 2024) works well with this kind of reward: sample several attempts, score each with the checker, and push toward the ones above the group mean.

De-duplication and leakage

Real workloads repeat. The same monthly variance question gets asked twelve times a year, with different numbers and the same shape. Left alone, those would dominate the gym and the agent would learn one trick very well. We cluster candidates by the structure of their solution rather than the wording of their question: the sequence of tool types used and the entity types touched. Each cluster contributes a capped number of environments.

Leakage is the subtler problem. Three guards are in place.

  • Time. Snapshots stop at the moment the question was asked. A later incident report that names the cause is outside the environment.
  • Split by cluster, not by row. Evaluation environments come from clusters that contribute nothing to training. Otherwise the eval set measures memory of near-identical cases.
  • Task rewriting is checked. A second pass searches the rewritten task for strings from the accepted answer. Any hit sends the candidate back for a manual rewrite.

Difficulty and order

Each environment gets a difficulty estimate from history: how often runs on similar tasks succeeded, how many steps they took, how often a reviewer corrected them. New environments start with that estimate and update as agents attempt them in the gym.

Fig. 2Tasks ordered by difficulty as skill rises: the sampler stays near the edge of what the current configuration can solve. Illustrative data.

We sample near the edge of competence. Environments that every attempt solves teach nothing, and environments that no attempt solves give a group of identical zero rewards, which carries no gradient for a group-relative method and no information for an evaluation. Keeping the solve rate for the sampled batch in a middle band is the single setting that most changed how useful a gym run was.2Curriculum learning in this sense goes back at least to Bengio et al., 2009. Our version is simpler: a target solve-rate band, not a hand-designed sequence.

Coverage, not just score

A high average score on the gym can hide a configuration that is excellent at one kind of task and poor at the rest. We track coverage on two axes, the task family and the difficulty band, and keep the best configuration per cell. This is a quality-diversity view: the goal is a map of what works where, not one champion.

That map is also how we decide what to mine next. Empty cells are task families the gym does not yet represent, and they become requests to look for reviewed runs of that shape.

The best environments are questions someone actually asked. With a checker the person who answered them would accept.

Limits

Mined environments inherit the biases of the workload. If nobody asks a certain kind of question, the gym will not test it, however important it is. We add hand-written environments for those gaps and label them as such. Checkers also lag reality: when a procedure is revised, environments built on the old revision become wrong and need to be retired. Every environment carries the provenance of the run it came from, so retiring them is a query, not a search.

The broader point is that the improvement loop and the audit trail are the same data. A run that keeps its sources and steps is, after review, a test case.

Read the evidence yourself. Every run keeps its sources.