Evaluate and improve

Your coding agent evaluates a Recurse agent revision by comparing it with a baseline on the same cases, evaluation criteria, model, and allowance. The comparison should show whether the change improves results without losing other capabilities, violating constraints, or using more resources than the task permits.

You review the requirements, trade-offs, and allowance. Your coding agent varies the approach, selects cases, runs the comparisons, and interprets the evidence. Each Recurse agent implements a candidate iterative refinement approach.

Establish a baseline

The coding agent tracks revisions of the "current" baseline and chooses a set of representative inputs, including cases where it currently fails. It records the prompt, tools, the dependency lockfile, model, and verifier versions so later comparisons can identify what changes in between trials.

When overseeing your coding agent, consider scrutinizing the following for each run:

The coding agent should always test tool behaviors locally before launching remote runs.

Testing improvement hypotheses

If its allowance permits, your coding agent will try multiple hypotheses to advance the baseline. For each thesis, it should state:

1. The problem this candidate will attempt to fix.
2. A summary of the change it proposes.
3. The effect/result it expects.

It then compares each revision with the baseline. The Recurse skill instructs your coding agent to prefer small prompt, tool, or diagnostic changes to large re-architectures unless there is a good justification for the latter.

When the evidence contradicts the hypothesis, the coding agent should perform some sanity checking; e.g., verify whether the implementation was correct, whether measurements capture the actual effect. As you oversee the agent design process, you may want to instruct your coding agent to scrutinize these things in greater detail if an obviously-correct thesis fails to show improvement.

Review and steer the comparison

Your review should focus on requirements or evidence your coding agent may interpret incorrectly:

• If a high-scoring result is unusable, identify the missing requirement or weak verifier with a concrete example.
• If a change helps one case but harms another, scrutinize verifiers and checks; they may not provide enough coverage for all use cases that matter.
• If attempts are valid but unproductive, supply relevant domain facts or identify a missing tool capability.
• If exploration is consuming the allowance without useful progress, push for concrete gains and scrutinize remaining hypotheses.

When a materially new requirement changes acceptance, confirm it before the coding agent changes verifiers. Note that scores from different evaluation mechanisms will often not be directly comparable.

Retain a revision or stop

Your coding agent should retain a revision, or update the baseline, when the evidence supports its benefit.

It will declare convergence and stop the design process when one of the following happens:

• It reaches the acceptance target.
• Further gains do not justify another attempt.
• It exhausts the given allowance.
• A blocker is preventing meaningful progress.

A timeout alone does not prove diminishing returns. Your coding agent should support the convergence decision with the history of gains, remaining approaches, or cost considerations. You may want to scrutinize its decision if it stops working pre-maturely.

If the final Recurse agent works well, save it alongside any results, known failures, and supporting artifacts. Continue with Deploy and connect through MCP when the Recurse agent is ready for ongoing use.