Public AI Leaderboards Miss the System Around the Model

Evaluate an AI agent as a complete working configuration, not as a model name or leaderboard score. Run repeatable trials on representative business tasks, verify the resulting state, and measure tool behavior, failure recovery, latency, cost and human effort.

Public rankings can narrow a shortlist, but they cannot establish whether an agent fits your customers, permissions, data or operating limits. That decision requires a private acceptance test built around the work the deployed system will actually perform.

Define the system you are evaluating

An agent combines a model with instructions, tools, retrieval, memory, planning logic, permissions and retry rules. A result belongs to that complete configuration: changing one component can change both performance and operating cost without changing the underlying model.

IBM Research’s Open Agent Leaderboard illustrates the distinction by comparing specific agent-and-model configurations, reporting success and cost, and showing different results for systems built around the same model. It is a useful counterexample to model-only rankings, but its general-purpose environments still cannot define success for a particular startup product.

Give every configuration a versioned record covering the model version, system instructions, available tools, retrieval settings, memory policy, context limits, retry behavior and approval rules. Without that record, a prompt or tool change can make two results with the same model label impossible to compare.

Turn real workflows into test cases

Start with the jobs for which the agent will be accountable. A support agent’s suite might include answering from an approved policy, retrieving an account, making an eligible change and escalating an exception. Cover frequent work, commercially important work and credible edge cases rather than selecting only tasks the agent already handles well.

Each test case needs five elements:

• Starting state: the request, available records, permissions and relevant conversation history.
• Allowed actions: the tools the agent may call and the changes it may make.
• Expected outcome: the observable state that must exist when the task ends.
• Forbidden outcomes: actions or disclosures that invalidate the run regardless of answer quality.
• Review method: a deterministic check, a human rubric or a defined combination of both.

Grade the resulting state rather than relying on the final message. An agent may say that it changed a customer record when no change occurred; it may also reach a correct result using wording that differs from a reference answer.

This product-specific design is consistent with Henry Wang’s TechRadar Pro analysis, which argues that private evaluations can use real workflows and business-relevant outcomes unavailable to generalized external scoring. Treat that article as industry opinion, while defining your own criteria from observable product requirements.

Measure outcomes, trajectories and recovery

A pass-or-fail result can gate a release, but it does not explain why a run succeeded or failed. Preserve the trace: tool calls and arguments, tool responses, state changes, retries, final output, elapsed time and attributable usage cost.

Score distinct dimensions so that one strong answer cannot conceal an unsafe trajectory:

• Task success: did the required business state change correctly?
• Tool correctness: did the agent choose permitted tools and provide valid arguments?
• Data integrity: were its claims grounded in the records available during the trial?
• Policy compliance: did it respect permissions, approval points and prohibited actions?
• Efficiency: what latency, usage cost and tool activity produced an accepted result?
• Human burden: did a reviewer approve, correct or redo the work?

Test recovery by deliberately returning a tool error, timeout, empty result, rejected write or conflicting instruction. Record whether the agent retries safely, selects an allowed alternative, asks for clarification, escalates or invents information. A configuration that succeeds in a clean demo may create costly retry loops or unsafe actions when dependencies fail.

Anthropic’s agent-evaluation guidance defines evaluations in terms of tasks, trials, graders, traces and observable outcomes; it also recommends multiple trials because model outputs vary. The same guidance distinguishes the agent harness from the model and combines automated grading with human judgment where correctness is contextual.

Make the worksheet a release gate

Before seeing results, classify each criterion as blocking, budgeted or diagnostic. Unauthorized disclosure may be blocking; latency, cost and review time can have explicit budgets; unnecessary retries may remain diagnostic while the team investigates their cause. Precommitting prevents an impressive final answer from excusing a dangerous action.

Use one worksheet row per task, trial and configuration, with columns for:

• configuration and trial identifiers;
• starting state and expected outcome;
• accepted, rejected or human-review status;
• failed checks and forbidden actions;
• tool errors, retries and recovery result;
• end-to-end latency and total run cost;
• review time and required correction;
• a reference to the saved trace.

Calculate both cost per attempt and cost per accepted result. The first measures raw operating expense; the second exposes the retries and human correction required to obtain usable work. Keep task-level results alongside aggregates so a high average cannot hide a critical failure category.

Set the model-switch rule before comparing candidates

Run the incumbent and candidate against the same task definitions, starting states, tools, graders and trial policy. Change one controlled component at a time when possible. Replacing the model, prompt and tool layer together may improve the product, but the result will not identify which change produced the improvement.

Adopt a candidate only if it clears every blocking requirement and improves the business metric chosen in advance without exceeding cost, latency or review budgets. A startup might accept a higher run cost when it materially reduces corrections on a valuable workflow, while rejecting a cheaper configuration that increases unauthorized actions. Those thresholds are product decisions, not properties a public leaderboard can supply.

After release, add verified production failures and meaningful user outcomes to the private suite, with sensitive data removed or appropriately controlled. Re-run it after changes to the model, instructions, tools, memory or policy. Public rankings remain useful discovery evidence; the acceptance test determines whether the complete agent is fit for the startup’s actual work.

Also read:

• What Is Physical AI and How Jeff Bezos’ Startup Prometheus Aims to “Hack” the System
• Sakana AI’s Fugu: Multi-Agent Orchestration as a Single Model
• Volkswagen Enters the E-Bike Market with a Safety-Focused “Smart” Model — But at Car-Like Prices