AI Agent Evaluation Metrics, Scorecards and Release Thresholds
An AI agent produces many measurable signals, but a long scorecard does not make a sound release decision. Management must first state which use, users and level of authority it is considering. The evaluation team can then measure the outcomes and controls that decision requires, preserve critical failures as release gates and report the uncertainty around every threshold.
Begin with the release decision
The same agent may be suitable for an internal pilot and unsuitable for autonomous customer action. A pilot may allow an employee to review every proposed response. A production workflow may send messages or change records without review. These uses expose the organisation to different consequences, so they cannot share one unqualified pass mark.
The evaluation brief should name the proposed use, population, environment and authority. It should also state the alternative under consideration. Management may be comparing two agent versions, an agent with an existing process or a multi-agent design with a simpler workflow. A metric belongs in the scorecard only when its result can affect that decision.
Define success through the completed business case
An agent can identify a user’s intent and still fail to complete the required task. It can also produce a fluent answer while leaving the wrong record, payment or booking in the business system. The primary measure should therefore describe the completed business outcome.
The evaluator must define the denominator and the evidence of completion. A customer-service case may pass only when the correct customer receives the permitted resolution, the case record is updated and no duplicate action occurs. Cases that the agent correctly refers to an employee still belong in the denominator when referral is part of the approved design.
Task completion is not always binary. A research agent may produce a usable report with missing evidence, while a coding agent may satisfy some tests and fail others. The scorecard can record partial completion when the business can use that distinction. However, it should not use partial credit to conceal a condition that makes the deliverable unusable.
Separate outcome, process and control measures
Outcome measures establish whether the case ended correctly. Process measures explain how the agent reached that state. They may cover routing, tool selection, argument accuracy, evidence use, handoff quality, retries, latency and cost. Microsoft’s agent evaluators make a similar distinction between task completion and tool-use process measures.
Control measures answer a different question: did the workflow remain within its authorised boundary? They should record permission breaches, missed approvals, disclosure, prohibited actions and unresolved commitments. A helpful response cannot offset an unauthorised payment, so a scorecard should not average these failures into general response quality.
Separating these measures allows the evaluation team to identify the required remediation. A correct outcome reached through repeated calls may indicate an efficiency problem. An incorrect outcome after correct tool use may indicate a decision-rule or evidence problem. A permission breach requires a control change even when the customer obtained the requested result.
Keep critical failures outside the weighted average
A weighted score can summarise performance across several dimensions. Yet the average can rise while a rare but unacceptable failure remains. For example, excellent task completion and low latency can mathematically outweigh one unauthorised disclosure if every measure enters the same total.
Management should define explicit gates for failures that cannot be traded against convenience. The relevant gates depend on the proposed authority and consequence. They may require zero observed cross-customer disclosure, zero unapproved payments or complete human review of a protected decision. A gate does not claim that the true failure rate is zero. It states that any observed breach blocks the release under the current evidence.
The remaining indicators can use weights when management can explain the trade-off. The scorecard should publish each weight, denominator, threshold and missing-data rule. Otherwise, a single total creates precision without showing which judgement produced it.
Segment averages before accepting them
An overall pass rate can hide the route that determines operating risk. Performance may differ by language, channel, tool, case difficulty, customer group, consequence or workflow path. The evaluation should define important segments before reviewing the results, then report both the overall result and each material segment.
Segmentation also tests whether the dataset represents the proposed use. If Arabic requests make up a material part of the deployment but only a few evaluation cases, the overall score cannot support a confident claim about Arabic performance. Management must either strengthen the test set or restrict the release rather than treat the sparse segment as equivalent evidence.
Analysts should avoid searching many small segments until one appears weak or strong. That practice can turn random variation into a persuasive story. Predefined segments, sufficient repeated trials and uncertainty intervals help management distinguish a persistent weakness from sampling noise.
Set thresholds from consequences and evidence
A threshold should express the minimum evidence required for a defined decision. It should not copy a convenient industry percentage or a vendor example without examining the business consequence. An 85 per cent task-completion threshold may be reasonable for one assisted pilot and indefensible for another use.
The team can begin with the current process, a simpler design and the consequence of each failure. The agent may need to improve cycle time without increasing correction or risk. A high-consequence action may require an approval gate even when average performance exceeds the operational target. These comparisons turn the threshold into a decision rule rather than a decorative target.
Trial variability also matters. Anthropic distinguishes the chance of at least one success across several attempts from the probability that every attempt succeeds. A design tool may benefit when one of several generated options works. A customer-facing workflow usually needs consistent first-attempt performance. The evaluation team must choose the reliability measure that matches actual use.
The team should record the release rule before reading the final results. If management changes a threshold after seeing the score, the report must state what changed and why. Otherwise, the organisation can unconsciously choose the rule that makes the preferred release appear acceptable. A pre-specified rule does not prevent revision, but it separates a genuine policy change from a result-driven exception.
Example: a hotel booking agent
A hotel proposes an agent that can modify reservations and issue permitted refunds. Management is considering a supervised pilot in which an employee approves every refund. The existing team already completes most standard changes correctly, so the agent must reduce handling time without increasing booking errors or unauthorised refunds.
The primary outcome measure records whether the requested reservation reaches the correct final state. Process measures record tool selection, argument accuracy, duplicate calls and time to completion. Control measures record cross-guest access, refund-limit breaches and missing approval. Guest-facing quality remains relevant, but it cannot cancel a control failure.
The scorecard segments standard changes, multi-room bookings, Arabic requests and exceptions requiring approval. The team runs repeated trials because routes and tool calls can vary. A single unapproved refund blocks autonomous release, while slower performance in a rare exception may support a restricted pilot and a targeted design change.
This result is more useful than a single score of 92 per cent. It tells management that the agent may enter a supervised pilot, identifies the routes that require more evidence and states which failure prevents wider authority.
Report uncertainty and the limits of the test set
An observed pass rate is an estimate derived from the selected cases and trials. The report should state the number of distinct cases, trials per case, sampling method, grader reliability and uncertainty interval. It should also explain whether the set represents expected production frequency or deliberately over-samples difficult and high-consequence cases.
A synthetic test mix can reveal whether the agent handles a risk. It does not estimate how often that risk will occur in production unless the sampling design supports that inference. Management needs both pieces of information: capability under controlled conditions and prevalence from monitored operational data.
What Marketways delivers
Marketways begins with the proposed operating decision and translates it into outcome, process and control claims. We define the denominator, critical gates, comparison baseline, segments, repeated-trial design and uncertainty report. We also test whether graders and data support the precision claimed by each measure.
The final scorecard states what the agent has demonstrated, which observed failures block release and which conclusions remain unsupported. Management can use that evidence to reject a release, approve a restricted pilot, require remediation or expand authority under defined conditions.