Continuous AI Agent Evaluation in Production
Pre-release tests show how an AI agent behaves under controlled conditions. Production introduces customers, data, tools, policies and timing combinations that the test team may not have anticipated. Continuous evaluation connects selected production traces with later business outcomes, investigates material failures and converts confirmed lessons into versioned regression tests.
Define what production evaluation must decide
Continuous evaluation should support named operating decisions. These decisions may include keeping the workflow live, restricting an action, reverting a release, changing a threshold or investigating a new failure pattern. Without a decision and responsible owner, monitoring can produce scores without changing the system.
The organisation should define the required evidence for each decision before collecting more data. A payment agent may require zero unauthorised transfers, while a service agent may permit minor wording variation but never false closure. The release claim, consequence and tolerance determine which traces, outcomes and reviewers matter.
Instrument the complete workflow, not only the final response
A final answer does not reveal which source the agent retrieved, which tool it called, what permission it used or whether the external system accepted the action. The production trace should record the agent and workflow version, input, retrieval, tool calls, handovers, approvals, external state and final response.
Tracing should use stable identifiers that connect events without copying unnecessary sensitive content. Access, retention and redaction should follow the organisation's privacy and security requirements. The resulting trace must allow an authorised investigator to reconstruct what the agent knew, attempted and changed.
Connect traces with delayed outcomes
Some failures appear during the interaction, such as a rejected tool call. Others appear later. A customer may reopen a case, a finance team may reverse a payment, or an employee may correct a record after the conversation has closed.
Production evaluation should join the trace with the outcome that establishes whether the task succeeded. The join requires a defined identifier, observation period and outcome owner. Conversation completion cannot substitute for service restoration, delivered fulfilment, settled payment or another business-specific result.
Sample by risk, novelty and business outcome
Reviewing every trace may be unnecessary or impractical. A random sample can estimate broad performance, but it may miss rare actions with serious consequences. Risk-based sampling should therefore include high-value actions, overrides, abstentions, errors, unusual routes, complaints and cases with adverse downstream outcomes.
The organisation should record why each trace entered the sample. It should also preserve the denominator needed to interpret rates. A targeted incident sample can reveal failure mechanisms, but its failure proportion does not estimate the rate across all production traffic.
Use automated evaluators within a measured boundary
Deterministic checks can verify schemas, permissions, tool results, approval presence and final system states. Model-based graders can support criteria that require contextual judgement, such as whether an explanation addresses the recorded failure. Each evaluator needs an explicit claim and a known decision boundary.
Before automated scores control release or triage, compare them with qualified human judgements on representative cases. Measure disagreement, investigate systematic errors and retain evaluator versions. Low-confidence, novel and high-consequence cases should reach accountable reviewers rather than inherit a convenient automated label.
Keep the evaluation collection separate from operational data
The agent may access live customer, product, transaction or policy data while performing its task. An evaluation platform may separately maintain traces, labels, expected outcomes and regression cases. These collections serve different purposes and should not be treated as one dataset.
A production trace can provide evidence and suggest a candidate test. However, the team should review the case, remove unnecessary sensitive content, reconstruct the relevant state and define the expected behaviour before adding it to the maintained evaluation collection. This step preserves provenance and prevents noisy production behaviour from becoming automatic ground truth.
Convert incidents and corrections into regression tests
An incident identifies what happened once. A regression test specifies the conditions, expected behaviour and acceptance rule needed to detect the same failure again. The test should include the relevant model, prompt, retrieval, tool, policy and external-state versions.
Run the reconstructed case against the proposed fix and adjacent supported behaviour. A narrow patch may stop one error while creating another. The release decision should therefore consider both incident resolution and regression across the established suite.
Trigger evaluation after material change
Production behaviour can change when the model, prompt, retrieval source, tool, permission, policy or workflow changes. It can also change when a third-party service updates without a planned release.
The monitoring plan should map each change class to a targeted or full evaluation suite. It should define who can approve, restrict, revert or pause the route and how the organisation maintains service while investigating. Version records make the affected cohort identifiable instead of forcing the team to interpret one blended trend.
Detect operating drift through business evidence
Input drift and score drift can indicate change, but they do not explain whether the agent still supports the business objective. The organisation should also track cycle time, rework, repeat contact, customer correction, human intervention, cost and downstream reversals where those measures apply.
Compare like case mixes and observation periods. A stable technical score can coexist with worsening business outcomes if the customer population, process or policy has changed. Conversely, a changed input distribution may be harmless when the workflow still produces verified results within its authority.
Operate a closed control loop
The complete loop connects detection, investigation, correction and release. Monitoring identifies a candidate issue. An authorised review establishes the failure and consequence. The team reconstructs the case, tests the correction, checks adjacent behaviour and updates the release evidence.
The operating record should show the alert, affected versions, reviewed traces, decision, correction, regression result and final disposition. This record supports accountability and prevents the organisation from repeatedly rediscovering the same failure without changing the workflow.
What Marketways can evaluate
Marketways can define the production evaluation plan, map trace and outcome joins, design risk-based sampling and validate automated evaluators against qualified review. We can also establish the incident-to-regression process, change triggers, release thresholds and management scorecard for the workflow's actual business objective.
The engagement can integrate with the client's chosen tracing and evaluation tools. Marketways provides evaluation design, implementation support and operating evidence. The client retains its production access, privacy, security, legal, risk and release decisions.