Worked Example: Evaluating a Multi-Agent Case with Persistent State
This illustrative evaluation follows one supplier-quality case across specialised agents, delayed evidence, a corrected record and an interrupted handoff. The test asks whether the complete workflow preserves ownership, authority and outstanding obligations until the business reaches a verified outcome. Plausible outputs from individual agents are insufficient.
The business must resolve one continuing case
Assume a manufacturer reports that components from a recent delivery fail inspection. An intake agent opens the case. An evidence agent retrieves the purchase order, delivery record, inspection report and supplier terms. A technical agent assesses whether the defect is substantiated. A coordinator prepares the approved disposition, such as replacement, return or further inspection.
The case may continue for several days while staff request photographs, confirm the affected batch and obtain approval. During that period, the organisation still owes the manufacturer and supplier a controlled resolution. The evaluation must therefore examine whether the workflow completes that obligation, rather than whether each specialist produces a convincing local response.
The multi-agent design needs a reason to exist
Specialised agents can keep instructions and tools focused. However, every handoff introduces another point at which the workflow can lose evidence, authority or responsibility. A multi-agent design is justified only when specialisation improves the case enough to outweigh that coordination burden.
The evaluation team should retain a simpler baseline. A single controlled workflow may be preferable if it reaches the same result with fewer handoffs and clearer ownership. Management should compare the multi-agent design with that baseline instead of assuming that more agents provide greater capability.
The case record must survive every handoff
The shared case record should identify the case owner, current status, accepted facts, source and version of each material record, pending evidence, approvals, completed actions and next obligation. A handoff should add the specialist task, the evidence available for that task, any unresolved uncertainty and the result expected in return.
The receiving agent should not receive all prior material by default. Access rules may restrict supplier pricing, employee information or legal advice to particular roles. The handoff contract must preserve enough context to complete the task while preventing one specialist's access from becoming every agent's access.
The evaluation changes facts while the case is active
The initial delivery record links the failed components to batch B17. After the technical agent begins its assessment, the warehouse corrects a scanning error and links the inspected components to batch B19. The test checks whether the workflow detects the change, withdraws conclusions based on B17 and reassesses the case against B19 evidence.
A fluent summary of the earlier record must not overrule the corrected source. The case record should retain both the earlier basis and the correction so that a reviewer can understand why the conclusion changed. If the workflow cannot determine which batch is authoritative, it should pause and request reconciliation instead of selecting the convenient answer.
A delayed approval tests ownership and resumption
Suppose the technical agent recommends rejecting the delivery and the coordinator requests commercial approval. The approver responds the following day, after revised supplier terms take effect. The workflow should resume the same case, confirm who owns it and revalidate any condition that could alter the approved disposition.
The evaluation fails if the case disappears from every active queue, if a specialist treats the delay as completion or if the coordinator applies an approval to materially changed terms without review. Persistent state must record the outstanding obligation as well as the history of completed steps.
An interrupted handoff tests duplicate action and recovery
In a separate run, the coordinator creates a supplier return request, but the acknowledgement does not reach the next agent. The workflow restarts from its latest checkpoint. Before creating another request, it should query the return system using the original case or transaction identifier.
The expected result is one return request and one continuing case. A second request reveals that the workflow preserved conversation history without preserving operational truth. Closing the case before confirming the return would reveal a different failure: the workflow remembered an attempted action but lost the obligation to verify its outcome.
Permission changes must apply to work already in progress
Assume the evidence specialist's access to supplier pricing is revoked while the case is waiting. A resumed agent should not rely on a credential or cached record merely because access existed earlier. The workflow should enforce the current permission and either obtain the required evidence through an authorised route or refer the task.
This condition tests more than login. It establishes whether persistent state preserves useful continuity without preserving obsolete authority. The case history may record that pricing informed an earlier conclusion, while the resumed agent remains unable to retrieve or disclose the restricted data.
Expected behaviour must be observable
The evaluation needs explicit expectations for allocation, handoff completeness, correction handling, resumption, permission enforcement, duplicate prevention and verified closure. These expectations should be expressed through observable records and actions. The test does not require access to private model reasoning.
Traces should identify the active agent, handoff payload, source versions, permission result, tool calls, case state and final disposition. The evaluator can then locate the step at which the workflow accepted stale evidence, lost ownership or repeated an action. A final answer alone cannot distinguish a robust case from a lucky sequence.
Findings must lead to operating-design changes
Suppose the workflow accepts the batch correction but loses the outstanding approval when the coordinator becomes unavailable. The finding is not simply that the coordinator agent failed. The operating design lacks an independent case owner or queue that preserves the obligation across agent availability.
The corrective action may add an explicit owner field, a timed escalation and a reallocation rule. The team should rerun the failed case and neighbouring cases in which the coordinator remains available or approval is rejected. The recommendation can then state whether the revised multi-agent pattern is supported or whether a simpler central coordinator should retain ownership.
How Marketways applies this worked-example approach
Marketways begins with the client's actual case lifecycle, authority boundaries and business records. We identify the decisions that justify specialist agents, define the shared state and handoff evidence, and construct controlled changes that expose stale facts, lost obligations, obsolete permissions and uncertain side effects.
The engagement produces reproducible findings, design recommendations and regression cases without publishing the private structure or selection logic of Marketways' evaluation framework. Management receives evidence about whether the workflow can own and complete a real case, not a demonstration that several agents can exchange messages.