How to Evaluate AI Agent Tool Use, Permissions and Side Effects
An AI agent uses tools to retrieve records and change business systems. A correct final response does not prove that the agent selected the right tool, used valid arguments, stayed within its authority or produced the intended external result. Tool evaluation must test the complete action, from the user’s request and permission check to the final business state.
A tool call proposes a business action
A model can draft an answer without changing anything outside the conversation. A tool extends that model into another system. The agent may read a customer record, send a message, change a booking, create a work order or initiate a payment. The organisation therefore assumes a different level of risk when it gives the agent access to each tool.
The evaluation team must answer five separate questions. Did the agent choose an appropriate action? Did it supply valid arguments? Was the action authorised for this user and case? Did the tool return a reliable result? Did the business system reach the intended final state? A test that combines these questions into one overall score cannot identify why an apparently successful case failed.
Define the action contract before testing
Before testing, the evaluation team must define an expected action contract for every scenario. The contract identifies the available tools, permitted resources, argument constraints, approval requirements, prohibited actions and expected side effect. It also states what the agent should do when the requested action is not permitted.
Authority depends on context. A service agent may read a booking but require approval to issue a refund above a threshold. It may update contact details for the authenticated customer but not another traveller. The test case must therefore record the user identity, role, account, resource, amount and approval state that determine the action boundary.
The application must enforce that boundary outside the model. OWASP identifies excessive functionality, permissions and autonomy as separate sources of excessive agency. A prompt that tells the model not to issue a refund cannot replace a tool permission that blocks the transaction. The evaluator should confirm both behaviours, but the enforced boundary supplies the stronger protection.
Test tool selection against the business objective
An agent can choose a valid tool that does not solve the user’s problem. For example, it may create a support ticket when the authorised workflow requires an immediate booking change. The evaluator must therefore judge tool selection against the business objective and the conditions that make each action appropriate.
The test set should include cases with one valid tool, several valid routes and no authorised action. Where several routes can produce the same acceptable outcome, the evaluator should not demand one rigid trajectory. Instead, the scoring rule should identify required evidence, prohibited steps and the final state that every acceptable route must reach.
The evaluator should also inspect unnecessary calls. Repeated searches can increase cost and latency. Contradictory writes can leave the record in an unpredictable state. An alternative tool may bypass an approval attached to the obvious route. These observations show whether the agent chose a defensible path rather than merely finding a path that completed.
The evaluation must judge the sequence as well as each call. OWASP’s current agentic guidance treats tool misuse as a distinct risk because individually permitted actions can form a harmful chain. An agent might read a confidential record through an authorised tool and then send its contents through another authorised tool. Neither call is necessarily invalid in isolation. The test must therefore compare the complete sequence with the user’s purpose, data boundary and authorised outcome.
Test arguments as business decisions
Typed arguments can still be wrong. A syntactically valid refund may use the wrong booking, customer, currency, amount or reason code. Schema validation detects format errors, but it cannot establish that the values correspond to the authorised case.
The evaluator should vary identifiers, boundaries and evidence freshness. Useful cases include two customers with similar names, an amount immediately below and above an approval limit, a changed booking version and a request that omits a required field. The agent should retrieve or ask for the missing evidence rather than invent a value.
Argument checks should also occur at the tool boundary. Microsoft’s agent safety guidance recommends type, range and path constraints, while its access guidance recommends an external policy check before execution. The trace should show which rule allowed, denied or referred the proposed call.
Change the tool response deliberately
Normal tool responses reveal little about recovery. The evaluation should deliberately return no record, stale data, conflicting records, a timeout, malformed output and partial success. Each condition creates a different obligation.
If a search returns no result, the agent may need clarification. If two systems disagree, it may need to identify the authoritative source. If a write times out, it must determine whether the action committed before it retries. The expected response follows from the business consequence, not from a generic instruction to retry.
Mock tools and sandbox databases make these conditions reproducible. The evaluator can seed an exact starting state, control the response and inspect every attempted side effect. Live production systems cannot safely provide the same control, particularly when a test involves payments, customer messages or destructive actions.
Verify the actual side effect
An API response describes what the service returned. It does not prove that the organisation’s obligation ended correctly. A refund endpoint may acknowledge a request before the ledger posts it. A booking tool may update the reservation but fail to send the required confirmation.
The evaluator must compare the authorised starting state with the actual final state. It should inspect the record, ledger, queue or work order affected by the call. It should also confirm that required downstream actions occurred and prohibited changes did not occur.
Interrupted calls require special attention. A timeout can hide a completed write, so a blind retry may duplicate the action. The test should repeat the request with the same case and idempotency key, then verify whether the system returned the earlier result or created another transaction. Where the action cannot be made idempotent, the workflow needs a status check or an authorised compensation procedure.
Example: a hotel booking refund
A guest asks an agent to cancel one room in a two-room booking and refund the refundable rate. The agent can read reservations, calculate the permitted refund, request approval above a threshold, cancel a room and submit a refund. It cannot alter another guest’s booking or refund a non-refundable charge without approval.
The evaluation creates two similar booking records and changes the target booking after the agent first reads it. It then makes the refund tool time out after the transaction commits. These conditions test identity, evidence freshness, argument selection, approval and duplicate prevention within one coherent case.
A dependable agent refreshes the booking before it acts, selects the correct room and calculates the refund from the current rate terms. If approval is required, the tool layer prevents execution until the authorised reviewer approves the exact amount and booking. After the timeout, the agent checks transaction status instead of submitting a second refund.
The final response matters, but it is not the primary evidence. The evaluator inspects the unchanged second room, the cancelled target room, one ledger entry, the recorded approval and the confirmation message. Those records establish whether the workflow completed the authorised obligation without an unintended side effect.
Preserve failures as regression cases
A confirmed failure should record the user request, identity, starting state, tool catalogue, permissions, proposed call, policy decision, tool response, trajectory and final state. This evidence allows another team to reproduce the failure and identify the control that requires revision.
After remediation, the team should rerun the failed case and neighbouring legitimate cases. A narrower permission may stop the unwanted refund but also block a valid cancellation. A stricter schema may reject a necessary exception. The regression set must therefore protect the business rule and the authorised service path.
What Marketways delivers
Marketways translates the client’s operating rules into action contracts and evaluation cases. We map tool capabilities, identities, permission scopes, approval conditions, argument rules, recovery obligations and observable final states. We then prepare mock responses and sandbox data, run repeated trials and inspect the complete trajectory.
In the report, Marketways distinguishes selection, argument, authority, recovery and side-effect failures. Confirmed failures become regression cases linked to the relevant business control. Management can then decide which tools the agent may use, which actions require approval and what evidence is required before wider deployment.