Agentic AI Cost, Latency and Capacity: Designing the Unit Economics
Agent economics should be measured per completed business case, not per model call. Model routing, context, retrieval, tool use, iteration limits and human review jointly determine cost, speed and capacity.
Measure the complete case
An agentic workflow may use several model calls, retrieval steps, tools and approvals before the business result is complete. Token price alone cannot show whether the workflow is economical. Measure cost, elapsed time and human effort from trigger to verified completion, including failed runs, retries, corrections and monitoring.
The useful denominator is a completed and acceptable business case, such as an invoice exception resolved or a customer request closed without reopening.
Establish the current baseline
Record current volume, arrival pattern, cycle time, handling effort, queue, rework, escalation, error and outcome. Separate ordinary cases from complex exceptions. The baseline reveals where speed matters and where a small number of costly failures dominate value.
Without that distribution, a team may optimise model latency while the workflow still waits hours for a system or person.
Route work by difficulty
Use the least expensive model and path that meets the quality threshold for each case class. Deterministic code should handle stable calculations and rules. Smaller models may classify or extract routine information. Deeper reasoning should be reserved for cases where the additional quality changes the outcome.
Routing itself needs evaluation. A cheap route is costly when it sends difficult cases forward with hidden errors.
Control context and retrieval
Long instructions, extensive history and broad retrieval increase tokens and delay. Retrieve only sources relevant to the current decision, reduce duplicated tool descriptions and store workflow facts as structured state. Cache stable material where policy allows, and invalidate it when the source changes.
Context reduction must preserve critical evidence. Test quality and failure by case type before treating a shorter prompt as an efficiency improvement.
Limit agent loops and tool use
Set maximum iterations, timeouts and token budgets. Record repeated tool calls, unsuccessful searches and plans that return to the same state. A loop limit should end in a controlled escalation with the evidence collected so far, not a false completion.
Tool costs can include search, browser, code execution, memory, third-party API and transaction charges. Include these in the case economics.
Design for arrival peaks and external limits
Capacity depends on request arrival, concurrency, model quotas, tool rate limits and the duration of stateful work. Simulate peaks and unavailable dependencies. Decide which tasks may queue, which need a faster fallback and which should stop when freshness expires.
A workflow that works for ten demonstrations may fail when hundreds of cases arrive after a service interruption.
Include the cost of control
Tracing, evaluation, sandboxing, security review and human approval add cost because they make the system operable. Removing them can create apparent savings while transferring expense into incidents and correction. Measure reviewer workload and design approval thresholds so people focus on material uncertainty rather than rubber-stamping every case.
A customer-support routing example
A support agent uses a small model to classify straightforward requests and retrieve one approved procedure. Material complaints, ambiguous identity and policy exceptions route to a stronger model and a human reviewer. The system limits repeated search and stores case state outside the prompt.
The scorecard compares completion, reopening, correction, latency and cost by route. If the cheaper route increases reopened cases, the business has not saved money. If the stronger route handles every case, response time and capacity may become unacceptable. The routing threshold balances both using observed evidence.
Connect economics to the business case
The agentic AI ROI guide examines investment and value across the workflow. Cost and capacity design supplies its operating assumptions: case mix, run cost, reviewer effort, throughput, failure and growth. Revisit those assumptions after deployment because models, prices, volumes and user behaviour change.
What Marketways delivers
Marketways builds the demand and case-mix baseline, maps the cost drivers, designs routing experiments and simulates capacity under normal and peak conditions. The output is a unit-economics model, latency and capacity budget, routing policy and monitoring scorecard connected to business outcomes.