Data Quality and Knowledge Readiness for Agentic AI Workflows

Begin with the decision and action

The required evidence follows from the business decision. An agent that prepares a refund may need the order, fulfilment event, payment status, prior remedy and current policy. An agent that drafts a maintenance order needs asset identity, alarm history, technician eligibility and access conditions. Neither route benefits equally from every available data source.

The readiness assessment should therefore name the decision, eligible cases, authorised action and consequence of error. This scope prevents a broad data-cleaning programme from replacing the practical question: can the workflow support this action reliably now?

Define an evidence contract

For each material decision, the contract should identify required fields, authoritative sources, acceptable age, validation rules and the meaning of missing evidence. It should also distinguish facts that must come from a system of record from contextual material that may come from policies, manuals or correspondence.

The contract turns vague data quality into testable conditions. A delivery date may be syntactically valid but too old to support a same-day promise. A policy document may mention the correct product but apply to another jurisdiction. Readiness depends on fitness for this decision, not on whether a field or document exists.

Establish reliable identities and relationships

Business evidence often sits under different customer, supplier, asset or product identifiers. If the workflow joins the wrong records, it can construct a coherent case about the wrong person or object. If it refuses every imperfect match, it may leave ordinary cases unresolved.

Identity rules should define authoritative identifiers, permissible matches, confidence or rule thresholds, fallback questions and referral conditions. The evaluation must include duplicates, changed names, reused contact details, merged accounts and apparently complete but incorrect matches. These cases show whether the workflow knows when it lacks a reliable entity.

Govern authority, version and time

Policies, contracts and operating instructions can conflict because they apply to different dates, products, locations or customer groups. A retrieval system that finds semantically similar text may still select the wrong rule. Each governed source therefore needs an owner, effective period, scope and status.

Superseded material should be removed from active retrieval or marked so that the workflow cannot treat it as current. Where two active sources conflict, a documented precedence rule or human referral must resolve the case. The model should not decide organisational authority from writing style or search rank.

Prepare sources for retrieval

Documents written for people are not automatically suitable for reliable retrieval. Tables may lose headings during extraction, scanned pages may contain recognition errors and a chunk may separate an exception from the rule it qualifies. Metadata can also disappear when material moves from the source system into an index.

Preparation should preserve document identity, section structure, effective dates, access labels and links between rules and exceptions. The team should inspect representative extracted content rather than assume that successful ingestion preserved meaning. Better embeddings cannot recover a condition that the extraction process removed.

Test retrieval before judging the answer

An answer can fail because the relevant evidence is absent from the knowledge base, because retrieval misses it or because the model misuses evidence that was retrieved correctly. These causes require different remedies. Adding a larger model will not repair an omitted source or an index that returns the wrong policy version.

Evaluation should therefore measure source coverage and retrieval for questions that represent the intended route. The test should check whether the system returns the applicable source, excludes misleading material and preserves enough context to support the decision. End-to-end answer tests remain necessary, but they cannot diagnose the retrieval layer on their own.

Preserve permissions and provenance

The agent should retrieve only material that the user, case and workflow are authorised to access. Existing oversharing becomes more consequential when software can search and combine records quickly. Source permissions and tenant boundaries must continue through indexing, retrieval and any cache or trace.

For every material conclusion or action, the record should identify the source, version and retrieval time. It should distinguish retrieved evidence, deterministic calculation, model inference and human judgement. This provenance allows a reviewer to reconstruct the case and identify which source or reasoning step requires correction.

Define a route for missing or conflicting evidence

No production source remains complete and available at all times. A system can time out, a required field can be missing or two records can disagree. The workflow needs an explicit response for each condition, such as requesting information, using an approved fallback, narrowing the action, pausing or referring the case.

Silence is not a safe substitute for evidence. The agent should not fill a required fact with plausible text or select the most convenient source. Readiness includes the ability to recognise when the evidence contract is not satisfied and to stop before an unsupported action.

Apply readiness to one customer with three identities

A service agent receives a complaint from an email address linked to two customer records. The order system uses a mobile number, while the loyalty platform retains an older spelling of the customer's name. Automatically joining every record risks exposing another person's history; refusing every match prevents the agent from addressing an ordinary service problem.

The evidence contract requires an order identifier and one additional verified attribute before the agent can display order details. It permits the agent to ask a limited verification question, but it blocks loyalty history until identity is resolved. If the attributes conflict, the case moves to an authorised employee rather than allowing the model to choose the most plausible record.

The readiness test includes a corrected record, a delayed order update, a reused email address and an apparently complete false match. It also changes the applicable remedy policy between two effective dates. Success requires the workflow to identify the right customer and current policy, or to stop at the defined boundary when either condition remains unresolved.

Make a bounded readiness decision

Marketways can define the evidence contract, map sources and entities, assess task-specific quality, test extraction and retrieval, and design missing-evidence routes. We can also connect data defects with the cases, decisions and consequences they affect so that remediation follows business priority.

The resulting readiness decision identifies routes that can proceed, routes that require source or identity repair and routes that must remain outside agent authority. Management can then fund the evidence improvements that change a real workflow decision rather than pursue an undefined goal of perfect enterprise data.

Continue through the agentic workflow series

• Tools, integrations and permissions
• The workflow lifecycle
• Deployment and monitoring
• Testing and evaluation